From ghang.naoc at gmail.com Sun Oct 5 10:24:39 2014 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Sun, 5 Oct 2014 22:24:39 +0800 Subject: [AstroPy] proper motion and parallax Message-ID: Hi evryone, I have a series of images for a single target. So a series of positions can be got. How to get proper motion and parallax? The two values are coupled, right? So I have to fit them simutaneously. The apparent orbit of a nearby star should be a Cycloid, (parallax makes circular motion and pm makes uniform rectilinear motion), right? I am totally a layman. Or simplely, is thee any pythonic modules which can derive pm and parallax? Thank you. best hang -------------- next part -------------- An HTML attachment was scrubbed... URL: From npkuin at gmail.com Sun Oct 5 10:59:46 2014 From: npkuin at gmail.com (Paul Kuin) Date: Sun, 5 Oct 2014 15:59:46 +0100 Subject: [AstroPy] proper motion and parallax In-Reply-To: References: Message-ID: You can perhaps learn a bit from Peter van de Kamps work: http://adsabs.harvard.edu/cgi-bin/nph-abs_connect?db_key=AST&db_key=PRE&qform=AST&arxiv_sel=astro-ph&arxiv_sel=cond-mat&arxiv_sel=cs&arxiv_sel=gr-qc&arxiv_sel=hep-ex&arxiv_sel=hep-lat&arxiv_sel=hep-ph&arxiv_sel=hep-th&arxiv_sel=math&arxiv_sel=math-ph&arxiv_sel=nlin&arxiv_sel=nucl-ex&arxiv_sel=nucl-th&arxiv_sel=physics&arxiv_sel=quant-ph&arxiv_sel=q-bio&sim_query=YES&ned_query=YES&adsobj_query=YES&aut_logic=OR&obj_logic=OR&author=van+de+kamp%2C+p&object=&start_mon=&start_year=1950&end_mon=&end_year=1985&ttl_logic=OR&title=&txt_logic=OR&text=&nr_to_return=200&start_nr=1&jou_pick=ALL&ref_stems=&data_and=ALL&group_and=ALL&start_entry_day=&start_entry_mon=&start_entry_year=&end_entry_day=&end_entry_mon=&end_entry_year=&min_score=&sort=SCORE&data_type=SHORT&aut_syn=YES&ttl_syn=YES&txt_syn=YES&aut_wt=1.0&obj_wt=1.0&ttl_wt=0.3&txt_wt=3.0&aut_wgt=YES&obj_wgt=YES&ttl_wgt=YES&txt_wgt=YES&ttl_sco=YES&txt_sco=YES&version=1 On Sun, Oct 5, 2014 at 3:24 PM, gonghang.naoc wrote: > Hi evryone, > I have a series of images for a single target. So a series of positions > can be got. > How to get proper motion and parallax? The two values are coupled, > right? So I have to fit them simutaneously. The apparent orbit of a > nearby star should be a Cycloid, (parallax makes circular motion and pm > makes uniform rectilinear motion), right? > I am totally a layman. Or simplely, is thee any pythonic modules which can > derive pm and parallax? > > Thank you. > best > hang > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) phone +44-(0)1483 (prefix) -204927 (work) mobile +44(0)7806985366 skype ID: npkuin Mullard Space Science Laboratory ? University College London ? Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucas.guibert at viacesi.fr Mon Oct 6 14:31:53 2014 From: lucas.guibert at viacesi.fr (GUIBERT LUCAS) Date: Mon, 6 Oct 2014 18:31:53 +0000 Subject: [AstroPy] Astropy Web Message-ID: <1412620313130.66684@viacesi.fr> ?Hello everybody. I'm working on a Web Project and I need Astropy working on a web browser. I'm also using Django. My question is, after reading all this tutorial (http://www.astropy.org/astropy-tutorials/FITS-images.html), I can't have the image on my browser... I have only "AxesImage(80,48;317.44x384) ?" If someone know something... Thanks ! Lucas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ewald.zietsman at gmail.com Mon Oct 6 14:53:59 2014 From: ewald.zietsman at gmail.com (Ewald Zietsman) Date: Mon, 6 Oct 2014 20:53:59 +0200 Subject: [AstroPy] Astropy Web In-Reply-To: <1412620313130.66684@viacesi.fr> References: <1412620313130.66684@viacesi.fr> Message-ID: It looks like you're return an AxesImage object to the page template instead of an image format the browser can understand. You need to change that object into a png or jpg then return that as part of the page view. Maybe if you can share a code snippet we can help. On 06 Oct 2014, at 8:31 PM, GUIBERT LUCAS wrote: > ?Hello everybody. > > I'm working on a Web Project and I need Astropy working on a web browser. I'm also using Django. > > My question is, after reading all this tutorial (http://www.astropy.org/astropy-tutorials/FITS-images.html), I can't have the image on my browser... > > I have only "AxesImage(80,48;317.44x384) ?" > > If someone know something... > > Thanks ! > > Lucas > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Mon Oct 6 14:54:43 2014 From: stsci.perry at gmail.com (Perry Greenfield) Date: Mon, 6 Oct 2014 14:54:43 -0400 Subject: [AstroPy] Astropy Web In-Reply-To: <1412620313130.66684@viacesi.fr> References: <1412620313130.66684@viacesi.fr> Message-ID: <7BE1A5DC-B73E-4F07-9937-D48182C1BC58@gmail.com> Hi Lucas, I'm reading this as a question of how to display an image to a browser. The simple way is to use matplotlib with the Agg backend, render the images in memory, then save them as jpg files, and then render those on the web page. If you need an interactive image display on the browser, that is also possible, but more involved. See this for examples: http://matplotlib.org/faq/howto_faq.html#matplotlib-in-a-web-application-server If you mean something else, please clarify. Cheers, Perry On Oct 6, 2014, at 2:31 PM, GUIBERT LUCAS wrote: > ?Hello everybody. > > I'm working on a Web Project and I need Astropy working on a web browser. I'm also using Django. > > My question is, after reading all this tutorial (http://www.astropy.org/astropy-tutorials/FITS-images.html), I can't have the image on my browser... > > I have only "AxesImage(80,48;317.44x384) ?" > > If someone know something... > > Thanks ! > > Lucas > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From charles.figura at wartburg.edu Tue Oct 7 08:27:46 2014 From: charles.figura at wartburg.edu (Charles Figura) Date: Tue, 07 Oct 2014 07:27:46 -0500 Subject: [AstroPy] Graph titles? Message-ID: <1412684866.27756.1.camel@nightfall> Is there a provision to add image titles to fitsFigures? All I can seem to find is the 'label' function, and I was hoping for a more direct top-of-image type title. -- Charles Figura charles.figura at wartburg.edu Professor of Physics and Astronomy http://mcsp.wartburg.edu/cfigura Director, Wartburg Platte Observatory http://mcsp.wartburg.edu/observatory (319) 352-8373 (319) 352-8606 (fax) "Ut visum notitia bona id temporis." From thomas.robitaille at gmail.com Tue Oct 7 09:19:02 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Tue, 7 Oct 2014 15:19:02 +0200 Subject: [AstroPy] APLpy 0.9.13 released (includes important note about 0.9.11 and 0.9.12) Message-ID: Hi everyone, I am pleased to announce the release of APLpy 0.9.13! APLpy is a Python module that makes it easy to interactively produce publication-quality plots of astronomical images: http://aplpy.github.io/ Important Note: this version fixes a bug that was introduced in 0.9.11 and was also present in 0.9.12 which caused the pixel scale to be wrongly estimated if the WCS included a significant rotation (e.g. CROTA2) component. This could lead to beam sizes, scalebars, and overlays in general to have incorrect sizes by up to a factor of sqrt(2) in the worst case, but does not affect images with no rotation. This bug is now fixed in 0.9.13, which is therefore a highly recommended update for all users. Versions prior to 0.9.11 were not affected. I am very sorry for allowing this bug to creep in and apologize for any inconvenience. Thanks to Adam Ginsburg for providing a fix! Over the last year, APLpy has been growing a robust set of tests that will make these kind of issues much less likely in future. In particular, thanks to work by Asra Nizami, an Astropy GSoC student this summer, APLpy now has robust pixel-by-pixel image tests. In addition to this critical bug fix, this version includes a number of other minor bug fixes, and adds the ability to set the title of a figure with FITSFigure.set_title. If you use pip or easy_install, you should now be able to update to 0.9.13: pip install aplpy --upgrade Please let me know if you have any issues with this release! The best place to report any issues is via GitHub: https://github.com/aplpy/aplpy/issues Thanks, Tom From thomas.robitaille at gmail.com Tue Oct 7 09:20:32 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Tue, 7 Oct 2014 15:20:32 +0200 Subject: [AstroPy] Graph titles? In-Reply-To: <1412684866.27756.1.camel@nightfall> References: <1412684866.27756.1.camel@nightfall> Message-ID: Hi Charles, If you update to APLpy 0.9.13 you can use FITSFigure.set_title. Let me know if you run into any issues! Cheers, Tom On 7 October 2014 14:27, Charles Figura wrote: > Is there a provision to add image titles to fitsFigures? All I can seem > to find is the 'label' function, and I was hoping for a more direct > top-of-image type title. > > > -- > Charles Figura charles.figura at wartburg.edu > Professor of Physics and Astronomy http://mcsp.wartburg.edu/cfigura > Director, Wartburg Platte Observatory > http://mcsp.wartburg.edu/observatory > (319) 352-8373 > (319) 352-8606 (fax) > > "Ut visum notitia bona id temporis." > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From octavi at live.unc.edu Wed Oct 8 23:06:50 2014 From: octavi at live.unc.edu (Octavi Fors) Date: Wed, 8 Oct 2014 23:06:50 -0400 Subject: [AstroPy] question about fpack and astropy.io.fits Message-ID: Hello everyone, this might sounds like a newbie question, but do you know if fpack compression package is supported by astropy.io.fits version 0.4.2? In http://docs.astropy.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data the module "astropy.io.fits.compression" is mentioned to access compressed HDUs, but it doesn't say if fpack is supported and how should this package be specified in fits.open method. My images images are 16 and -32bit. and I would like to make use of tiling pattern (FZTILE='(n,m)', see fpack manual ). For the -32bit images, any advise about noise-sensitive scaling (SCALE_FACTOR?) and subtractive dithering to achieve high compression ratios while preserving the scientific content of data, would be most than welcome. Thanks in advance, Octavi. -- Octavi Fors Postdoctoral Research Associate Department of Physics and Astronomy The University of North Carolina at Chapel Hill CB #3255, #157 Phillips Hall Chapel Hill, NC 27599 Office: (919) 962-3606 Fax: (919) 962-0480 http://octavi.web.unc.edu/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Thu Oct 9 16:24:28 2014 From: embray at stsci.edu (Erik Bray) Date: Thu, 9 Oct 2014 16:24:28 -0400 Subject: [AstroPy] question about fpack and astropy.io.fits In-Reply-To: References: Message-ID: <5436EEFC.5000101@stsci.edu> Hi Octavi, On 10/08/2014 11:06 PM, Octavi Fors wrote: > Hello everyone, > > this might sounds like a newbie question, but do you know if fpack compression > package is supported by astropy.io.fits version 0.4.2? It depends on what you mean, exactly. It would help to add some clarification about what *fpack* is--apologies if this isn't new to you but it's worth mentioning for anyone else reading. fpack itself (and its counterpart funpack (which I always enjoy reading "fun pack")) are just command-line programs that write new FITS files with one or more of the HDUs compressed according the FITS Tile Compression convention--or decompress them as the case may be. In other words, the convention states how compressed FITS files should be read and written, while fpack is simply a program that knows how to write files with that convention. There's no sense (I can think of) in which it makes sense for a general purpose FITS library (like astropy.io.fits) to support fpack itself. It does however support the same compression scheme and can read files compressed by fpack. And likewise can write compressed files that can be read by funpack. > In > http://docs.astropy.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data > the module "astropy.io.fits.compression" is mentioned to access compressed HDUs, > but it doesn't say if fpack is supported and how should this package be > specified in fits.open method. The docs you linked to at http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data could probably be expanded on with more examples, but as they indicate files containing compressed HDUs are handled transparently. The compressed HDUs are accessed the same way as normal uncompressed IMAGE HDUs. The data is automatically decompressed when it is read. fpack has no role to play in that. > My images images are 16 and -32bit. and I would like to make use of tiling > pattern (FZTILE='(n,m)', see fpack manual > ). > For the -32bit images, any advise about noise-sensitive scaling (SCALE_FACTOR?) > and subtractive dithering to achieve high compression ratios while preserving > the scientific content of data, would be most than welcome. The FZTILE keyword, as well as the other "Fpack compression directive keywords" listed in section 3.3 of the fpack guide are not standard FITS keywords and are only understood by the fpack program itself. These can be used, for example, in a pipeline where raw data is written out uncompressed in a FITS file. The file is later processed through fpack which will compress any HDUs marked with these keywords according to the indicated settings. The end result of that is a new HDU compatible with the tile compression convention, and keywords like FZTILE become ZTILEn, or for example FZALGOR becomes ZCMPTYPE. In other words, those FZ keywords indicate how the HDU *should* be compressed when passed through fpack. The keywords on the actually compressed HDU (ZCMPTYPE, ZTILEn, etc.) indicate how the HDU *was* compressed. The FZ* keywords have no effect in astropy.io.fits (though it might not be a bad idea to create a facility that understands them, for creating compressed HDUs from existing uncompressed HDUs). Instead, to create a compressed HDU you just create a CompImageHDU object as demonstrated here: http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#creating-a-compressed-image-hdu This works the same as other HDU types in the library. Though it does take several optional keywords that are described in more detail in the API docs: http://astropy.readthedocs.org/en/stable/io/fits/api/images.html#astropy.io.fits.CompImageHDU For example to specify a tiling use >>> hdu = fits.CompImageHDU(data=data, header=header, tile_size=(n, m)) Subtractive dithering is used by default where applicable. It otherwise supports all the features of the convention except perhaps the ZMASKCMP option (not for any particular technical reason--it's definitely something that can and should be done, though I haven't had any requests for it either...) As to how to find a balance between compression quality and data quality I think others with more experience may wish to weigh in on that, though I will point out that documentation for the tiled image convention has some useful tips on that: http://fits.gsfc.nasa.gov/registry/tilecompression/tilecompression2.3.pdf Hope that helps, Erik From octavi at live.unc.edu Thu Oct 9 20:01:45 2014 From: octavi at live.unc.edu (Octavi Fors) Date: Thu, 9 Oct 2014 20:01:45 -0400 Subject: [AstroPy] question about fpack and astropy.io.fits In-Reply-To: <5436EEFC.5000101@stsci.edu> References: <5436EEFC.5000101@stsci.edu> Message-ID: Thanks a lot Erik for your detailed reply. Your answer allows me to focus my questions/concerns about astropy.io.fits and fpack in my initial message: In other words, the convention states how compressed FITS files should be > read and written, while fpack is simply a program that knows how to write > files with that convention. There's no sense (I can think of) in which it > makes sense for a general purpose FITS library (like astropy.io.fits) to > support fpack itself. It does however support the same compression scheme > and can read files compressed by fpack. And likewise can write compressed > files that can be read by funpack. > 1-by "it supports the same compression scheme" do you mean that astropy.io.fits reads/writes tile compressed FITS images in the same *exact* way (in terms of compression ratio and checksum file) as fpack does from command line (of course using the same set of parameters)? 2-my other concern is speed when reading/writing tiled-compressed FITS images. Is astropy.io.fits performance comparable to fpack command line one? Our example is 6576x4384pix images, with tiles size of 100x100pix. 3-one of our pipeline most intensive I/O operations is to extract small (500x500pix) subimages from the big ones (6576x4384pix) upon ~10^3 targets (RA,DEC) x ~10^6 exposures = ~10^9 requests. In the fpack website I've seen these two features: -Each HDU of a multi-extension FITS file is compressed separately, so it is > not necessary to uncompress the entire file to read a single image in a > multi-extension file. > > -Dividing the image into tiles before compression enables faster access to > small subsections of the image. > I know it's a basic question, but...: are the compressed tiles stored into separate HDUs? In other words, if my big (6576x4384pix) images are tile-compressed with fpack (or with astropy.io.fits if possible) and properly multi-extension HDU arranged, could astropy.io.fits (or pyfits) support the first of the two fpack features mentioned above? That would really improve the reading performance for subimage extraction process. Thanks in advance, Octavi. On Thu, Oct 9, 2014 at 4:24 PM, Erik Bray wrote: > Hi Octavi, > > On 10/08/2014 11:06 PM, Octavi Fors wrote: > > Hello everyone, > > > > this might sounds like a newbie question, but do you know if fpack > compression > > package is supported by astropy.io.fits version 0.4.2? > > It depends on what you mean, exactly. It would help to add some > clarification > about what *fpack* is--apologies if this isn't new to you but it's worth > mentioning for anyone else reading. > > fpack itself (and its counterpart funpack (which I always enjoy reading > "fun > pack")) are just command-line programs that write new FITS files with one > or > more of the HDUs compressed according the FITS Tile Compression > convention--or > decompress them as the case may be. > > In other words, the convention states how compressed FITS files should be > read > and written, while fpack is simply a program that knows how to write files > with > that convention. There's no sense (I can think of) in which it makes > sense for > a general purpose FITS library (like astropy.io.fits) to support fpack > itself. > It does however support the same compression scheme and can read files > compressed by fpack. And likewise can write compressed files that can be > read > by funpack. > > > In > > > http://docs.astropy.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data > > the module "astropy.io.fits.compression" is mentioned to access > compressed HDUs, > > but it doesn't say if fpack is supported and how should this package be > > specified in fits.open method. > > The docs you linked to at > > > http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data > > could probably be expanded on with more examples, but as they indicate > files > containing compressed HDUs are handled transparently. The compressed HDUs > are > accessed the same way as normal uncompressed IMAGE HDUs. The data is > automatically decompressed when it is read. fpack has no role to play in > that. > > > > My images images are 16 and -32bit. and I would like to make use of > tiling > > pattern (FZTILE='(n,m)', see fpack manual > > >). > > For the -32bit images, any advise about noise-sensitive scaling > (SCALE_FACTOR?) > > and subtractive dithering to achieve high compression ratios while > preserving > > the scientific content of data, would be most than welcome. > > The FZTILE keyword, as well as the other "Fpack compression directive > keywords" > listed in section 3.3 of the fpack guide are not standard FITS keywords > and are > only understood by the fpack program itself. > > These can be used, for example, in a pipeline where raw data is written out > uncompressed in a FITS file. The file is later processed through fpack > which > will compress any HDUs marked with these keywords according to the > indicated > settings. The end result of that is a new HDU compatible with the tile > compression convention, and keywords like FZTILE become ZTILEn, or for > example > FZALGOR becomes ZCMPTYPE. > > In other words, those FZ keywords indicate how the HDU *should* be > compressed > when passed through fpack. The keywords on the actually compressed HDU > (ZCMPTYPE, ZTILEn, etc.) indicate how the HDU *was* compressed. > > The FZ* keywords have no effect in astropy.io.fits (though it might not be > a bad > idea to create a facility that understands them, for creating compressed > HDUs > from existing uncompressed HDUs). Instead, to create a compressed HDU you > just > create a CompImageHDU object as demonstrated here: > > > http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#creating-a-compressed-image-hdu > > This works the same as other HDU types in the library. Though it does take > several optional keywords that are described in more detail in the API > docs: > > > http://astropy.readthedocs.org/en/stable/io/fits/api/images.html#astropy.io.fits.CompImageHDU > > For example to specify a tiling use > > >>> hdu = fits.CompImageHDU(data=data, header=header, tile_size=(n, m)) > > Subtractive dithering is used by default where applicable. It otherwise > supports all the features of the convention except perhaps the ZMASKCMP > option > (not for any particular technical reason--it's definitely something that > can and > should be done, though I haven't had any requests for it either...) > > As to how to find a balance between compression quality and data quality I > think > others with more experience may wish to weigh in on that, though I will > point > out that documentation for the tiled image convention has some useful tips > on that: > > http://fits.gsfc.nasa.gov/registry/tilecompression/tilecompression2.3.pdf > > > Hope that helps, > Erik > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Octavi Fors Postdoctoral Research Associate Department of Physics and Astronomy The University of North Carolina at Chapel Hill CB #3255, #157 Phillips Hall Chapel Hill, NC 27599 Office: (919) 962-3606 Fax: (919) 962-0480 -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Fri Oct 10 10:46:50 2014 From: embray at stsci.edu (Erik Bray) Date: Fri, 10 Oct 2014 10:46:50 -0400 Subject: [AstroPy] question about fpack and astropy.io.fits In-Reply-To: References: <5436EEFC.5000101@stsci.edu> Message-ID: <5437F15A.6010803@stsci.edu> On 10/09/2014 08:01 PM, Octavi Fors wrote: > Thanks a lot Erik for your detailed reply. > > Your answer allows me to focus my questions/concerns about astropy.io.fits and > fpack in my initial message: > > In other words, the convention states how compressed FITS files should be > read and written, while fpack is simply a program that knows how to write > files with that convention. There's no sense (I can think of) in which it > makes sense for a general purpose FITS library (like astropy.io.fits) to > support fpack itself. It does however support the same compression scheme > and can read files compressed by fpack. And likewise can write compressed > files that can be read by funpack. > > > 1-by "it supports the same compression scheme" do you mean that astropy.io.fits > reads/writes tile compressed FITS images in the same *exact* way (in terms of > compression ratio and checksum file) as fpack does from command line (of course > using the same set of parameters)? Yes, for the most part they are using the same code. The checksum should be the same for the data itself, but may not be identical for the whole file due to possible minor differences in the header, such as the order in which keywords are written, but those are trivial differences. If using the FITS checksum convention this means that the CHECKSUM *might* not be identical, but the DATASUM should be. Another gotcha to be aware of when comparing files is that when using subtractive dithering a random seed is used by default to initialize the randomizer. To avoid that for testing purposes, make sure to pass the dither_seed=-1 argument into the CompImageHDU intitializer (this is covered in more detail in the API docs I linked to). This uses a checksum of the tile data to determine the random seed, and so will always be the same for the same file. > 2-my other concern is speed when reading/writing tiled-compressed FITS images. > Is astropy.io.fits performance comparable to fpack command line one? > Our example is 6576x4384pix images, with tiles size of 100x100pix. That would be something for you determine in your particular application. > 3-one of our pipeline most intensive I/O operations is to extract small > (500x500pix) subimages from the big ones (6576x4384pix) upon ~10^3 targets > (RA,DEC) x ~10^6 exposures = ~10^9 requests. > > In the fpack website I've seen these two > features: > > -Each HDU of a multi-extension FITS file is compressed separately, so it is > not necessary to uncompress the entire file to read a single image in a > multi-extension file. > > > -Dividing the image into tiles before compression enables faster access to > small subsections of the image. > > > I know it's a basic question, but...: are the compressed tiles stored into > separate HDUs? > > In other words, if my big (6576x4384pix) images are tile-compressed with fpack > (or with astropy.io.fits if possible) and properly multi-extension HDU arranged, > could astropy.io.fits (or pyfits) support the first of the two fpack features > mentioned above? These aren't features of fpack so much as just how the Tiled Image convention works. I strongly recommend reading the documentation for the convention if doing any significant work with it (even if just using fpack to do the work): http://fits.gsfc.nasa.gov/registry/tilecompression/tilecompression2.3.pdf > That would really improve the reading performance for subimage extraction process. One thing that astropy.io.fits/PyFITS does not currently support is decompressing one tile at a time for reading slices of an image. Instead it just decompresses the entire image at once (sort of defeating the benefit of tiling to some extent, though it still helps things on the CFITSIO level). Obviously that's on the TODO list but I haven't had time to work on that feature myself. Patches are welcome. Erik P.S. on the matter of reading slices of the compressed image, the fitsio library might currently do better: https://github.com/esheldon/fitsio though I haven't tried it for that use case. > On Thu, Oct 9, 2014 at 4:24 PM, Erik Bray > wrote: > > Hi Octavi, > > On 10/08/2014 11:06 PM, Octavi Fors wrote: > > Hello everyone, > > > > this might sounds like a newbie question, but do you know if fpack compression > > package is supported by astropy.io.fits version 0.4.2? > > It depends on what you mean, exactly. It would help to add some clarification > about what *fpack* is--apologies if this isn't new to you but it's worth > mentioning for anyone else reading. > > fpack itself (and its counterpart funpack (which I always enjoy reading "fun > pack")) are just command-line programs that write new FITS files with one or > more of the HDUs compressed according the FITS Tile Compression convention--or > decompress them as the case may be. > > In other words, the convention states how compressed FITS files should be read > and written, while fpack is simply a program that knows how to write files with > that convention. There's no sense (I can think of) in which it makes sense for > a general purpose FITS library (like astropy.io.fits) to support fpack itself. > It does however support the same compression scheme and can read files > compressed by fpack. And likewise can write compressed files that can be read > by funpack. > > > In > >http://docs.astropy.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data > > the module "astropy.io.fits.compression" is mentioned to access compressed HDUs, > > but it doesn't say if fpack is supported and how should this package be > > specified in fits.open method. > > The docs you linked to at > > http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data > > could probably be expanded on with more examples, but as they indicate files > containing compressed HDUs are handled transparently. The compressed HDUs are > accessed the same way as normal uncompressed IMAGE HDUs. The data is > automatically decompressed when it is read. fpack has no role to play in that. > > > > My images images are 16 and -32bit. and I would like to make use of tiling > > pattern (FZTILE='(n,m)', see fpack manual > > ). > > For the -32bit images, any advise about noise-sensitive scaling (SCALE_FACTOR?) > > and subtractive dithering to achieve high compression ratios while preserving > > the scientific content of data, would be most than welcome. > > The FZTILE keyword, as well as the other "Fpack compression directive keywords" > listed in section 3.3 of the fpack guide are not standard FITS keywords and are > only understood by the fpack program itself. > > These can be used, for example, in a pipeline where raw data is written out > uncompressed in a FITS file. The file is later processed through fpack which > will compress any HDUs marked with these keywords according to the indicated > settings. The end result of that is a new HDU compatible with the tile > compression convention, and keywords like FZTILE become ZTILEn, or for example > FZALGOR becomes ZCMPTYPE. > > In other words, those FZ keywords indicate how the HDU *should* be compressed > when passed through fpack. The keywords on the actually compressed HDU > (ZCMPTYPE, ZTILEn, etc.) indicate how the HDU *was* compressed. > > The FZ* keywords have no effect in astropy.io.fits (though it might not be a bad > idea to create a facility that understands them, for creating compressed HDUs > from existing uncompressed HDUs). Instead, to create a compressed HDU you just > create a CompImageHDU object as demonstrated here: > > http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#creating-a-compressed-image-hdu > > This works the same as other HDU types in the library. Though it does take > several optional keywords that are described in more detail in the API docs: > > http://astropy.readthedocs.org/en/stable/io/fits/api/images.html#astropy.io.fits.CompImageHDU > > For example to specify a tiling use > > >>> hdu = fits.CompImageHDU(data=data, header=header, tile_size=(n, m)) > > Subtractive dithering is used by default where applicable. It otherwise > supports all the features of the convention except perhaps the ZMASKCMP option > (not for any particular technical reason--it's definitely something that can and > should be done, though I haven't had any requests for it either...) > > As to how to find a balance between compression quality and data quality I think > others with more experience may wish to weigh in on that, though I will point > out that documentation for the tiled image convention has some useful tips > on that: > > http://fits.gsfc.nasa.gov/registry/tilecompression/tilecompression2.3.pdf > > > Hope that helps, > Erik > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Octavi Fors > Postdoctoral Research Associate > Department of Physics and Astronomy > The University of North Carolina at Chapel Hill > CB #3255, #157 Phillips Hall > Chapel Hill, NC 27599 > Office: (919) 962-3606 > Fax: (919) 962-0480 > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From rplume at ucalgary.ca Fri Oct 10 16:30:44 2014 From: rplume at ucalgary.ca (Rene Plume) Date: Fri, 10 Oct 2014 20:30:44 +0000 Subject: [AstroPy] Convolving FITS images Message-ID: Hi! I'm fairly new to python and have been trying to figure out how to use it (using both astropy and aplpy) to do some rather standard things without too much success. For example, how would I read in a FITS image (I figured this part out), plot it in wcs coordinates, and then convolve the image with a telescope beam of a fixed angular FWHM? I then want to take a vertical slice of my image so that I have an output ascii file of (say) RA (or RA offset from some reference position) versus flux. Thanks! ########################## Ren? Plume Department of Physics & Astronomy University of Calgary Phone:(403) 220-6909 URL: www.ism.ucalgary.ca email: rplume at ucalgary.ca "I will be so brief I have already finished" - Salvador Dali in the World's shortest speech -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.g.ginsburg at gmail.com Fri Oct 10 19:11:26 2014 From: adam.g.ginsburg at gmail.com (Adam Ginsburg) Date: Sat, 11 Oct 2014 01:11:26 +0200 Subject: [AstroPy] Convolving FITS images In-Reply-To: References: Message-ID: Hi Rene, Everything you asked for can be accomplished with a few astropy utilities: from astropy.io import fits data = fits.getdata('filename.fits') header = fits.getheader('filename.fits') from astropy.convolution import convolve,Gaussian2DKernel fwhm = 5 smoothed = convolve(data, Gaussian2DKernel(stddev=fwhm/2.35) import aplpy smoothed_figure = aplpy.FITSFigure(fits.PrimaryHDU(data=smoothed, header=header)) smoothed_figure.show_grayscale() # Alternatively, display and smooth simultaneously: fig = aplpy.FITSFigure('filename.fits') fig.show_grayscale(smooth=fwhm/2.35) # but this is less flexible Vertical slicing is a little more difficult if you want to preserve world coordinate information, but doable: from astropy import wcs w = wcs.WCS(header) # Determine the approximate pixel scale, assuming square pixels [you are likely to get an important warning here] # Also, this may require the development version of astropy cd = wcs.utils.celestial_pixel_scale(w) slice = data[50,:] # grab a slice along one direction offsets = np.arange(slice) * cd # then, print to file: with open('outfile','w') as f: for position,value in zip(offsets,slice): f.write("{0:10f} {1:10f}\n".format(position,value) This is all untested, so excuse typos, but it provides at least a skeleton working example, if I understood your question. On Fri, Oct 10, 2014 at 10:30 PM, Rene Plume wrote: > Hi! > > I'm fairly new to python and have been trying to figure out how to use it > (using both astropy and aplpy) to do some rather standard things without too > much success. For example, how would I read in a FITS image (I figured this > part out), plot it in wcs coordinates, and then convolve the image with a > telescope beam of a fixed angular FWHM? I then want to take a vertical > slice of my image so that I have an output ascii file of (say) RA (or RA > offset from some reference position) versus flux. > > Thanks! > > > ########################## > Ren? Plume > Department of Physics & Astronomy > University of Calgary > > Phone:(403) 220-6909 > URL: www.ism.ucalgary.ca > email: rplume at ucalgary.ca > > "I will be so brief I have already finished" > > - Salvador Dali in the World's shortest speech > > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Adam Ginsburg Fellow, European Southern Observatory http://www.adamgginsburg.com/ From rplume at ucalgary.ca Sat Oct 11 11:32:41 2014 From: rplume at ucalgary.ca (Rene Plume) Date: Sat, 11 Oct 2014 15:32:41 +0000 Subject: [AstroPy] Convolving FITS images In-Reply-To: References: , Message-ID: <5C987141-A2A3-4D61-B130-54AD50F4DD1B@ucalgary.ca> Thanks Adam! This should prove quite helpful. I'll test it out when I get the next chance (travelling go for the next 2 weeks). Cheers! R Sent from my iPad > On Oct 10, 2014, at 5:11 PM, Adam Ginsburg wrote: > > Hi Rene, > Everything you asked for can be accomplished with a few astropy utilities: > > from astropy.io import fits > data = fits.getdata('filename.fits') > header = fits.getheader('filename.fits') > > from astropy.convolution import convolve,Gaussian2DKernel > fwhm = 5 > smoothed = convolve(data, Gaussian2DKernel(stddev=fwhm/2.35) > > import aplpy > smoothed_figure = aplpy.FITSFigure(fits.PrimaryHDU(data=smoothed, > header=header)) > smoothed_figure.show_grayscale() > > # Alternatively, display and smooth simultaneously: > fig = aplpy.FITSFigure('filename.fits') > fig.show_grayscale(smooth=fwhm/2.35) > # but this is less flexible > > > Vertical slicing is a little more difficult if you want to preserve > world coordinate information, but doable: > from astropy import wcs > w = wcs.WCS(header) > # Determine the approximate pixel scale, assuming square pixels [you > are likely to get an important warning here] > # Also, this may require the development version of astropy > cd = wcs.utils.celestial_pixel_scale(w) > slice = data[50,:] # grab a slice along one direction > offsets = np.arange(slice) * cd > > # then, print to file: > with open('outfile','w') as f: > for position,value in zip(offsets,slice): > f.write("{0:10f} {1:10f}\n".format(position,value) > > > This is all untested, so excuse typos, but it provides at least a > skeleton working example, if I understood your question. > >> On Fri, Oct 10, 2014 at 10:30 PM, Rene Plume wrote: >> Hi! >> >> I'm fairly new to python and have been trying to figure out how to use it >> (using both astropy and aplpy) to do some rather standard things without too >> much success. For example, how would I read in a FITS image (I figured this >> part out), plot it in wcs coordinates, and then convolve the image with a >> telescope beam of a fixed angular FWHM? I then want to take a vertical >> slice of my image so that I have an output ascii file of (say) RA (or RA >> offset from some reference position) versus flux. >> >> Thanks! >> >> >> ########################## >> Ren? Plume >> Department of Physics & Astronomy >> University of Calgary >> >> Phone:(403) 220-6909 >> URL: www.ism.ucalgary.ca >> email: rplume at ucalgary.ca >> >> "I will be so brief I have already finished" >> >> - Salvador Dali in the World's shortest speech >> >> >> >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Adam Ginsburg > Fellow, European Southern Observatory > http://www.adamgginsburg.com/ > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From sse at iaa.es Wed Oct 15 04:22:20 2014 From: sse at iaa.es (Susana Sanchez Exposito) Date: Wed, 15 Oct 2014 10:22:20 +0200 Subject: [AstroPy] saving tables as VOtables with missing values Message-ID: Hi all, I work on an interface where the user can view and edit tables and save them as VOtables, and for that I use the Astropy library. I keep the data of the tables in python lists, so to save them as VOtable I do this: list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')]] #transform the list of rows into list of columns list_cols= zip(*list) tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) votable=astropy.io.votable.from_table(tab) votable.to_xml("/home/sse/Desktop/test.xml") The table.Table method interprets correctly the type of each column except for the 3rd column. This column contains a "missing value" or maybe a value deleted by the user, so the whole columns is marked as "string" type, when actually is float type. I could transform all empty strings into NaN but this will be strange for those columns containing strings . I have tried to transform the python list into a masked array, but without success: problems to mask empty strings. Maybe I should to find out the type of each columns, going over the table and calculating the type of the majority of the column item, and then pass this type array to the table.Table method in some way ?? So before, to continue investigating, I would like to ask you for some tips to solve the missing values problems with astropy, or maybe even if there is a specific method for that. Thanks in advanced. Susana. -- Susana S?nchez Exp?sito Instituto de Astrof?sica de Andaluc?a IAA (CSIC) Camino Bajo de Hu?tor, 50. Granada E-18008 Tel:(+34) 958 121 311 / (+34) 958 230 618 Fax:(+34) 958 814 530 e-mail: sse at iaa.es -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Wed Oct 15 05:42:20 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 15 Oct 2014 10:42:20 +0100 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: References: Message-ID: <543E417C.6070602@gmail.com> Hi Susana, Just to simplify your current workflow, what you are doing is equivalent to: tab = Table(rows=list, names=...) tab.write('test.xml', format='votable') so no need for the zip(*) call and the call to votable.from_table. Now in terms of the masked values, I think the easiest is actually to just give some integer value to the missing value and then set the mask, so: In [19]: t = Table(rows=list, names=["A", "B", "C", "D", "E"], masked=True) In [20]: t['D'].mask[0] = True In [21]: print(t) A B C D E ---- --- --- --- --- CIG1 1.2 3.0 -- 3.0 CIG1 3.0 2.0 5.0 2.0 CIG3 2.0 3.0 2.6 nan and then write out to votable with: t.write('test.xml', format='votable') which should preserve the masks. If you chose to use a special value (e.g. np.nan) to indicate masked values, then you can do: t['D'].mask = np.isnan(t['D'].mask) Let me know if any of the above isn't clear, or if it doesn't solve your issue! Cheers, Tom Susana Sanchez Exposito wrote: > > Hi all, > > I work on an interface where the user can view and edit tables and save > them as VOtables, and for that I use the Astropy library. > > I keep the data of the tables in python lists, so to save them as > VOtable I do this: > > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, > 3,2.6, float('nan')]] > > #transform the list of rows into list of columns > list_cols= zip(*list) > > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > The table.Table method interprets correctly the type of each column > except for the 3rd column. This column contains a "missing value" or > maybe a value deleted by the user, so the whole columns is marked as > "string" type, when actually is float type. > > I could transform all empty strings into NaN but this will be strange > for those columns containing strings . > > I have tried to transform the python list into a masked array, but > without success: problems to mask empty strings. > > Maybe I should to find out the type of each columns, going over the > table and calculating the type of the majority of the column item, and > then pass this type array to the table.Table method in some way ?? > > So before, to continue investigating, I would like to ask you for some > tips to solve the missing values problems with astropy, or maybe even if > there is a specific method for that. > > Thanks in advanced. > > Susana. > > > > > > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From thomas.robitaille at gmail.com Wed Oct 15 05:42:20 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 15 Oct 2014 10:42:20 +0100 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: References: Message-ID: <543E417C.6070602@gmail.com> Hi Susana, Just to simplify your current workflow, what you are doing is equivalent to: tab = Table(rows=list, names=...) tab.write('test.xml', format='votable') so no need for the zip(*) call and the call to votable.from_table. Now in terms of the masked values, I think the easiest is actually to just give some integer value to the missing value and then set the mask, so: In [19]: t = Table(rows=list, names=["A", "B", "C", "D", "E"], masked=True) In [20]: t['D'].mask[0] = True In [21]: print(t) A B C D E ---- --- --- --- --- CIG1 1.2 3.0 -- 3.0 CIG1 3.0 2.0 5.0 2.0 CIG3 2.0 3.0 2.6 nan and then write out to votable with: t.write('test.xml', format='votable') which should preserve the masks. If you chose to use a special value (e.g. np.nan) to indicate masked values, then you can do: t['D'].mask = np.isnan(t['D'].mask) Let me know if any of the above isn't clear, or if it doesn't solve your issue! Cheers, Tom Susana Sanchez Exposito wrote: > > Hi all, > > I work on an interface where the user can view and edit tables and save > them as VOtables, and for that I use the Astropy library. > > I keep the data of the tables in python lists, so to save them as > VOtable I do this: > > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, > 3,2.6, float('nan')]] > > #transform the list of rows into list of columns > list_cols= zip(*list) > > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > The table.Table method interprets correctly the type of each column > except for the 3rd column. This column contains a "missing value" or > maybe a value deleted by the user, so the whole columns is marked as > "string" type, when actually is float type. > > I could transform all empty strings into NaN but this will be strange > for those columns containing strings . > > I have tried to transform the python list into a masked array, but > without success: problems to mask empty strings. > > Maybe I should to find out the type of each columns, going over the > table and calculating the type of the majority of the column item, and > then pass this type array to the table.Table method in some way ?? > > So before, to continue investigating, I would like to ask you for some > tips to solve the missing values problems with astropy, or maybe even if > there is a specific method for that. > > Thanks in advanced. > > Susana. > > > > > > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From sse at iaa.es Wed Oct 15 09:20:03 2014 From: sse at iaa.es (Susana Sanchez Exposito) Date: Wed, 15 Oct 2014 15:20:03 +0200 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: <543E417C.6070602@gmail.com> References: <543E417C.6070602@gmail.com> Message-ID: Thanks for your answer Tom! I have tested what you proposed, and I found two things: 1) If you replace the empty string by a value (e.g. np.nan) in order to get a mask array, this value will remain in the resulting VOTable, and in some cases it could be strange. I illustrate this with this example (note the first column of the table is the object name): list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')], ] to_nan=[] for r in list: to_nan.append([numpy.nan if (str(x).upper()=="NAN" or x=="") else x for x in r]) tab=astropy.table.Table(rows=to_nan, names=["A", "B", "C", "D", "E"], masked=True) votable=astropy.io.votable.from_table(tab) votable.to_xml("/home/sse/Desktop/test.xml") If you open the resulting VOtable (attached as test.xml) in Topcat you will see the 'nan' string in the object name column, which is very strange. Ok. I can build the mask array in other ways, BUT: 2) Even providing a mask, the column types are misinterpret. The following example illustrates what I mean: list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')] ] mask_empty=[[False, False, False, True, False], [True, False, False, False, False], [False, False, False,False, True] ] tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], masked=True) tab.mask= numpy.array (mask_empty) votable=astropy.io.votable.from_table(tab) votable.to_xml("/home/sse/Desktop/test.xml") If you open the resulting VOtable (attached as test2.xml) you will see that the 4th column is typed as char (), when the only string in the column is masked! Maybe I am doing something wrong? 2014-10-15 11:42 GMT+02:00 Thomas Robitaille : > Hi Susana, > > Just to simplify your current workflow, what you are doing is equivalent > to: > > tab = Table(rows=list, names=...) > tab.write('test.xml', format='votable') > > so no need for the zip(*) call and the call to votable.from_table. > > Now in terms of the masked values, I think the easiest is actually to > just give some integer value to the missing value and then set the mask, > so: > > In [19]: t = Table(rows=list, > names=["A", "B", "C", "D", "E"], > masked=True) > > In [20]: t['D'].mask[0] = True > > In [21]: print(t) > A B C D E > ---- --- --- --- --- > CIG1 1.2 3.0 -- 3.0 > CIG1 3.0 2.0 5.0 2.0 > CIG3 2.0 3.0 2.6 nan > > and then write out to votable with: > > t.write('test.xml', format='votable') > > which should preserve the masks. > > If you chose to use a special value (e.g. np.nan) to indicate masked > values, then you can do: > > t['D'].mask = np.isnan(t['D'].mask) > > Let me know if any of the above isn't clear, or if it doesn't solve your > issue! > > Cheers, > Tom > > Susana Sanchez Exposito wrote: > > > > Hi all, > > > > I work on an interface where the user can view and edit tables and save > > them as VOtables, and for that I use the Astropy library. > > > > I keep the data of the tables in python lists, so to save them as > > VOtable I do this: > > > > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, > > 3,2.6, float('nan')]] > > > > #transform the list of rows into list of columns > > list_cols= zip(*list) > > > > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) > > votable=astropy.io.votable.from_table(tab) > > votable.to_xml("/home/sse/Desktop/test.xml") > > > > The table.Table method interprets correctly the type of each column > > except for the 3rd column. This column contains a "missing value" or > > maybe a value deleted by the user, so the whole columns is marked as > > "string" type, when actually is float type. > > > > I could transform all empty strings into NaN but this will be strange > > for those columns containing strings . > > > > I have tried to transform the python list into a masked array, but > > without success: problems to mask empty strings. > > > > Maybe I should to find out the type of each columns, going over the > > table and calculating the type of the majority of the column item, and > > then pass this type array to the table.Table method in some way ?? > > > > So before, to continue investigating, I would like to ask you for some > > tips to solve the missing values problems with astropy, or maybe even if > > there is a specific method for that. > > > > Thanks in advanced. > > > > Susana. > > > > > > > > > > > > > > > > -- > > Susana S?nchez Exp?sito > > > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > > Camino Bajo de Hu?tor, 50. Granada E-18008 > > Tel:(+34) 958 121 311 / (+34) 958 230 618 > > Fax:(+34) 958 814 530 > > e-mail: sse at iaa.es > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Susana S?nchez Exp?sito Instituto de Astrof?sica de Andaluc?a IAA (CSIC) Camino Bajo de Hu?tor, 50. Granada E-18008 Tel:(+34) 958 121 311 / (+34) 958 230 618 Fax:(+34) 958 814 530 e-mail: sse at iaa.es -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.xml Type: text/xml Size: 1033 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test2.xml Type: text/xml Size: 1016 bytes Desc: not available URL: From sse at iaa.es Wed Oct 15 09:20:03 2014 From: sse at iaa.es (Susana Sanchez Exposito) Date: Wed, 15 Oct 2014 15:20:03 +0200 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: <543E417C.6070602@gmail.com> References: <543E417C.6070602@gmail.com> Message-ID: Thanks for your answer Tom! I have tested what you proposed, and I found two things: 1) If you replace the empty string by a value (e.g. np.nan) in order to get a mask array, this value will remain in the resulting VOTable, and in some cases it could be strange. I illustrate this with this example (note the first column of the table is the object name): list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')], ] to_nan=[] for r in list: to_nan.append([numpy.nan if (str(x).upper()=="NAN" or x=="") else x for x in r]) tab=astropy.table.Table(rows=to_nan, names=["A", "B", "C", "D", "E"], masked=True) votable=astropy.io.votable.from_table(tab) votable.to_xml("/home/sse/Desktop/test.xml") If you open the resulting VOtable (attached as test.xml) in Topcat you will see the 'nan' string in the object name column, which is very strange. Ok. I can build the mask array in other ways, BUT: 2) Even providing a mask, the column types are misinterpret. The following example illustrates what I mean: list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')] ] mask_empty=[[False, False, False, True, False], [True, False, False, False, False], [False, False, False,False, True] ] tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], masked=True) tab.mask= numpy.array (mask_empty) votable=astropy.io.votable.from_table(tab) votable.to_xml("/home/sse/Desktop/test.xml") If you open the resulting VOtable (attached as test2.xml) you will see that the 4th column is typed as char (), when the only string in the column is masked! Maybe I am doing something wrong? 2014-10-15 11:42 GMT+02:00 Thomas Robitaille : > Hi Susana, > > Just to simplify your current workflow, what you are doing is equivalent > to: > > tab = Table(rows=list, names=...) > tab.write('test.xml', format='votable') > > so no need for the zip(*) call and the call to votable.from_table. > > Now in terms of the masked values, I think the easiest is actually to > just give some integer value to the missing value and then set the mask, > so: > > In [19]: t = Table(rows=list, > names=["A", "B", "C", "D", "E"], > masked=True) > > In [20]: t['D'].mask[0] = True > > In [21]: print(t) > A B C D E > ---- --- --- --- --- > CIG1 1.2 3.0 -- 3.0 > CIG1 3.0 2.0 5.0 2.0 > CIG3 2.0 3.0 2.6 nan > > and then write out to votable with: > > t.write('test.xml', format='votable') > > which should preserve the masks. > > If you chose to use a special value (e.g. np.nan) to indicate masked > values, then you can do: > > t['D'].mask = np.isnan(t['D'].mask) > > Let me know if any of the above isn't clear, or if it doesn't solve your > issue! > > Cheers, > Tom > > Susana Sanchez Exposito wrote: > > > > Hi all, > > > > I work on an interface where the user can view and edit tables and save > > them as VOtables, and for that I use the Astropy library. > > > > I keep the data of the tables in python lists, so to save them as > > VOtable I do this: > > > > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, > > 3,2.6, float('nan')]] > > > > #transform the list of rows into list of columns > > list_cols= zip(*list) > > > > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) > > votable=astropy.io.votable.from_table(tab) > > votable.to_xml("/home/sse/Desktop/test.xml") > > > > The table.Table method interprets correctly the type of each column > > except for the 3rd column. This column contains a "missing value" or > > maybe a value deleted by the user, so the whole columns is marked as > > "string" type, when actually is float type. > > > > I could transform all empty strings into NaN but this will be strange > > for those columns containing strings . > > > > I have tried to transform the python list into a masked array, but > > without success: problems to mask empty strings. > > > > Maybe I should to find out the type of each columns, going over the > > table and calculating the type of the majority of the column item, and > > then pass this type array to the table.Table method in some way ?? > > > > So before, to continue investigating, I would like to ask you for some > > tips to solve the missing values problems with astropy, or maybe even if > > there is a specific method for that. > > > > Thanks in advanced. > > > > Susana. > > > > > > > > > > > > > > > > -- > > Susana S?nchez Exp?sito > > > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > > Camino Bajo de Hu?tor, 50. Granada E-18008 > > Tel:(+34) 958 121 311 / (+34) 958 230 618 > > Fax:(+34) 958 814 530 > > e-mail: sse at iaa.es > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Susana S?nchez Exp?sito Instituto de Astrof?sica de Andaluc?a IAA (CSIC) Camino Bajo de Hu?tor, 50. Granada E-18008 Tel:(+34) 958 121 311 / (+34) 958 230 618 Fax:(+34) 958 814 530 e-mail: sse at iaa.es -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.xml Type: text/xml Size: 1033 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test2.xml Type: text/xml Size: 1016 bytes Desc: not available URL: From mdroe at stsci.edu Wed Oct 15 12:37:40 2014 From: mdroe at stsci.edu (Michael Droettboom) Date: Wed, 15 Oct 2014 12:37:40 -0400 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: References: <543E417C.6070602@gmail.com> Message-ID: <543EA2D4.8020202@stsci.edu> On 10/15/2014 09:20 AM, Susana Sanchez Exposito wrote: > Thanks for your answer Tom! > > I have tested what you proposed, and I found two things: > > 1) If you replace the empty string by a value (e.g. np.nan) in order > to get a mask array, this value will remain in the resulting VOTable, > and in some cases it could be strange. I illustrate this with this > example (note the first column of the table is the object name): > > list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, > 3,2.6, float('nan')], ] > to_nan=[] > for r in list: > to_nan.append([numpy.nan if (str(x).upper()=="NAN" or x=="") else > x for x in r]) > > tab=astropy.table.Table(rows=to_nan, names=["A", "B", "C", "D", "E"], > masked=True) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > If you open the resulting VOtable (attached as test.xml) in Topcat you > will see the 'nan' string in the object name column, which is very > strange. The loop you provide above is putting the string ?nan?, not the floating-point value NaN in the column ? since the column is a string column, there?s really nothing else it can do. Signs point to using your approach below, with an explicit mask, for this reason. > > > Ok. I can build the mask array in other ways, BUT: > > 2) Even providing a mask, the column types are misinterpret. The > following example illustrates what I mean: > > > list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, > 3,2.6, float('nan')] ] > > mask_empty=[[False, False, False, True, False], [True, False, False, > False, False], [False, False, False,False, True] ] > tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], > masked=True) > tab.mask= numpy.array (mask_empty) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > If you open the resulting VOtable (attached as test2.xml) you will see > that the 4th column is typed as char ( datatype="char" name="D"/>), when the only string in the column is masked! > > Maybe I am doing something wrong? The determination of column types is independent of masking, unfortunately, and happens on the line |tab=astropy.table.Table(...| above before the table is even made aware of the mask. In general, the masked values need to be of the column type. But since it is masked, it shouldn?t matter what that actual value is. Mike > > > > 2014-10-15 11:42 GMT+02:00 Thomas Robitaille > >: > > Hi Susana, > > Just to simplify your current workflow, what you are doing is > equivalent to: > > tab = Table(rows=list, names=...) > tab.write('test.xml', format='votable') > > so no need for the zip(*) call and the call to votable.from_table. > > Now in terms of the masked values, I think the easiest is actually to > just give some integer value to the missing value and then set the > mask, so: > > In [19]: t = Table(rows=list, > names=["A", "B", "C", "D", "E"], > masked=True) > > In [20]: t['D'].mask[0] = True > > In [21]: print(t) > A B C D E > ---- --- --- --- --- > CIG1 1.2 3.0 -- 3.0 > CIG1 3.0 2.0 5.0 2.0 > CIG3 2.0 3.0 2.6 nan > > and then write out to votable with: > > t.write('test.xml', format='votable') > > which should preserve the masks. > > If you chose to use a special value (e.g. np.nan) to indicate masked > values, then you can do: > > t['D'].mask = np.isnan(t['D'].mask) > > Let me know if any of the above isn't clear, or if it doesn't > solve your > issue! > > Cheers, > Tom > > Susana Sanchez Exposito wrote: > > > > Hi all, > > > > I work on an interface where the user can view and edit tables > and save > > them as VOtables, and for that I use the Astropy library. > > > > I keep the data of the tables in python lists, so to save them as > > VOtable I do this: > > > > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, > > 3,2.6, float('nan')]] > > > > #transform the list of rows into list of columns > > list_cols= zip(*list) > > > > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) > > votable=astropy.io.votable.from_table(tab) > > votable.to_xml("/home/sse/Desktop/test.xml") > > > > The table.Table method interprets correctly the type of each column > > except for the 3rd column. This column contains a "missing value" or > > maybe a value deleted by the user, so the whole columns is marked as > > "string" type, when actually is float type. > > > > I could transform all empty strings into NaN but this will be > strange > > for those columns containing strings . > > > > I have tried to transform the python list into a masked array, but > > without success: problems to mask empty strings. > > > > Maybe I should to find out the type of each columns, going over the > > table and calculating the type of the majority of the column > item, and > > then pass this type array to the table.Table method in some way ?? > > > > So before, to continue investigating, I would like to ask you > for some > > tips to solve the missing values problems with astropy, or maybe > even if > > there is a specific method for that. > > > > Thanks in advanced. > > > > Susana. > > > > > > > > > > > > > > > > -- > > Susana S?nchez Exp?sito > > > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > > Camino Bajo de Hu?tor, 50. Granada E-18008 > > Tel:(+34) 958 121 311 / (+34) 958 230 618 > > Fax:(+34) 958 814 530 > > e-mail: sse at iaa.es > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy ? -- Michael Droettboom Science Software Branch Space Telescope Science Institute http://www.droettboom.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sse at iaa.es Thu Oct 16 05:39:00 2014 From: sse at iaa.es (Susana Sanchez Exposito) Date: Thu, 16 Oct 2014 11:39:00 +0200 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: <543EA2D4.8020202@stsci.edu> References: <543E417C.6070602@gmail.com> <543EA2D4.8020202@stsci.edu> Message-ID: 2014-10-15 18:37 GMT+02:00 Michael Droettboom : > On 10/15/2014 09:20 AM, Susana Sanchez Exposito wrote: > > Thanks for your answer Tom! > > I have tested what you proposed, and I found two things: > > 1) If you replace the empty string by a value (e.g. np.nan) in order to > get a mask array, this value will remain in the resulting VOTable, and in > some cases it could be strange. I illustrate this with this example (note > the first column of the table is the object name): > > list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, > float('nan')], ] > to_nan=[] > for r in list: > to_nan.append([numpy.nan if (str(x).upper()=="NAN" or x=="") else x > for x in r]) > > tab=astropy.table.Table(rows=to_nan, names=["A", "B", "C", "D", "E"], > masked=True) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > If you open the resulting VOtable (attached as test.xml) in Topcat you > will see the 'nan' string in the object name column, which is very strange. > > The loop you provide above is putting the string ?nan?, not the > floating-point value NaN in the column ? since the column is a string > column, there?s really nothing else it can do. Signs point to using your > approach below, with an explicit mask, for this reason. > mmm, I am pretty sure the loop above is putting the float value NaN. This ipython output shows that: In [2]: list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')], ] In [3]: to_nan=[] In [4]: for r in list: ...: to_nan.append([numpy.nan if (str(x).upper()=="NAN" or x=="") else x for x in r]) ...: In [5]: print to_nan [['CIG1', 1.2, 3, nan, 3], [nan, 3, 2.0, 5, 2], ['CIG3', 2, 3, 2.6, nan]] Or maybe you mean other thing ? > > > Ok. I can build the mask array in other ways, BUT: > > 2) Even providing a mask, the column types are misinterpret. The > following example illustrates what I mean: > > > list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, > float('nan')] ] > > mask_empty=[[False, False, False, True, False], [True, False, False, > False, False], [False, False, False,False, True] ] > tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], > masked=True) > tab.mask= numpy.array (mask_empty) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > If you open the resulting VOtable (attached as test2.xml) you will see > that the 4th column is typed as char ( datatype="char" name="D"/>), when the only string in the column is masked! > > Maybe I am doing something wrong? > > The determination of column types is independent of masking, > unfortunately, and happens on the line tab=astropy.table.Table(... above > before the table is even made aware of the mask. In general, the masked > values need to be of the column type. But since it is masked, it shouldn?t > matter what that actual value is. > Ok.This is what I was imaging. Thanks for your answer, Mike! > Mike > > > > > 2014-10-15 11:42 GMT+02:00 Thomas Robitaille > : > >> Hi Susana, >> >> Just to simplify your current workflow, what you are doing is equivalent >> to: >> >> tab = Table(rows=list, names=...) >> tab.write('test.xml', format='votable') >> >> so no need for the zip(*) call and the call to votable.from_table. >> >> Now in terms of the masked values, I think the easiest is actually to >> just give some integer value to the missing value and then set the mask, >> so: >> >> In [19]: t = Table(rows=list, >> names=["A", "B", "C", "D", "E"], >> masked=True) >> >> In [20]: t['D'].mask[0] = True >> >> In [21]: print(t) >> A B C D E >> ---- --- --- --- --- >> CIG1 1.2 3.0 -- 3.0 >> CIG1 3.0 2.0 5.0 2.0 >> CIG3 2.0 3.0 2.6 nan >> >> and then write out to votable with: >> >> t.write('test.xml', format='votable') >> >> which should preserve the masks. >> >> If you chose to use a special value (e.g. np.nan) to indicate masked >> values, then you can do: >> >> t['D'].mask = np.isnan(t['D'].mask) >> >> Let me know if any of the above isn't clear, or if it doesn't solve your >> issue! >> >> Cheers, >> Tom >> >> Susana Sanchez Exposito wrote: >> > >> > Hi all, >> > >> > I work on an interface where the user can view and edit tables and save >> > them as VOtables, and for that I use the Astropy library. >> > >> > I keep the data of the tables in python lists, so to save them as >> > VOtable I do this: >> > >> > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, >> > 3,2.6, float('nan')]] >> > >> > #transform the list of rows into list of columns >> > list_cols= zip(*list) >> > >> > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) >> > votable=astropy.io.votable.from_table(tab) >> > votable.to_xml("/home/sse/Desktop/test.xml") >> > >> > The table.Table method interprets correctly the type of each column >> > except for the 3rd column. This column contains a "missing value" or >> > maybe a value deleted by the user, so the whole columns is marked as >> > "string" type, when actually is float type. >> > >> > I could transform all empty strings into NaN but this will be strange >> > for those columns containing strings . >> > >> > I have tried to transform the python list into a masked array, but >> > without success: problems to mask empty strings. >> > >> > Maybe I should to find out the type of each columns, going over the >> > table and calculating the type of the majority of the column item, and >> > then pass this type array to the table.Table method in some way ?? >> > >> > So before, to continue investigating, I would like to ask you for some >> > tips to solve the missing values problems with astropy, or maybe even if >> > there is a specific method for that. >> > >> > Thanks in advanced. >> > >> > Susana. >> > >> > >> > >> > >> > >> > >> > >> > -- >> > Susana S?nchez Exp?sito >> > >> > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) >> > Camino Bajo de Hu?tor, 50. Granada E-18008 >> > Tel:(+34) 958 121 311 / (+34) 958 230 618 >> > Fax:(+34) 958 814 530 >> > e-mail: sse at iaa.es >> > >> > _______________________________________________ >> > AstroPy mailing list >> > AstroPy at scipy.org >> > http://mail.scipy.org/mailman/listinfo/astropy >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > > > _______________________________________________ > AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy > > ? > > -- > Michael Droettboom > Science Software Branch > Space Telescope Science Institute > http://www.droettboom.com > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- Susana S?nchez Exp?sito Instituto de Astrof?sica de Andaluc?a IAA (CSIC) Camino Bajo de Hu?tor, 50. Granada E-18008 Tel:(+34) 958 121 311 / (+34) 958 230 618 Fax:(+34) 958 814 530 e-mail: sse at iaa.es -------------- next part -------------- An HTML attachment was scrubbed... URL: From sse at iaa.es Thu Oct 16 05:49:42 2014 From: sse at iaa.es (Susana Sanchez Exposito) Date: Thu, 16 Oct 2014 11:49:42 +0200 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: References: <543E417C.6070602@gmail.com> <543EA2D4.8020202@stsci.edu> Message-ID: Hi again, I tried to use None,which in python it is the best representation of a missing value, and the result is the astropy.table.Table method considers None as the string "None" list = [["CIG1", 1.2, 3, None, 3], [None, 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')] ] tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], masked=True) votable=astropy.io.votable.from_table(tab) votable.to_xml("/home/sse/Desktop/test.xml") And the VOTable: 2014-10-16 11:39 GMT+02:00 Susana Sanchez Exposito : > > > 2014-10-15 18:37 GMT+02:00 Michael Droettboom : > >> On 10/15/2014 09:20 AM, Susana Sanchez Exposito wrote: >> >> Thanks for your answer Tom! >> >> I have tested what you proposed, and I found two things: >> >> 1) If you replace the empty string by a value (e.g. np.nan) in order to >> get a mask array, this value will remain in the resulting VOTable, and in >> some cases it could be strange. I illustrate this with this example (note >> the first column of the table is the object name): >> >> list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, >> float('nan')], ] >> to_nan=[] >> for r in list: >> to_nan.append([numpy.nan if (str(x).upper()=="NAN" or x=="") else x >> for x in r]) >> >> tab=astropy.table.Table(rows=to_nan, names=["A", "B", "C", "D", "E"], >> masked=True) >> votable=astropy.io.votable.from_table(tab) >> votable.to_xml("/home/sse/Desktop/test.xml") >> >> If you open the resulting VOtable (attached as test.xml) in Topcat you >> will see the 'nan' string in the object name column, which is very strange. >> >> The loop you provide above is putting the string ?nan?, not the >> floating-point value NaN in the column ? since the column is a string >> column, there?s really nothing else it can do. Signs point to using your >> approach below, with an explicit mask, for this reason. >> > mmm, I am pretty sure the loop above is putting the float value NaN. This > ipython output shows that: > > > In [2]: list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, > 3,2.6, float('nan')], ] > In [3]: to_nan=[] > In [4]: for r in list: > ...: to_nan.append([numpy.nan if (str(x).upper()=="NAN" or > x=="") else x for x in r]) > ...: > In [5]: print to_nan > [['CIG1', 1.2, 3, nan, 3], [nan, 3, 2.0, 5, 2], ['CIG3', 2, 3, 2.6, nan]] > > Or maybe you mean other thing ? > > > >> >> >> Ok. I can build the mask array in other ways, BUT: >> >> 2) Even providing a mask, the column types are misinterpret. The >> following example illustrates what I mean: >> >> >> list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, >> float('nan')] ] >> >> mask_empty=[[False, False, False, True, False], [True, False, False, >> False, False], [False, False, False,False, True] ] >> tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], >> masked=True) >> tab.mask= numpy.array (mask_empty) >> votable=astropy.io.votable.from_table(tab) >> votable.to_xml("/home/sse/Desktop/test.xml") >> >> If you open the resulting VOtable (attached as test2.xml) you will see >> that the 4th column is typed as char (> datatype="char" name="D"/>), when the only string in the column is masked! >> >> Maybe I am doing something wrong? >> >> The determination of column types is independent of masking, >> unfortunately, and happens on the line tab=astropy.table.Table(... above >> before the table is even made aware of the mask. In general, the masked >> values need to be of the column type. But since it is masked, it shouldn?t >> matter what that actual value is. >> > Ok.This is what I was imaging. > > Thanks for your answer, Mike! > > >> Mike >> >> >> >> >> 2014-10-15 11:42 GMT+02:00 Thomas Robitaille > >: >> >>> Hi Susana, >>> >>> Just to simplify your current workflow, what you are doing is equivalent >>> to: >>> >>> tab = Table(rows=list, names=...) >>> tab.write('test.xml', format='votable') >>> >>> so no need for the zip(*) call and the call to votable.from_table. >>> >>> Now in terms of the masked values, I think the easiest is actually to >>> just give some integer value to the missing value and then set the mask, >>> so: >>> >>> In [19]: t = Table(rows=list, >>> names=["A", "B", "C", "D", "E"], >>> masked=True) >>> >>> In [20]: t['D'].mask[0] = True >>> >>> In [21]: print(t) >>> A B C D E >>> ---- --- --- --- --- >>> CIG1 1.2 3.0 -- 3.0 >>> CIG1 3.0 2.0 5.0 2.0 >>> CIG3 2.0 3.0 2.6 nan >>> >>> and then write out to votable with: >>> >>> t.write('test.xml', format='votable') >>> >>> which should preserve the masks. >>> >>> If you chose to use a special value (e.g. np.nan) to indicate masked >>> values, then you can do: >>> >>> t['D'].mask = np.isnan(t['D'].mask) >>> >>> Let me know if any of the above isn't clear, or if it doesn't solve your >>> issue! >>> >>> Cheers, >>> Tom >>> >>> Susana Sanchez Exposito wrote: >>> > >>> > Hi all, >>> > >>> > I work on an interface where the user can view and edit tables and save >>> > them as VOtables, and for that I use the Astropy library. >>> > >>> > I keep the data of the tables in python lists, so to save them as >>> > VOtable I do this: >>> > >>> > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2, >>> > 3,2.6, float('nan')]] >>> > >>> > #transform the list of rows into list of columns >>> > list_cols= zip(*list) >>> > >>> > tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"]) >>> > votable=astropy.io.votable.from_table(tab) >>> > votable.to_xml("/home/sse/Desktop/test.xml") >>> > >>> > The table.Table method interprets correctly the type of each column >>> > except for the 3rd column. This column contains a "missing value" or >>> > maybe a value deleted by the user, so the whole columns is marked as >>> > "string" type, when actually is float type. >>> > >>> > I could transform all empty strings into NaN but this will be strange >>> > for those columns containing strings . >>> > >>> > I have tried to transform the python list into a masked array, but >>> > without success: problems to mask empty strings. >>> > >>> > Maybe I should to find out the type of each columns, going over the >>> > table and calculating the type of the majority of the column item, and >>> > then pass this type array to the table.Table method in some way ?? >>> > >>> > So before, to continue investigating, I would like to ask you for some >>> > tips to solve the missing values problems with astropy, or maybe even >>> if >>> > there is a specific method for that. >>> > >>> > Thanks in advanced. >>> > >>> > Susana. >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > -- >>> > Susana S?nchez Exp?sito >>> > >>> > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) >>> > Camino Bajo de Hu?tor, 50. Granada E-18008 >>> > Tel:(+34) 958 121 311 / (+34) 958 230 618 >>> > Fax:(+34) 958 814 530 >>> > e-mail: sse at iaa.es >>> > >>> > _______________________________________________ >>> > AstroPy mailing list >>> > AstroPy at scipy.org >>> > http://mail.scipy.org/mailman/listinfo/astropy >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >> >> >> >> -- >> Susana S?nchez Exp?sito >> >> Instituto de Astrof?sica de Andaluc?a IAA (CSIC) >> Camino Bajo de Hu?tor, 50. Granada E-18008 >> Tel:(+34) 958 121 311 / (+34) 958 230 618 >> Fax:(+34) 958 814 530 >> e-mail: sse at iaa.es >> >> >> _______________________________________________ >> AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy >> >> ? >> >> -- >> Michael Droettboom >> Science Software Branch >> Space Telescope Science Institute >> http://www.droettboom.com >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > -- Susana S?nchez Exp?sito Instituto de Astrof?sica de Andaluc?a IAA (CSIC) Camino Bajo de Hu?tor, 50. Granada E-18008 Tel:(+34) 958 121 311 / (+34) 958 230 618 Fax:(+34) 958 814 530 e-mail: sse at iaa.es -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Oct 16 06:24:03 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 16 Oct 2014 12:24:03 +0200 Subject: [AstroPy] saving tables as VOtables with missing values In-Reply-To: References: <543E417C.6070602@gmail.com> <543EA2D4.8020202@stsci.edu> Message-ID: <543F9CC3.9070400@gmail.com> Hi Susana, Susana Sanchez Exposito wrote: > Hi again, > I tried to use None,which in python it is the best representation of a > missing value, and the result is the astropy.table.Table method > considers None as the string "None" Indeed - at the moment the only way to get the behavior you want is to use a value with the same type as the column for the 'null' values, so for example if you want the column to be an integer column, you should have only integer values in that column (so no None, empty string, or not even nan since that is floating-point). For floating point columns you can use nan but not None or not empty strings. As Mike said, provided you set the mask correctly, the values you use don't matter. For example: from astropy.table import Table list = [["CIG1", 1.2, 3, 0, 3], ["CIG2", 3, 2.0, 5, 2], ["CIG3", 2, 3,2.6, float('nan')] ] tab=Table(rows=list, names=["A", "B", "C", "D", "E"], masked=True) tab['B'].mask[1] = True tab['D'].mask[0] = True tab.write('test.xml', format='votable') produces:
CIG1 1.2 3 None 3
None 3 2 5 2
CIG3 2 3 2.6 NaN
CIG1 1.2 3 3
CIG2 2 5 2
CIG3 2 3 2.6000000000000001 NaN
As you can see, the masked entries are save as: Now we could consider recognizing None as a masked value, but this could be tricky and slow things down a lot. Maybe Tom Aldcroft can comment on this? Cheers, Tom > > list = [["CIG1", 1.2, 3, None, 3], [None, 3, 2.0, 5, 2], ["CIG3", 2, > 3,2.6, float('nan')] ] > tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", "E"], > masked=True) > votable=astropy.io.votable.from_table(tab) > votable.to_xml("/home/sse/Desktop/test.xml") > > And the VOTable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2014-10-16 11:39 GMT+02:00 Susana Sanchez Exposito >: > > > > 2014-10-15 18:37 GMT+02:00 Michael Droettboom >: > > On 10/15/2014 09:20 AM, Susana Sanchez Exposito wrote: > >> Thanks for your answer Tom! >> >> I have tested what you proposed, and I found two things: >> >> 1) If you replace the empty string by a value (e.g. np.nan) in >> order to get a mask array, this value will remain in the >> resulting VOTable, and in some cases it could be strange. I >> illustrate this with this example (note the first column of >> the table is the object name): >> >> list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", >> 2, 3,2.6, float('nan')], ] >> to_nan=[] >> for r in list: >> to_nan.append([numpy.nan if (str(x).upper()=="NAN" or >> x=="") else x for x in r]) >> >> tab=astropy.table.Table(rows=to_nan, names=["A", "B", "C", >> "D", "E"], masked=True) >> votable=astropy.io.votable.from_table(tab) >> votable.to_xml("/home/sse/Desktop/test.xml") >> >> If you open the resulting VOtable (attached as test.xml) in >> Topcat you will see the 'nan' string in the object name >> column, which is very strange. > > The loop you provide above is putting the string ?nan?, not the > floating-point value NaN in the column ? since the column is a > string column, there?s really nothing else it can do. Signs > point to using your approach below, with an explicit mask, for > this reason. > > mmm, I am pretty sure the loop above is putting the float value NaN. > This ipython output shows that: > > > In [2]: list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], > ["CIG3", 2, 3,2.6, float('nan')], ] > In [3]: to_nan=[] > In [4]: for r in list: > ...: to_nan.append([numpy.nan if (str(x).upper()=="NAN" > or x=="") else x for x in r]) > ...: > In [5]: print to_nan > [['CIG1', 1.2, 3, nan, 3], [nan, 3, 2.0, 5, 2], ['CIG3', 2, 3, 2.6, > nan]] > > Or maybe you mean other thing ? > > > >> >> >> Ok. I can build the mask array in other ways, BUT: >> >> 2) Even providing a mask, the column types are misinterpret. >> The following example illustrates what I mean: >> >> >> list = [["CIG1", 1.2, 3, "", 3], ["", 3, 2.0, 5, 2], ["CIG3", >> 2, 3,2.6, float('nan')] ] >> >> mask_empty=[[False, False, False, True, False], [True, False, >> False, False, False], [False, False, False,False, True] ] >> tab=astropy.table.Table(rows=list, names=["A", "B", "C", "D", >> "E"], masked=True) >> tab.mask= numpy.array (mask_empty) >> votable=astropy.io.votable.from_table(tab) >> votable.to_xml("/home/sse/Desktop/test.xml") >> >> If you open the resulting VOtable (attached as test2.xml) you >> will see that the 4th column is typed as char (> arraysize="3" datatype="char" name="D"/>), when the only >> string in the column is masked! >> >> Maybe I am doing something wrong? > > The determination of column types is independent of masking, > unfortunately, and happens on the line > |tab=astropy.table.Table(...| above before the table is even > made aware of the mask. In general, the masked values need to be > of the column type. But since it is masked, it shouldn?t matter > what that actual value is. > > Ok.This is what I was imaging. > > Thanks for your answer, Mike! > > > Mike > >> >> >> >> 2014-10-15 11:42 GMT+02:00 Thomas Robitaille >> > >: >> >> Hi Susana, >> >> Just to simplify your current workflow, what you are doing >> is equivalent to: >> >> tab = Table(rows=list, names=...) >> tab.write('test.xml', format='votable') >> >> so no need for the zip(*) call and the call to >> votable.from_table. >> >> Now in terms of the masked values, I think the easiest is >> actually to >> just give some integer value to the missing value and then >> set the mask, so: >> >> In [19]: t = Table(rows=list, >> names=["A", "B", "C", "D", "E"], >> masked=True) >> >> In [20]: t['D'].mask[0] = True >> >> In [21]: print(t) >> A B C D E >> ---- --- --- --- --- >> CIG1 1.2 3.0 -- 3.0 >> CIG1 3.0 2.0 5.0 2.0 >> CIG3 2.0 3.0 2.6 nan >> >> and then write out to votable with: >> >> t.write('test.xml', format='votable') >> >> which should preserve the masks. >> >> If you chose to use a special value (e.g. np.nan) to >> indicate masked >> values, then you can do: >> >> t['D'].mask = np.isnan(t['D'].mask) >> >> Let me know if any of the above isn't clear, or if it >> doesn't solve your >> issue! >> >> Cheers, >> Tom >> >> Susana Sanchez Exposito wrote: >> > >> > Hi all, >> > >> > I work on an interface where the user can view and edit >> tables and save >> > them as VOtables, and for that I use the Astropy library. >> > >> > I keep the data of the tables in python lists, so to >> save them as >> > VOtable I do this: >> > >> > list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], >> ["CIG3", 2, >> > 3,2.6, float('nan')]] >> > >> > #transform the list of rows into list of columns >> > list_cols= zip(*list) >> > >> > tab=astropy.table.Table(list_cols, names=["A", "B", "C", >> "D", "E"]) >> > votable=astropy.io.votable.from_table(tab) >> > votable.to_xml("/home/sse/Desktop/test.xml") >> > >> > The table.Table method interprets correctly the type of >> each column >> > except for the 3rd column. This column contains a >> "missing value" or >> > maybe a value deleted by the user, so the whole columns >> is marked as >> > "string" type, when actually is float type. >> > >> > I could transform all empty strings into NaN but this >> will be strange >> > for those columns containing strings . >> > >> > I have tried to transform the python list into a masked >> array, but >> > without success: problems to mask empty strings. >> > >> > Maybe I should to find out the type of each columns, >> going over the >> > table and calculating the type of the majority of the >> column item, and >> > then pass this type array to the table.Table method in >> some way ?? >> > >> > So before, to continue investigating, I would like to >> ask you for some >> > tips to solve the missing values problems with astropy, >> or maybe even if >> > there is a specific method for that. >> > >> > Thanks in advanced. >> > >> > Susana. >> > >> > >> > >> > >> > >> > >> > >> > -- >> > Susana S?nchez Exp?sito >> > >> > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) >> > Camino Bajo de Hu?tor, 50. Granada E-18008 >> > Tel:(+34) 958 121 311 >> / (+34) 958 230 618 >> >> > Fax:(+34) 958 814 530 >> > e-mail: sse at iaa.es >> > >> > >> > _______________________________________________ >> > AstroPy mailing list >> > AstroPy at scipy.org >> > http://mail.scipy.org/mailman/listinfo/astropy >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> >> -- >> Susana S?nchez Exp?sito >> >> Instituto de Astrof?sica de Andaluc?a IAA (CSIC) >> Camino Bajo de Hu?tor, 50. Granada E-18008 >> Tel:(+34) 958 121 311 / >> (+34) 958 230 618 >> Fax:(+34) 958 814 530 >> e-mail: sse at iaa.es >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > ? > > -- > Michael Droettboom > Science Software Branch > Space Telescope Science Institute > > http://www.droettboom.com > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) > 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > > > > > -- > Susana S?nchez Exp?sito > > Instituto de Astrof?sica de Andaluc?a IAA (CSIC) > Camino Bajo de Hu?tor, 50. Granada E-18008 > Tel:(+34) 958 121 311 / (+34) 958 230 618 > Fax:(+34) 958 814 530 > e-mail: sse at iaa.es > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From bnordgren at fs.fed.us Fri Oct 17 19:45:17 2014 From: bnordgren at fs.fed.us (Nordgren, Bryce L -FS) Date: Fri, 17 Oct 2014 23:45:17 +0000 Subject: [AstroPy] Rise/set/transit times Message-ID: <82E7C9A01FD0764CACDD35D10F5DFB6E73816E@001FSN2MPN1-045.001f.mgd2.msft.net> I made an Ipython notebook to walk through Example 15a in "Astronomical Algorithms", 2nd ed. [1] This was a debugging tool for me, and I hope it's interesting to you too. I try to do things one step at a time and compare against the "book answer" at each stage. The process does expose some ambiguities in the Astronomical Algorithms text, at least to a non-astronomer like me. This mostly involves what units should be assigned to the arbitrary constants. Could I pick your brains on this topic? Assuming you said "yes", there are a number of places where Meeus converts between fractional days and degrees. I've been trying to make his "fractional day" variables work out to "u.sday", converting to degrees where instructed by multiplying by 360 u.deg/u.sday. All is well and good until we hit "input block 21" on my notebook. He then wants to multiply his fractional day variable by 360.985647. I tried assigning the units u.deg/u.day, but then my sidereal time (theta0 in notebook) is off by a degree. I'm guessing that as long as theta0 is OK, all the subsequent calculations will be OK and the unit system will convert hourangle to real time for me. However, this seems an awful lot like subverting the unit system. Anyone have any ideas what Meeus is doing, and maybe how I could express it better using astropy.units? Thanks, Bryce [1] http://nbviewer.ipython.org/github/firelab/met_utils/blob/master/tests/ex15a.ipynb This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rplume at ucalgary.ca Mon Oct 20 06:26:47 2014 From: rplume at ucalgary.ca (Rene Plume) Date: Mon, 20 Oct 2014 10:26:47 +0000 Subject: [AstroPy] Convolving FITS images In-Reply-To: References: Message-ID: <179E5F0C-600B-47BC-8F5A-EB15B4AD1E1F@ucalgary.ca> Hi Adam et al. When I use the following command sequence to convolve my data: from astropy.io import fits data = fits.getdata('filename.fits') header = fits.getheader('filename.fits') from astropy.convolution import convolve,Gaussian2DKernel fwhm = 5 smoothed = convolve(data, Gaussian2DKernel(stddev=fwhm/2.35)) I get the following error messages: # Check that the number of dimensions is compatible 144 if array_internal.ndim != kernel_internal.ndim: --> 145 raise Exception('array and kernel have differing number of ' 146 'dimensions.') 147 Exception: array and kernel have differing number of dimensions. When I use the next two commands, however, I get: # Alternatively, display and smooth simultaneously: fig = aplpy.FITSFigure('filename.fits') fig.show_grayscale(smooth=fwhm/2.35) # but this is less flexible TypeError: x_size should be an integer But when I put in an actual integer (say 3), these two steps work just fine. Any thoughts? -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.g.ginsburg at gmail.com Mon Oct 20 06:35:51 2014 From: adam.g.ginsburg at gmail.com (Adam Ginsburg) Date: Mon, 20 Oct 2014 12:35:51 +0200 Subject: [AstroPy] Convolving FITS images In-Reply-To: <179E5F0C-600B-47BC-8F5A-EB15B4AD1E1F@ucalgary.ca> References: <179E5F0C-600B-47BC-8F5A-EB15B4AD1E1F@ucalgary.ca> Message-ID: It seems that you have a data cube or some non-two-dimensional FITS file. If that's unexpected, try: data = fits.getdata('filename.fits').squeeze() which will remove any length-1 dimensions. But you should examine your data (look at data.shape) for the dimensionality issue. As for the other issue, the most recent version of aplpy gives a more helpful message and tells you that you should directly pass in a kernel with the kernel= keyword (i.e., pass in the Gaussian2DKernel to aplpy directly). So, it was my mistake originally - the `smooth` keyword in aplpy should be limited to integers. On Mon, Oct 20, 2014 at 12:26 PM, Rene Plume wrote: > Hi Adam et al. > > When I use the following command sequence to convolve my data: > > > from astropy.io import fits > data = fits.getdata('filename.fits') > header = fits.getheader('filename.fits') > > from astropy.convolution import convolve,Gaussian2DKernel > fwhm = 5 > smoothed = convolve(data, Gaussian2DKernel(stddev=fwhm/2.35)) > > > I get the following error messages: > # Check that the number of dimensions is compatible > 144 if array_internal.ndim != kernel_internal.ndim: > --> 145 raise Exception('array and kernel have differing number of ' > 146 'dimensions.') > 147 > > Exception: array and kernel have differing number of dimensions. > > > > When I use the next two commands, however, I get: > > # Alternatively, display and smooth simultaneously: > fig = aplpy.FITSFigure('filename.fits') > fig.show_grayscale(smooth=fwhm/2.35) > # but this is less flexible > > > TypeError: x_size should be an integer > > But when I put in an actual integer (say 3), these two steps work just fine. > > Any thoughts? > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Adam Ginsburg Fellow, European Southern Observatory http://www.adamgginsburg.com/ From jslavin at cfa.harvard.edu Tue Oct 21 11:48:17 2014 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Tue, 21 Oct 2014 11:48:17 -0400 Subject: [AstroPy] pyregion and astropy -- differences of pywcs and astropy.wcs Message-ID: Hi, I've run into an issue with using pyregion that seems to be related to differences in the API for pywcs vs. astropy.wcs. This may be discussed somewhere, but I didn't run across it. The problem is that pyregion wants to invoke a wcs_sky2pix method for an astropy.wcs.wcs.WCS object. Apparently pywcs.WCS objects have this method. So I'm wondering, is the pywcs wcs_sky2pix method the same as the astropy.wcs method wcs_world2pix? If so, then the fix to pyregion is easy. Regards, Jon -- ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jslavin at cfa.harvard.edu Tue Oct 21 13:05:50 2014 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Tue, 21 Oct 2014 13:05:50 -0400 Subject: [AstroPy] pyregion and astropy -- differences of pywcs and astropy.wcs Message-ID: ?This problem has been fixed in the most recent version of pyregion on github, but somehow that version was not correctly copied to pypi -- so my pip install of it had the old version of wcs_helper.py despite the version no. indicating it was up-to-date. Jon? On Tue, Oct 21, 2014 at 1:00 PM, wrote: > Hi, > > I've run into an issue with using pyregion that seems to be related to > differences in the API for pywcs vs. astropy.wcs. This may be discussed > somewhere, but I didn't run across it. The problem is that pyregion wants > to invoke a wcs_sky2pix method for an astropy.wcs.wcs.WCS object. > Apparently pywcs.WCS objects have this method. So I'm wondering, is the > pywcs wcs_sky2pix method the same as the astropy.wcs method wcs_world2pix? > If so, then the fix to pyregion is easy. > > Regards, > Jon > -- ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From C.Frohmaier at soton.ac.uk Wed Oct 22 09:23:45 2014 From: C.Frohmaier at soton.ac.uk (frohmaier c. (cf5g09)) Date: Wed, 22 Oct 2014 13:23:45 +0000 Subject: [AstroPy] erfa_time astropy.time Message-ID: Hi, I?ve recently installed astropy to my local area on a HPC in the UK (Iridis4). I need to use the fits and time capabilities of astropy. I can use fits (from astropy.io import fits) However, when i try to use time: from astropy.time import Time I get this error message: from astropy.time import Time Traceback (most recent call last): File "", line 1, in File "/home/cf5g09/.local/lib/python2.7/site-packages/astropy-0.4.2-py2.7-linux-x86_64.egg/astropy/time/__init__.py", line 2, in from .core import * File "/home/cf5g09/.local/lib/python2.7/site-packages/astropy-0.4.2-py2.7-linux-x86_64.egg/astropy/time/core.py", line 42, in from . import erfa_time ImportError: /home/cf5g09/.local/lib/python2.7/site-packages/astropy-0.4.2-py2.7-linux-x86_64.egg/astropy/time/erfa_time.so: undefined symbol: __svml_sin2 Could someone explain to me how to fix this error or how I can reinstall astropy so that Time is usable? Cheers, Chris From embray at stsci.edu Wed Oct 22 09:46:37 2014 From: embray at stsci.edu (Erik Bray) Date: Wed, 22 Oct 2014 09:46:37 -0400 Subject: [AstroPy] erfa_time astropy.time In-Reply-To: References: Message-ID: <5447B53D.4040507@stsci.edu> On 10/22/2014 09:23 AM, frohmaier c. (cf5g09) wrote: > Hi, > > I?ve recently installed astropy to my local area on a HPC in the UK (Iridis4). I need to use the fits and time capabilities of astropy. > I can use fits (from astropy.io import fits) > > However, when i try to use time: > from astropy.time import Time > > I get this error message: > > from astropy.time import Time > Traceback (most recent call last): > File "", line 1, in > File "/home/cf5g09/.local/lib/python2.7/site-packages/astropy-0.4.2-py2.7-linux-x86_64.egg/astropy/time/__init__.py", line 2, in > from .core import * > File "/home/cf5g09/.local/lib/python2.7/site-packages/astropy-0.4.2-py2.7-linux-x86_64.egg/astropy/time/core.py", line 42, in > from . import erfa_time > ImportError: /home/cf5g09/.local/lib/python2.7/site-packages/astropy-0.4.2-py2.7-linux-x86_64.egg/astropy/time/erfa_time.so: undefined symbol: __svml_sin2 > > Could someone explain to me how to fix this error or how I can reinstall astropy so that Time is usable? Based one what I'm seeing here you might want to have a look at this thread about building Astropy with the Intel C compiler: http://mail.scipy.org/pipermail/astropy/2013-November/002914.html Erik From mdroe at stsci.edu Wed Oct 22 09:59:10 2014 From: mdroe at stsci.edu (Michael Droettboom) Date: Wed, 22 Oct 2014 09:59:10 -0400 Subject: [AstroPy] pyregion and astropy -- differences of pywcs and astropy.wcs In-Reply-To: References: Message-ID: <5447B82E.5080305@stsci.edu> It seems you're sorted, but just to answer your original question: yes -- world2pix is the new name for the deprecated sky2pix method. Mike On 10/21/2014 01:05 PM, Slavin, Jonathan wrote: > ?This problem has been fixed in the most recent version of pyregion on > github, but somehow that version was not correctly copied to pypi -- > so my pip install of it had the old version of wcs_helper.py despite > the version no. indicating it was up-to-date. > > Jon? > > On Tue, Oct 21, 2014 at 1:00 PM, > wrote: > > Hi, > > I've run into an issue with using pyregion that seems to be related to > differences in the API for pywcs vs. astropy.wcs. This may be > discussed > somewhere, but I didn't run across it. The problem is that > pyregion wants > to invoke a wcs_sky2pix method for an astropy.wcs.wcs.WCS object. > Apparently pywcs.WCS objects have this method. So I'm wondering, > is the > pywcs wcs_sky2pix method the same as the astropy.wcs method > wcs_world2pix? > If so, then the fix to pyregion is easy. > > Regards, > Jon > > > > > > -- > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden > Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Michael Droettboom Science Software Branch Space Telescope Science Institute http://www.droettboom.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From indy.leclercq at manchester.ac.uk Thu Oct 30 11:39:37 2014 From: indy.leclercq at manchester.ac.uk (Indy Leclercq) Date: Thu, 30 Oct 2014 15:39:37 +0000 Subject: [AstroPy] Fits file missing END card error when END is present in the header Message-ID: Hello, I?ve been processing a batch of similar of FITS files in python using astropy 0.4.2 on Mac OSX 10.9, and on one subset of them (with a header in identical format to the others), fits.open throws an error saying that there is a missing END card in the file. Viewing the header in the terminal shows that the END card is there, and as I said before, the routine works fine on files with essentially the same header. Could anyone shed some light on this, or is this a weird bug? I can bypass this with the ignore_missing_end kwarg, but still find it weird. Here are example of working and non-working headers (copied from a terminal more command) This file opens fine: SIMPLE = T / Standard FITS file BITPIX = -32 / IEEE 4-byte float NAXIS = 3 / Number of image dimensions NAXIS1 = 5155 / Size of 1st dimension in pixels NAXIS2 = 1070 / Size of 2nd dimension in pixels NAXIS3 = 376 / OBJECT = 'GALFACTS_S1 Stokes I' / Object name CTYPE1 = 'RA---CAR' / 1st axis type CRVAL1 = 102.050003 / Reference pixel value CRPIX1 = 2578.00 / Reference pixel CDELT1 = -0.0166667 / Pixel size in world coordinate units CROTA1 = 0.0000 / Axis rotation in degrees CTYPE2 = 'DEC--CAR' / 2nd axis type CRVAL2 = 0.000000 / Reference pixel value CRPIX2 = 53.25 / Reference pixel CDELT2 = 0.0166667 / Pixel size in world coordinate units CROTA2 = 0.0000 / Axis rotation in degrees CTYPE3 = 'FREQ' / 3rd axis type CRVAL3 = 1524717952.000000 / Reference pixel value CRPIX3 = 1.00 / Reference pixel CDELT3 = -420000.0000000 / Pixel size in world coordinate units CROTA3 = 0.0000 / Axis rotation in degrees EQUINOX = 2000.00 / Equinox of coordinates (if any) BUNIT = 'Kelvin' / Units of pixel data values END But this one doesn?t: SIMPLE = T / Standard FITS file BITPIX = -32 / IEEE 4-byte float NAXIS = 3 / Number of image dimensions NAXIS1 = 6000 / Size of 1st dimension in pixels NAXIS2 = 1074 / Size of 2nd dimension in pixels NAXIS3 = 376 / OBJECT = 'GALFACTS_N4 Stokes I' / Object name CTYPE1 = 'RA---CAR' / 1st axis type CRVAL1 = 335.000000 / Reference pixel value CRPIX1 = 3000.50 / Reference pixel CDELT1 = -0.0166667 / Pixel size in world coordinate units CROTA1 = 0.0000 / Axis rotation in degrees CTYPE2 = 'DEC--CAR' / 2nd axis type CRVAL2 = 0.000000 / Reference pixel value CRPIX2 = -1181.50 / Reference pixel CDELT2 = 0.0166667 / Pixel size in world coordinate units CROTA2 = 0.0000 / Axis rotation in degrees CTYPE3 = 'FREQ' / 3rd axis type CRVAL3 = 1524717952.000000 / Reference pixel value CRPIX3 = 1.00 / Reference pixel CDELT3 = -420000.0000000 / Pixel size in world coordinate units CROTA3 = 0.0000 / Axis rotation in degrees EQUINOX = 2000.00 / Equinox of coordinates (if any) BUNIT = 'Kelvin' / Units of pixel data values END Thanks for any help you could provide! Cheers, Indy ------------------------------------------------------- Indy Leclercq Jodrell Bank Centre for Astrophysics Alan Turing Building, Manchester M13 9PL indy.leclercq at manchester.ac.uk ------------------------------------------------------- Indy Leclercq Jodrell Bank Centre for Astrophysics Alan Turing Building, Manchester M13 9PL indy.leclercq at manchester.ac.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Thu Oct 30 11:51:38 2014 From: embray at stsci.edu (Erik Bray) Date: Thu, 30 Oct 2014 11:51:38 -0400 Subject: [AstroPy] Fits file missing END card error when END is present in the header In-Reply-To: References: Message-ID: <54525E8A.9040600@stsci.edu> One suggestion I can offer is that there are some cases where the "missing END card" error is a bit misleading (patches welcome to fix this issue). This is because there is no way in FITS to know how many HDUs *should* be in a file, so if there are extra bytes at the end of a FITS file it will start reading those bytes as though it's supposed to be the start of another FITS header. But then when it doesn't find an END card before the end of the file it raises that error. One case I've seen very commonly is if somebody opened in a FITS file in a text editor and then closed it again, many editors will append a newline to the end of the file. Other cases have occurred where the software that wrote the FITS files appended extra bytes incorrectly. Erik On 10/30/2014 11:39 AM, Indy Leclercq wrote: > Hello, > > I?ve been processing a batch of similar of FITS files in python using astropy > 0.4.2 on Mac OSX 10.9, and on one subset of them (with a header in identical > format to the others), fits.open throws an error saying that there is a missing > END card in the file. Viewing the header in the terminal shows that the END card > is there, and as I said before, the routine works fine on files with essentially > the same header. > > Could anyone shed some light on this, or is this a weird bug? I can bypass this > with the ignore_missing_end kwarg, but still find it weird. > > Here are example of working and non-working headers (copied from a terminal more > command) > > This file opens fine: > > SIMPLE = T / Standard FITS file > BITPIX = -32 / IEEE 4-byte float > NAXIS = 3 / Number of image dimensions > NAXIS1 = 5155 / Size of 1st dimension in pixels > NAXIS2 = 1070 / Size of 2nd dimension in pixels > NAXIS3 = 376 / > OBJECT = 'GALFACTS_S1 Stokes I' / Object name > CTYPE1 = 'RA---CAR' / 1st axis type > CRVAL1 = 102.050003 / Reference pixel value > CRPIX1 = 2578.00 / Reference pixel > CDELT1 = -0.0166667 / Pixel size in world coordinate units > CROTA1 = 0.0000 / Axis rotation in degrees > CTYPE2 = 'DEC--CAR' / 2nd axis type > CRVAL2 = 0.000000 / Reference pixel value > CRPIX2 = 53.25 / Reference pixel > CDELT2 = 0.0166667 / Pixel size in world coordinate units > CROTA2 = 0.0000 / Axis rotation in degrees > CTYPE3 = 'FREQ' / 3rd axis type > CRVAL3 = 1524717952.000000 / Reference pixel value > CRPIX3 = 1.00 / Reference pixel > CDELT3 = -420000.0000000 / Pixel size in world coordinate units > CROTA3 = 0.0000 / Axis rotation in degrees > EQUINOX = 2000.00 / Equinox of coordinates (if any) > BUNIT = 'Kelvin' / Units of pixel data values > END > > > But this one doesn?t: > > SIMPLE = T / Standard FITS file > BITPIX = -32 / IEEE 4-byte float > NAXIS = 3 / Number of image dimensions > NAXIS1 = 6000 / Size of 1st dimension in pixels > NAXIS2 = 1074 / Size of 2nd dimension in pixels > NAXIS3 = 376 / > OBJECT = 'GALFACTS_N4 Stokes I' / Object name > CTYPE1 = 'RA---CAR' / 1st axis type > CRVAL1 = 335.000000 / Reference pixel value > CRPIX1 = 3000.50 / Reference pixel > CDELT1 = -0.0166667 / Pixel size in world coordinate units > CROTA1 = 0.0000 / Axis rotation in degrees > CTYPE2 = 'DEC--CAR' / 2nd axis type > CRVAL2 = 0.000000 / Reference pixel value > CRPIX2 = -1181.50 / Reference pixel > CDELT2 = 0.0166667 / Pixel size in world coordinate units > CROTA2 = 0.0000 / Axis rotation in degrees > CTYPE3 = 'FREQ' / 3rd axis type > CRVAL3 = 1524717952.000000 / Reference pixel value > CRPIX3 = 1.00 / Reference pixel > CDELT3 = -420000.0000000 / Pixel size in world coordinate units > CROTA3 = 0.0000 / Axis rotation in degrees > EQUINOX = 2000.00 / Equinox of coordinates (if any) > BUNIT = 'Kelvin' / Units of pixel data values > END > > Thanks for any help you could provide! From indy.leclercq at manchester.ac.uk Thu Oct 30 12:03:42 2014 From: indy.leclercq at manchester.ac.uk (Indy Leclercq) Date: Thu, 30 Oct 2014 16:03:42 +0000 Subject: [AstroPy] Fits file missing END card error when END is present in the header In-Reply-To: <54525E8A.9040600@stsci.edu> References: <54525E8A.9040600@stsci.edu> Message-ID: <8BC90CE9-EE6E-4135-BBA6-2AE25F736A81@manchester.ac.uk> Hi Erik, Thanks for pointing that out ? I didn?t make these files myself and after poking around a bit it would seem that there are indeed some extra bytes in the files that don?t open properly. I think that?s what the problem was here ? will have to talk to the person who made the files! Cheers, Indy On 30 Oct 2014, at 17:51, Erik Bray wrote: > One suggestion I can offer is that there are some cases where the "missing END > card" error is a bit misleading (patches welcome to fix this issue). This is > because there is no way in FITS to know how many HDUs *should* be in a file, so > if there are extra bytes at the end of a FITS file it will start reading those > bytes as though it's supposed to be the start of another FITS header. But then > when it doesn't find an END card before the end of the file it raises that error. > > One case I've seen very commonly is if somebody opened in a FITS file in a text > editor and then closed it again, many editors will append a newline to the end > of the file. Other cases have occurred where the software that wrote the FITS > files appended extra bytes incorrectly. > > Erik > > > On 10/30/2014 11:39 AM, Indy Leclercq wrote: >> Hello, >> >> I?ve been processing a batch of similar of FITS files in python using astropy >> 0.4.2 on Mac OSX 10.9, and on one subset of them (with a header in identical >> format to the others), fits.open throws an error saying that there is a missing >> END card in the file. Viewing the header in the terminal shows that the END card >> is there, and as I said before, the routine works fine on files with essentially >> the same header. >> >> Could anyone shed some light on this, or is this a weird bug? I can bypass this >> with the ignore_missing_end kwarg, but still find it weird. >> >> Here are example of working and non-working headers (copied from a terminal more >> command) >> >> This file opens fine: >> >> SIMPLE = T / Standard FITS file >> BITPIX = -32 / IEEE 4-byte float >> NAXIS = 3 / Number of image dimensions >> NAXIS1 = 5155 / Size of 1st dimension in pixels >> NAXIS2 = 1070 / Size of 2nd dimension in pixels >> NAXIS3 = 376 / >> OBJECT = 'GALFACTS_S1 Stokes I' / Object name >> CTYPE1 = 'RA---CAR' / 1st axis type >> CRVAL1 = 102.050003 / Reference pixel value >> CRPIX1 = 2578.00 / Reference pixel >> CDELT1 = -0.0166667 / Pixel size in world coordinate units >> CROTA1 = 0.0000 / Axis rotation in degrees >> CTYPE2 = 'DEC--CAR' / 2nd axis type >> CRVAL2 = 0.000000 / Reference pixel value >> CRPIX2 = 53.25 / Reference pixel >> CDELT2 = 0.0166667 / Pixel size in world coordinate units >> CROTA2 = 0.0000 / Axis rotation in degrees >> CTYPE3 = 'FREQ' / 3rd axis type >> CRVAL3 = 1524717952.000000 / Reference pixel value >> CRPIX3 = 1.00 / Reference pixel >> CDELT3 = -420000.0000000 / Pixel size in world coordinate units >> CROTA3 = 0.0000 / Axis rotation in degrees >> EQUINOX = 2000.00 / Equinox of coordinates (if any) >> BUNIT = 'Kelvin' / Units of pixel data values >> END >> >> >> But this one doesn?t: >> >> SIMPLE = T / Standard FITS file >> BITPIX = -32 / IEEE 4-byte float >> NAXIS = 3 / Number of image dimensions >> NAXIS1 = 6000 / Size of 1st dimension in pixels >> NAXIS2 = 1074 / Size of 2nd dimension in pixels >> NAXIS3 = 376 / >> OBJECT = 'GALFACTS_N4 Stokes I' / Object name >> CTYPE1 = 'RA---CAR' / 1st axis type >> CRVAL1 = 335.000000 / Reference pixel value >> CRPIX1 = 3000.50 / Reference pixel >> CDELT1 = -0.0166667 / Pixel size in world coordinate units >> CROTA1 = 0.0000 / Axis rotation in degrees >> CTYPE2 = 'DEC--CAR' / 2nd axis type >> CRVAL2 = 0.000000 / Reference pixel value >> CRPIX2 = -1181.50 / Reference pixel >> CDELT2 = 0.0166667 / Pixel size in world coordinate units >> CROTA2 = 0.0000 / Axis rotation in degrees >> CTYPE3 = 'FREQ' / 3rd axis type >> CRVAL3 = 1524717952.000000 / Reference pixel value >> CRPIX3 = 1.00 / Reference pixel >> CDELT3 = -420000.0000000 / Pixel size in world coordinate units >> CROTA3 = 0.0000 / Axis rotation in degrees >> EQUINOX = 2000.00 / Equinox of coordinates (if any) >> BUNIT = 'Kelvin' / Units of pixel data values >> END >> >> Thanks for any help you could provide! > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From embray at stsci.edu Thu Oct 30 13:09:56 2014 From: embray at stsci.edu (Erik Bray) Date: Thu, 30 Oct 2014 13:09:56 -0400 Subject: [AstroPy] Fits file missing END card error when END is present in the header In-Reply-To: <8BC90CE9-EE6E-4135-BBA6-2AE25F736A81@manchester.ac.uk> References: <54525E8A.9040600@stsci.edu> <8BC90CE9-EE6E-4135-BBA6-2AE25F736A81@manchester.ac.uk> Message-ID: <545270E4.5050807@stsci.edu> Great--I'm glad it was as simple as that. It's on my todo list to specifically detect the case of extra trailing bytes so that a more specific error can be raised (or probably just a warning since trailing bytes don't make the rest of the file unreadable). Erik On 10/30/2014 12:03 PM, Indy Leclercq wrote: > Hi Erik, > > Thanks for pointing that out ? I didn?t make these files myself and after poking around a bit it would seem that there are indeed some extra bytes in the files that don?t open properly. I think that?s what the problem was here ? will have to talk to the person who made the files! > > Cheers, > > Indy > > > On 30 Oct 2014, at 17:51, Erik Bray wrote: > >> One suggestion I can offer is that there are some cases where the "missing END >> card" error is a bit misleading (patches welcome to fix this issue). This is >> because there is no way in FITS to know how many HDUs *should* be in a file, so >> if there are extra bytes at the end of a FITS file it will start reading those >> bytes as though it's supposed to be the start of another FITS header. But then >> when it doesn't find an END card before the end of the file it raises that error. >> >> One case I've seen very commonly is if somebody opened in a FITS file in a text >> editor and then closed it again, many editors will append a newline to the end >> of the file. Other cases have occurred where the software that wrote the FITS >> files appended extra bytes incorrectly. >> >> Erik >> >> >> On 10/30/2014 11:39 AM, Indy Leclercq wrote: >>> Hello, >>> >>> I?ve been processing a batch of similar of FITS files in python using astropy >>> 0.4.2 on Mac OSX 10.9, and on one subset of them (with a header in identical >>> format to the others), fits.open throws an error saying that there is a missing >>> END card in the file. Viewing the header in the terminal shows that the END card >>> is there, and as I said before, the routine works fine on files with essentially >>> the same header. >>> >>> Could anyone shed some light on this, or is this a weird bug? I can bypass this >>> with the ignore_missing_end kwarg, but still find it weird. >>> >>> Here are example of working and non-working headers (copied from a terminal more >>> command) >>> >>> This file opens fine: >>> >>> SIMPLE = T / Standard FITS file >>> BITPIX = -32 / IEEE 4-byte float >>> NAXIS = 3 / Number of image dimensions >>> NAXIS1 = 5155 / Size of 1st dimension in pixels >>> NAXIS2 = 1070 / Size of 2nd dimension in pixels >>> NAXIS3 = 376 / >>> OBJECT = 'GALFACTS_S1 Stokes I' / Object name >>> CTYPE1 = 'RA---CAR' / 1st axis type >>> CRVAL1 = 102.050003 / Reference pixel value >>> CRPIX1 = 2578.00 / Reference pixel >>> CDELT1 = -0.0166667 / Pixel size in world coordinate units >>> CROTA1 = 0.0000 / Axis rotation in degrees >>> CTYPE2 = 'DEC--CAR' / 2nd axis type >>> CRVAL2 = 0.000000 / Reference pixel value >>> CRPIX2 = 53.25 / Reference pixel >>> CDELT2 = 0.0166667 / Pixel size in world coordinate units >>> CROTA2 = 0.0000 / Axis rotation in degrees >>> CTYPE3 = 'FREQ' / 3rd axis type >>> CRVAL3 = 1524717952.000000 / Reference pixel value >>> CRPIX3 = 1.00 / Reference pixel >>> CDELT3 = -420000.0000000 / Pixel size in world coordinate units >>> CROTA3 = 0.0000 / Axis rotation in degrees >>> EQUINOX = 2000.00 / Equinox of coordinates (if any) >>> BUNIT = 'Kelvin' / Units of pixel data values >>> END >>> >>> >>> But this one doesn?t: >>> >>> SIMPLE = T / Standard FITS file >>> BITPIX = -32 / IEEE 4-byte float >>> NAXIS = 3 / Number of image dimensions >>> NAXIS1 = 6000 / Size of 1st dimension in pixels >>> NAXIS2 = 1074 / Size of 2nd dimension in pixels >>> NAXIS3 = 376 / >>> OBJECT = 'GALFACTS_N4 Stokes I' / Object name >>> CTYPE1 = 'RA---CAR' / 1st axis type >>> CRVAL1 = 335.000000 / Reference pixel value >>> CRPIX1 = 3000.50 / Reference pixel >>> CDELT1 = -0.0166667 / Pixel size in world coordinate units >>> CROTA1 = 0.0000 / Axis rotation in degrees >>> CTYPE2 = 'DEC--CAR' / 2nd axis type >>> CRVAL2 = 0.000000 / Reference pixel value >>> CRPIX2 = -1181.50 / Reference pixel >>> CDELT2 = 0.0166667 / Pixel size in world coordinate units >>> CROTA2 = 0.0000 / Axis rotation in degrees >>> CTYPE3 = 'FREQ' / 3rd axis type >>> CRVAL3 = 1524717952.000000 / Reference pixel value >>> CRPIX3 = 1.00 / Reference pixel >>> CDELT3 = -420000.0000000 / Pixel size in world coordinate units >>> CROTA3 = 0.0000 / Axis rotation in degrees >>> EQUINOX = 2000.00 / Equinox of coordinates (if any) >>> BUNIT = 'Kelvin' / Units of pixel data values >>> END >>> >>> Thanks for any help you could provide! >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From goutam2475 at gmail.com Thu Oct 30 13:16:34 2014 From: goutam2475 at gmail.com (Goutam Das) Date: Thu, 30 Oct 2014 22:46:34 +0530 Subject: [AstroPy] WISHING TO CONTRIBUTE Message-ID: Hi, I am a newbie and I want to start contributing to the project. Can someone please point to some good first bugs to solve or suggest some other way to get started. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Oct 30 14:39:06 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 30 Oct 2014 18:39:06 +0000 Subject: [AstroPy] WISHING TO CONTRIBUTE In-Reply-To: References: Message-ID: <545285CA.1050601@gmail.com> Hi Goutam, Great that you are interested in contributing! If you go here: http://www.astropy.org/contribute.html you will find information about contributing, and links to documentation explaining how to submit contributions. We have a number of issues tagged as 'package-novice' which means that they require minimal knowledge of the infrastructure of astropy to get started: https://github.com/astropy/astropy/issues?q=is%3Aopen+is%3Aissue+label%3APackage-novice If you have any questions about any of the issues, you can either leave a comment on the issue, or email the astropy-dev mailing list: https://groups.google.com/forum/#!forum/astropy-dev Cheers, Tom Goutam Das wrote: > Hi, I am a newbie and I want to start contributing to the project. Can > someone please point to some good first bugs to solve or suggest some > other way to get started. > Thank you > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From eisenhamer at stsci.edu Thu Oct 30 15:35:58 2014 From: eisenhamer at stsci.edu (Jonathan Eisenhamer) Date: Thu, 30 Oct 2014 19:35:58 +0000 Subject: [AstroPy] Fits file missing END card error when END is present in the header In-Reply-To: <8BC90CE9-EE6E-4135-BBA6-2AE25F736A81@manchester.ac.uk> References: <54525E8A.9040600@stsci.edu>, <8BC90CE9-EE6E-4135-BBA6-2AE25F736A81@manchester.ac.uk> Message-ID: <6788A02107EA644693721D7DEF3F408B476C29D8@EXCHMAIL1.stsci.edu> If you absolutely need to read this without modification and would like to live dangerously, you "may" be able to add the parameter ignore_missing_end=True to the fits.open call. I've had this issue before and seem to remember this working out. Its documented here as part of the PrimaryHDU class, hence I'm not sure if fits.open will pass the parameter down this far or not. jde Jonathan D. Eisenhamer STScI Science Software Branch eisenhamer at stsci.edu +1 410 338 5073 g+/gtalk: stscieisenhamer at gmail.com aim/skype/icq: triccare http://www.stsci.edu/ http://hubblesite.org/ http://amazing-space.stsci.edu/ ________________________________________ From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Indy Leclercq [indy.leclercq at manchester.ac.uk] Sent: Thursday, October 30, 2014 12:03 To: Astronomical Python mailing list Subject: Re: [AstroPy] Fits file missing END card error when END is present in the header Hi Erik, Thanks for pointing that out ? I didn?t make these files myself and after poking around a bit it would seem that there are indeed some extra bytes in the files that don?t open properly. I think that?s what the problem was here ? will have to talk to the person who made the files! Cheers, Indy On 30 Oct 2014, at 17:51, Erik Bray wrote: > One suggestion I can offer is that there are some cases where the "missing END > card" error is a bit misleading (patches welcome to fix this issue). This is > because there is no way in FITS to know how many HDUs *should* be in a file, so > if there are extra bytes at the end of a FITS file it will start reading those > bytes as though it's supposed to be the start of another FITS header. But then > when it doesn't find an END card before the end of the file it raises that error. > > One case I've seen very commonly is if somebody opened in a FITS file in a text > editor and then closed it again, many editors will append a newline to the end > of the file. Other cases have occurred where the software that wrote the FITS > files appended extra bytes incorrectly. > > Erik > > > On 10/30/2014 11:39 AM, Indy Leclercq wrote: >> Hello, >> >> I?ve been processing a batch of similar of FITS files in python using astropy >> 0.4.2 on Mac OSX 10.9, and on one subset of them (with a header in identical >> format to the others), fits.open throws an error saying that there is a missing >> END card in the file. Viewing the header in the terminal shows that the END card >> is there, and as I said before, the routine works fine on files with essentially >> the same header. >> >> Could anyone shed some light on this, or is this a weird bug? I can bypass this >> with the ignore_missing_end kwarg, but still find it weird. >> >> Here are example of working and non-working headers (copied from a terminal more >> command) >> >> This file opens fine: >> >> SIMPLE = T / Standard FITS file >> BITPIX = -32 / IEEE 4-byte float >> NAXIS = 3 / Number of image dimensions >> NAXIS1 = 5155 / Size of 1st dimension in pixels >> NAXIS2 = 1070 / Size of 2nd dimension in pixels >> NAXIS3 = 376 / >> OBJECT = 'GALFACTS_S1 Stokes I' / Object name >> CTYPE1 = 'RA---CAR' / 1st axis type >> CRVAL1 = 102.050003 / Reference pixel value >> CRPIX1 = 2578.00 / Reference pixel >> CDELT1 = -0.0166667 / Pixel size in world coordinate units >> CROTA1 = 0.0000 / Axis rotation in degrees >> CTYPE2 = 'DEC--CAR' / 2nd axis type >> CRVAL2 = 0.000000 / Reference pixel value >> CRPIX2 = 53.25 / Reference pixel >> CDELT2 = 0.0166667 / Pixel size in world coordinate units >> CROTA2 = 0.0000 / Axis rotation in degrees >> CTYPE3 = 'FREQ' / 3rd axis type >> CRVAL3 = 1524717952.000000 / Reference pixel value >> CRPIX3 = 1.00 / Reference pixel >> CDELT3 = -420000.0000000 / Pixel size in world coordinate units >> CROTA3 = 0.0000 / Axis rotation in degrees >> EQUINOX = 2000.00 / Equinox of coordinates (if any) >> BUNIT = 'Kelvin' / Units of pixel data values >> END >> >> >> But this one doesn?t: >> >> SIMPLE = T / Standard FITS file >> BITPIX = -32 / IEEE 4-byte float >> NAXIS = 3 / Number of image dimensions >> NAXIS1 = 6000 / Size of 1st dimension in pixels >> NAXIS2 = 1074 / Size of 2nd dimension in pixels >> NAXIS3 = 376 / >> OBJECT = 'GALFACTS_N4 Stokes I' / Object name >> CTYPE1 = 'RA---CAR' / 1st axis type >> CRVAL1 = 335.000000 / Reference pixel value >> CRPIX1 = 3000.50 / Reference pixel >> CDELT1 = -0.0166667 / Pixel size in world coordinate units >> CROTA1 = 0.0000 / Axis rotation in degrees >> CTYPE2 = 'DEC--CAR' / 2nd axis type >> CRVAL2 = 0.000000 / Reference pixel value >> CRPIX2 = -1181.50 / Reference pixel >> CDELT2 = 0.0166667 / Pixel size in world coordinate units >> CROTA2 = 0.0000 / Axis rotation in degrees >> CTYPE3 = 'FREQ' / 3rd axis type >> CRVAL3 = 1524717952.000000 / Reference pixel value >> CRPIX3 = 1.00 / Reference pixel >> CDELT3 = -420000.0000000 / Pixel size in world coordinate units >> CROTA3 = 0.0000 / Axis rotation in degrees >> EQUINOX = 2000.00 / Equinox of coordinates (if any) >> BUNIT = 'Kelvin' / Units of pixel data values >> END >> >> Thanks for any help you could provide! > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL:
CIG11.23None3
None3252
CIG3232.6NaN