From stephen.walton at csun.edu Fri Jun 10 19:57:24 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Fri, 10 Jun 2005 16:57:24 -0700 Subject: [AstroPy] WCS query Message-ID: <42AA28E4.7090007@csun.edu> Hi, This seemed the best list to try to ask this on. I have a 1024 square full disk solar image with a simple linear WCS which defines Bill Thompson's SOLX and SOLY coordinates: x=0,y=1 is the solar N limb, x=1,y=0 is the solar W limb. If I do the following in IRAF: disp image 1 xs=0.8 ys=0.8 fill+ wcslab image 1 vl=0.1 vr=0.9 vb=0.1 vt=0.9 fill+ the WCS looks fine; in particular, the (x=(0,1),y=(0,1)) contours are nicely tangent to the solar limb. However, if I read the WCS with PyFITS and try to compute a WCS array in Python, things go wrong. Here's what I did: I created a 2x2 cd array in Python where cd[k-1,m-1] was set to CD_km from the FITS header; I also read CRPIX1 and CRPIX2 from the FITS header into crpix1 and crpix2 variables. Two arrays, x and y, were produced, where x[k,m]=k and y[k,m]=m, k,m=0,1023. Finally, I created xx and yy as follows: xx = cd[0,0]*(x-crpix1+1) + cd[0,1]*(y-crpix2+1) yy=cd[1,0]*(x-crpix1+1)+cd[1,1]*(y-crpix2+1) (the +1 in the above is because crpix1 and crpix2 assume the array indexing is one-based, not zero-based). Then, using matplotlib, I did contour(image) contour(xx,[-1,0,1]) contour(yy,[-1,0,1]) The lines at +1 and -1 are definitely not tangent to the limb. The strange thing is, if I do xx.transpose() and yy.transpose() and repeat the above three matplotlib commands the +1 and -1 lines are tangent to the limb. Since the image is both off center and tilted with respect to solar N-S, this cannot be a coincidence. Can anyone enlighten me as to what went wrong here? From perry at stsci.edu Wed Jun 15 11:16:01 2005 From: perry at stsci.edu (Perry Greenfield) Date: Wed, 15 Jun 2005 11:16:01 -0400 Subject: [AstroPy] WCS query In-Reply-To: <42AA28E4.7090007@csun.edu> References: <42AA28E4.7090007@csun.edu> Message-ID: <348bcb8dd613607f19a8ec2b4435b715@stsci.edu> Hi Steve, I have a feeling this is related to your matplotlib query. On Jun 10, 2005, at 7:57 PM, Stephen Walton wrote: > Hi, > > This seemed the best list to try to ask this on. I have a 1024 square > full disk solar image with a simple linear WCS which defines Bill > Thompson's SOLX and SOLY coordinates: x=0,y=1 is the solar N limb, > x=1,y=0 is the solar W limb. If I do the following in IRAF: > > disp image 1 xs=0.8 ys=0.8 fill+ > wcslab image 1 vl=0.1 vr=0.9 vb=0.1 vt=0.9 fill+ > > the WCS looks fine; in particular, the (x=(0,1),y=(0,1)) contours are > nicely tangent to the solar limb. > > However, if I read the WCS with PyFITS and try to compute a WCS array > in Python, things go wrong. Here's what I did: I created a 2x2 cd > array in Python where cd[k-1,m-1] was set to CD_km from the FITS > header; I also read CRPIX1 and CRPIX2 from the FITS header into crpix1 > and crpix2 variables. Two arrays, x and y, were produced, where > x[k,m]=k and y[k,m]=m, k,m=0,1023. Finally, I created xx and yy as > follows: > The above suggests you were associating x coordinates with the first index and y with the second. Of course with numarray (and Numeric) that's wrong. Is this the crux of the problem? > xx = cd[0,0]*(x-crpix1+1) + cd[0,1]*(y-crpix2+1) > yy=cd[1,0]*(x-crpix1+1)+cd[1,1]*(y-crpix2+1) > > (the +1 in the above is because crpix1 and crpix2 assume the array > indexing is one-based, not zero-based). Then, using matplotlib, I did > > contour(image) > contour(xx,[-1,0,1]) > contour(yy,[-1,0,1]) > > The lines at +1 and -1 are definitely not tangent to the limb. The > strange thing is, if I do xx.transpose() and yy.transpose() and repeat > the above three matplotlib commands the +1 and -1 lines are tangent to > the limb. Since the image is both off center and tilted with respect > to solar N-S, this cannot be a coincidence. > > Can anyone enlighten me as to what went wrong here? > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.net > http://www.scipy.net/mailman/listinfo/astropy From stephen.walton at csun.edu Wed Jun 15 13:02:29 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Wed, 15 Jun 2005 10:02:29 -0700 Subject: [AstroPy] WCS query In-Reply-To: <348bcb8dd613607f19a8ec2b4435b715@stsci.edu> References: <42AA28E4.7090007@csun.edu> <348bcb8dd613607f19a8ec2b4435b715@stsci.edu> Message-ID: <42B05F25.8060208@csun.edu> Perry Greenfield wrote: > Hi Steve, > > I have a feeling this is related to your matplotlib query. Yes. > > On Jun 10, 2005, at 7:57 PM, Stephen Walton wrote: > >> However, if I read the WCS with PyFITS and try to compute a WCS array >> in Python, things go wrong. Here's what I did: I created a 2x2 cd >> array in Python where cd[k-1,m-1] was set to CD_km from the FITS >> header; I also read CRPIX1 and CRPIX2 from the FITS header into >> crpix1 and crpix2 variables. Two arrays, x and y, were produced, >> where x[k,m]=k and y[k,m]=m, k,m=0,1023. Finally, I created xx and >> yy as follows: >> > The above suggests you were associating x coordinates with the first > index and y with the second. Of course with numarray (and Numeric) > that's wrong. Is this the crux of the problem? As it turns out, yes, and the code below looks correct and seems to produce the right result: x,y=meshgrid(range(1024),range(1024)) # x[i,j] == j, y[i,j] == i xx=cd[0,0]*(x-crpix1+1)+cd[0,1]*(y-crpix2+1) yy=cd[1,0]*(x-crpix1+1)+cd[1,1]*(y-crpix2+1) But I still find it somewhat awkward, and users here are going to find it very confusing, that what we old Fortran hands think of as FITS pixel (i,j) winds up displayed in matplotlib at Cartesian coordinates (i-1,j-1) even though that pixel would have to be addressed as data[j-1,i-1] in software. This is because John Hunter, not unreasonably, adopted the MATLAB convention for imshow(), contour(), and their relatives that the first array coordinate is vertical and the second horizontal. Effectively, then, imshow() 'undoes' the transpose done by PyFITS so that the displayed image actually comes out in the same orientation on the screen as it does in IRAF (if origin='lower' is used in imshow, as I do) even though the actual array has to be addressed with the indices in reverse order from Fortran. Incidentally, MATLAB's provided fitsread routine transposes FITS data on input, so in MATLAB data(i,j) refers to the same pixel as in Fortran (and image displays *look* transposed). I suppose long-time CFITSIO and PyFITS users are just used to all of this. From perry at stsci.edu Wed Jun 15 16:02:54 2005 From: perry at stsci.edu (Perry Greenfield) Date: Wed, 15 Jun 2005 16:02:54 -0400 Subject: [AstroPy] WCS query In-Reply-To: <42B05F25.8060208@csun.edu> References: <42AA28E4.7090007@csun.edu> <348bcb8dd613607f19a8ec2b4435b715@stsci.edu> <42B05F25.8060208@csun.edu> Message-ID: <95aadc78359af1f8b637c6fb740bc34f@stsci.edu> On Jun 15, 2005, at 1:02 PM, Stephen Walton wrote: > But I still find it somewhat awkward, and users here are going to find > it very confusing, that what we old Fortran hands think of as FITS > pixel (i,j) winds up displayed in matplotlib at Cartesian coordinates > (i-1,j-1) even though that pixel would have to be addressed as > data[j-1,i-1] in software. This is because John Hunter, not > unreasonably, adopted the MATLAB convention for imshow(), contour(), > and their relatives that the first array coordinate is vertical and > the second horizontal. Effectively, then, imshow() 'undoes' the > transpose done by PyFITS so that the displayed image actually comes > out in the same orientation on the screen as it does in IRAF (if > origin='lower' is used in imshow, as I do) even though the actual > array has to be addressed with the indices in reverse order from > Fortran. > This whole area hinges on the definitions of what the conventions are (which can be confusing). I'm not sure I'd classify it as John Hunter's convention but more properly a Numeric/numarray convention. That's because the data as read from the FITS file retain the same order they did in the FITS file (PyFITS doesn't reorder the data). Since the last index for Numeric/numarray represents the adjacent data values, that's where the issue really arises, not in matplotlib other than it is adopting the convention that adjacent data values are associated with x as far as images go (but having two different conventions for which way y should be displayed, i.e., up or down). As I've mentioned, this is probably the biggest sore point astronomers are going to have with numarray (or Numeric). Different approaches could have been used, but all have their drawbacks: 1) numarray should have adopted the other convention. It could have, but then it would have been incompatible with Numeric in this respect and that would have been bad. [Numeric chose (I believe) to do it that way since it means that arr[i,j] == arr[i][j] instead of arr[i,j]==arr[j][i] and that was thought important enough to be consistent for] 2) PyFITS could actually reorder the data on reading. I suppose, but this can be an expensive operation for large datasets and it renders memory mapping useless. Even after doing so, the presumption that x corresponds to adjacent data values is no longer true (so if algorithms are written that presume that, large inefficiencies will result) 3) PyFITS could read the data in the normal order and fiddle with the array stride attributes to have the same effect. This would allow continued use of memory mapping. Although now the first index represents adjacent data, all other Numeric/numarray libraries that presume the last index usually represents adjacent data will be wrong. Any of these choices results in some sort of mismatch. We concluded that, although initially painful, the best thing was to remain as consistent with the existing language and array package semantics rather than introduce tricks to make it look like another language's semantics. The former is much more painful up front, the latter introduces long-term problems that never go away. Perhaps that's a bad marketing strategy. It's a little like the 0-based vs 1-based indexing issue. One can try to set things up so that you can use 1-based indexing in a 0-based language, but in the end it's just more trouble than learning to use it the way the language works (in my opinion anyway). But I'll agree that this is probably the single biggest irritation that astronomers will face. > Incidentally, MATLAB's provided fitsread routine transposes FITS data > on input, so in MATLAB data(i,j) refers to the same pixel as in > Fortran (and image displays *look* transposed). > > I suppose long-time CFITSIO and PyFITS users are just used to all of > this. > That depends. There was never consensus even in our group about which approach was best (though there did seem to be an age correlation, but maybe I imagined that). Perry