From rays at blue-cove.com Sat Feb 1 17:50:22 2014 From: rays at blue-cove.com (RayS) Date: Sat, 01 Feb 2014 14:50:22 -0800 Subject: [AstroPy] striding through arbitrarily large files Message-ID: <201402012250.s11MoPVM003787@blue-cove.com> I hope this isn't too off-topic for astro, but I know many here work with huge files. I was struggling yesterday with methods of reading large disk files into numpy efficiently (not FITS, just raw files of IEEE floats). When loading arbitrarily large files it would be nice to not bother reading more than the plot can display before zooming in. With a 2GB file, I want to read n (like 4096) evenly sampled points out of it. I tried making a dtype, and other tricks, to read "Pythonically", but failed. I broke down and used a for loop with fh.seek() and fromstring() num_channels = 9 desired_len = 4096 bytes_per_val = numpy.dtype(numpy.float32).itemsize f_obj = open(path, 'rb') f_obj.seek(0,2) file_length = f_obj.tell() f_obj.seek(0,0) bytes_per_smp = num_channels * bytes_per_val num_samples = file_length / bytes_per_smp stride_smps = num_samples / desired_len ## an int stride_bytes = stride_smps * bytes_per_smp arr = numpy.zeros((desired_len, 9)) for i in range(0, desired_len, 1): f_obj.seek(i*stride_bytes, 0) arr[i] = numpy.fromstring(f_obj.read(36), dtype='f32', count=9) So, is there a better way to move the pointer through the file without a for loop? Would a generator be much faster? The dtype and other methods like mmap fail with memoryError, although apparently you can mmap with 64bit systems. - Ray -------------- next part -------------- An HTML attachment was scrubbed... URL: From indiajoe at gmail.com Sun Feb 2 09:10:21 2014 From: indiajoe at gmail.com (Joe Philip Ninan) Date: Sun, 2 Feb 2014 19:40:21 +0530 Subject: [AstroPy] Inheriting HDUList Message-ID: Hi, Is there any online example of how to create a new child object from HDUList of astropy.io.fits ? (The object astropy.io.fits.open() returns). I want to create an object class which inherits everything from HDUList, and then add some additional functions. The part i cannot figure out is how to initialize the HDUList inside the new class. class MyFitsObject(HDUList): def __init__(self,filename): # How do i initialise HDUList here, when all i have is the Fits File name? -cheers joe -- /--------------------------------------------------------------- "GNU/Linux: because a PC is a terrible thing to waste" - GNU Generation ************************************************ Joe Philip Ninan Research Scholar /________________\ DAA, | Vadakeparambil | TIFR, | Pullad P.O. | Mumbai-05, India. | Kerala, India | Ph: +917738438212 | PIN:689548 | ------------------------------\_______________/-------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jzuhone at gmail.com Sun Feb 2 09:34:58 2014 From: jzuhone at gmail.com (John ZuHone) Date: Sun, 2 Feb 2014 09:34:58 -0500 Subject: [AstroPy] Inheriting HDUList In-Reply-To: References: Message-ID: <7EE3B1C0-678D-40F8-9B6A-BD91B65498C4@gmail.com> Hi Joe, Have a look at this one: https://bitbucket.org/yt_analysis/yt/src/6120c77c828f195f8691806fc724bb730fe90680/yt/utilities/fits_image.py?at=yt-3.0 Best, John Z On Feb 2, 2014, at 9:10 AM, Joe Philip Ninan wrote: > Hi, > Is there any online example of how to create a new child object from HDUList of astropy.io.fits ? (The object astropy.io.fits.open() returns). > I want to create an object class which inherits everything from HDUList, and then add some additional functions. > The part i cannot figure out is how to initialize the HDUList inside the new class. > > class MyFitsObject(HDUList): > def __init__(self,filename): > # How do i initialise HDUList here, when all i have is the Fits File name? > > -cheers > joe > > > -- > /--------------------------------------------------------------- > "GNU/Linux: because a PC is a terrible thing to waste" - GNU Generation > > ************************************************ > Joe Philip Ninan > Research Scholar /________________\ > DAA, | Vadakeparambil | > TIFR, | Pullad P.O. | > Mumbai-05, India. | Kerala, India | > Ph: +917738438212 | PIN:689548 | > ------------------------------\_______________/-------------- > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From indiajoe at gmail.com Mon Feb 3 06:35:12 2014 From: indiajoe at gmail.com (Joe Philip Ninan) Date: Mon, 3 Feb 2014 17:05:12 +0530 Subject: [AstroPy] Inheriting HDUList In-Reply-To: <7EE3B1C0-678D-40F8-9B6A-BD91B65498C4@gmail.com> References: <7EE3B1C0-678D-40F8-9B6A-BD91B65498C4@gmail.com> Message-ID: Hi John, Thanks a lot for the link. On a quick look, it doesn't seem very obvious to me how exactly to load the data along with initialising the HDUList. But i shall go through it more carefully to understand. Thanks again, -cheers joe On 2 February 2014 20:04, John ZuHone wrote: > Hi Joe, > > Have a look at this one: > > https://bitbucket.org/yt_analysis/yt/src/6120c77c828f195f8691806fc724bb730fe90680/yt/utilities/fits_image.py?at=yt-3.0 > > Best, > > John Z > > On Feb 2, 2014, at 9:10 AM, Joe Philip Ninan wrote: > >> Hi, >> Is there any online example of how to create a new child object from HDUList of astropy.io.fits ? (The object astropy.io.fits.open() returns). >> I want to create an object class which inherits everything from HDUList, and then add some additional functions. >> The part i cannot figure out is how to initialize the HDUList inside the new class. >> >> class MyFitsObject(HDUList): >> def __init__(self,filename): >> # How do i initialise HDUList here, when all i have is the Fits File name? >> >> -cheers >> joe >> >> >> -- >> /--------------------------------------------------------------- >> "GNU/Linux: because a PC is a terrible thing to waste" - GNU Generation >> >> ************************************************ >> Joe Philip Ninan >> Research Scholar /________________\ >> DAA, | Vadakeparambil | >> TIFR, | Pullad P.O. | >> Mumbai-05, India. | Kerala, India | >> Ph: +917738438212 | PIN:689548 | >> ------------------------------\_______________/-------------- >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > -- /--------------------------------------------------------------- "GNU/Linux: because a PC is a terrible thing to waste" - GNU Generation ************************************************ Joe Philip Ninan Research Scholar /________________\ DAA, | Vadakeparambil | TIFR, | Pullad P.O. | Mumbai-05, India. | Kerala, India | Ph: +917738438212 | PIN:689548 | ------------------------------\_______________/-------------- From embray at stsci.edu Mon Feb 3 14:16:32 2014 From: embray at stsci.edu (Erik Bray) Date: Mon, 3 Feb 2014 14:16:32 -0500 Subject: [AstroPy] striding through arbitrarily large files In-Reply-To: <201402012250.s11MoPVM003787@blue-cove.com> References: <201402012250.s11MoPVM003787@blue-cove.com> Message-ID: <52EFEB10.1090502@stsci.edu> Indeed, normally I would suggest just to use mmap, but if you really have to support 32-bit systems that won't help you as much as one might like. As it is what you are doing is mostly what you need to do for Numpy, but you don't need to do thankfully is read a string from the file and then use numpy.fromstring. I actually only just discovered this myself, as it is undocumented, and have yet to fix it in PyFITS. But if you pass numpy.fromfile an open file object it will read starting from wherever the file pointer is positioned, rather than the beginning of the file. So you can just: f = open('array.raw', 'rb') f.seek(np.dtype('float32').itemsize * offset) section = np.fromfile(f, dtype='float32') So that at least saves you from having to perform the read() first. Erik On 02/01/2014 05:50 PM, RayS wrote: > I hope this isn't too off-topic for astro, but I know many here work with huge > files. > > I was struggling yesterday with methods of reading large disk files into numpy > efficiently (not FITS, just raw files of IEEE floats). When loading arbitrarily > large files it would be nice to not bother reading more than the plot can > display before zooming in. > > With a 2GB file, I want to read n (like 4096) evenly sampled points out of it. > I tried making a dtype, and other tricks, to read "Pythonically", but failed. I > broke down and used a for loop with fh.seek() and fromstring() > > num_channels = 9 > desired_len = 4096 > bytes_per_val = numpy.dtype(numpy.float32).itemsize > f_obj = open(path, 'rb') > f_obj.seek(0,2) > file_length = f_obj.tell() > f_obj.seek(0,0) > bytes_per_smp = num_channels * bytes_per_val > num_samples = file_length / bytes_per_smp > stride_smps = num_samples / desired_len ## an int > stride_bytes = stride_smps * bytes_per_smp > > arr = numpy.zeros((desired_len, 9)) > for i in range(0, desired_len, 1): > f_obj.seek(i*stride_bytes, 0) > arr[i] = numpy.fromstring(f_obj.read(36), dtype='f32', count=9) > > So, is there a better way to move the pointer through the file without a for loop? > Would a generator be much faster? > > The dtype and other methods like mmap fail with memoryError, although apparently > you can mmap with 64bit systems. > > - Ray > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From embray at stsci.edu Mon Feb 3 14:30:09 2014 From: embray at stsci.edu (Erik Bray) Date: Mon, 3 Feb 2014 14:30:09 -0500 Subject: [AstroPy] Inheriting HDUList In-Reply-To: References: Message-ID: <52EFEE41.6020908@stsci.edu> On 02/02/2014 09:10 AM, Joe Philip Ninan wrote: > Hi, > Is there any online example of how to create a new child object from HDUList of > astropy.io.fits ? (The object astropy.io.fits.open() returns). > I want to create an object class which inherits everything from HDUList, and > then add some additional functions. > The part i cannot figure out is how to initialize the HDUList inside the new class. > > class MyFitsObject(HDUList): > def __init__(self,filename): > # How do i initialise HDUList here, when all i have is the Fits File name? Hi, It depends on exactly what you want to do--you have to be careful as HDUList has a bit of internal state that needs to be managed, but for the most part if you just want to extend it with some new methods it's safe enough. But it's hard to help without knowing exactly what you need to do. One thing I would emphatically *not* recommend is to pass a filename to your __init__, but again it depends on exactly what you need the subclass to do. Normally HDUList is treated as a more abstract datastructure that may or may not have actually come from a file on disk. HDUList.fromfile should be used to read an HDUList from a file (in fact it's basically what pyfits.open() wraps). The base HDUList.__init__ simply accepts a list of HDU objects to add to the list. Erik From embray at stsci.edu Mon Feb 3 14:32:30 2014 From: embray at stsci.edu (Erik Bray) Date: Mon, 3 Feb 2014 14:32:30 -0500 Subject: [AstroPy] Inheriting HDUList In-Reply-To: <52EFEE41.6020908@stsci.edu> References: <52EFEE41.6020908@stsci.edu> Message-ID: <52EFEECE.2000401@stsci.edu> On 02/03/2014 02:30 PM, Erik Bray wrote: > On 02/02/2014 09:10 AM, Joe Philip Ninan wrote: >> Hi, >> Is there any online example of how to create a new child object from HDUList of >> astropy.io.fits ? (The object astropy.io.fits.open() returns). >> I want to create an object class which inherits everything from HDUList, and >> then add some additional functions. >> The part i cannot figure out is how to initialize the HDUList inside the new class. >> >> class MyFitsObject(HDUList): >> def __init__(self,filename): >> # How do i initialise HDUList here, when all i have is the Fits File name? > > Hi, > > It depends on exactly what you want to do--you have to be careful as HDUList has > a bit of internal state that needs to be managed, but for the most part if you > just want to extend it with some new methods it's safe enough. But it's hard to > help without knowing exactly what you need to do. > > One thing I would emphatically *not* recommend is to pass a filename to your > __init__, but again it depends on exactly what you need the subclass to do. > Normally HDUList is treated as a more abstract datastructure that may or may not > have actually come from a file on disk. HDUList.fromfile should be used to read > an HDUList from a file (in fact it's basically what pyfits.open() wraps). > > The base HDUList.__init__ simply accepts a list of HDU objects to add to the list. I should add, the example John sent from the yt project is a good example of how to extend the HDUList class. +1 to it. Erik From rays at blue-cove.com Mon Feb 3 16:27:42 2014 From: rays at blue-cove.com (RayS) Date: Mon, 03 Feb 2014 13:27:42 -0800 Subject: [AstroPy] striding through arbitrarily large files In-Reply-To: <52EFEB10.1090502@stsci.edu> References: <201402012250.s11MoPVM003787@blue-cove.com> <52EFEB10.1090502@stsci.edu> Message-ID: <201402032127.s13LRj1L025946@blue-cove.com> Nice tip, thank you. I will add the count= option to the fromfile() call to get my 9 channels of the sample - otherwise the default is to read the rest of the file. I have to support 32bit Python for now (on 64 bit machines), until mayavi supports 64. I haven't found a better, easily usable and fast 3D surface library. Ray At 11:16 AM 2/3/2014, Erik Bray wrote: >Indeed, normally I would suggest just to use mmap, but if you really have to >support 32-bit systems that won't help you as much as one might >like. As it is >what you are doing is mostly what you need to do for Numpy, but you >don't need >to do thankfully is read a string from the file and then use numpy.fromstring. > >I actually only just discovered this myself, as it is undocumented, >and have yet >to fix it in PyFITS. But if you pass numpy.fromfile an open file >object it will >read starting from wherever the file pointer is positioned, rather than the >beginning of the file. > >So you can just: > >f = open('array.raw', 'rb') >f.seek(np.dtype('float32').itemsize * offset) >section = np.fromfile(f, dtype='float32') > >So that at least saves you from having to perform the read() first. > >Erik > >On 02/01/2014 05:50 PM, RayS wrote: > > I hope this isn't too off-topic for astro, but I know many here > work with huge > > files. > > > > I was struggling yesterday with methods of reading large disk > files into numpy > > efficiently (not FITS, just raw files of IEEE floats). When > loading arbitrarily > > large files it would be nice to not bother reading more than the plot can > > display before zooming in. > > > > With a 2GB file, I want to read n (like 4096) evenly sampled > points out of it. > > I tried making a dtype, and other tricks, to read "Pythonically", > but failed. I > > broke down and used a for loop with fh.seek() and fromstring() > > > > num_channels = 9 > > desired_len = 4096 > > bytes_per_val = numpy.dtype(numpy.float32).itemsize > > f_obj = open(path, 'rb') > > f_obj.seek(0,2) > > file_length = f_obj.tell() > > f_obj.seek(0,0) > > bytes_per_smp = num_channels * bytes_per_val > > num_samples = file_length / bytes_per_smp > > stride_smps = num_samples / desired_len ## an int > > stride_bytes = stride_smps * bytes_per_smp > > > > arr = numpy.zeros((desired_len, 9)) > > for i in range(0, desired_len, 1): > > f_obj.seek(i*stride_bytes, 0) > > arr[i] = numpy.fromstring(f_obj.read(36), dtype='f32', count=9) > > > > So, is there a better way to move the pointer through the file > without a for loop? > > Would a generator be much faster? > > > > The dtype and other methods like mmap fail with memoryError, > although apparently > > you can mmap with 64bit systems. > > > > - Ray > > > > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > >_______________________________________________ >AstroPy mailing list >AstroPy at scipy.org >http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From trive at astro.su.se Fri Feb 7 12:49:25 2014 From: trive at astro.su.se (=?ISO-8859-1?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Fri, 07 Feb 2014 18:49:25 +0100 Subject: [AstroPy] Coordinates help Message-ID: <52F51CA5.6010502@astro.su.se> Hello; I am doingmy first foraging into the Astropy coordinates and wcs modules. My job is to check pointings on some blind-offset spectroscopy observations from the aqcuisition frames. What I have is this: 1) RA, DEC of reference object. 2) dx, dy offsets in arcsecs along the image axes to the object 3) RA, DEC of the object What I need to do now is to check that, starting at the center pixel and moving along, I actually end up at 3) (and secondarily, that center pixel and 1) agree). 2) is really the difficult step here. Can anyone give some hints? From d.berry at jach.hawaii.edu Fri Feb 7 13:11:27 2014 From: d.berry at jach.hawaii.edu (David Berry) Date: Fri, 7 Feb 2014 18:11:27 +0000 Subject: [AstroPy] Coordinates help In-Reply-To: <52F51CA5.6010502@astro.su.se> References: <52F51CA5.6010502@astro.su.se> Message-ID: On 7 February 2014 17:49, Th?ger Emil Rivera-Thorsen wrote: > Hello; > > I am doingmy first foraging into the Astropy coordinates and wcs modules. > My job is to check pointings on some blind-offset spectroscopy > observations from the aqcuisition frames. What I have is this: > > 1) RA, DEC of reference object. > 2) dx, dy offsets in arcsecs along the image axes to the object > 3) RA, DEC of the object > > What I need to do now is to check that, starting at the center pixel and > moving along, I actually end up at 3) (and secondarily, that center > pixel and 1) agree). I think to do this you will also need to know the orientation of the image axes with respect to RA and Dec. David > 2) is really the difficult step here. Can anyone give some hints? > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From bkloppenborg at gmail.com Fri Feb 7 23:22:55 2014 From: bkloppenborg at gmail.com (Brian Kloppenborg) Date: Fri, 07 Feb 2014 23:22:55 -0500 Subject: [AstroPy] SQL-like queries against astropy.table objects Message-ID: <52F5B11F.60403@gmail.com> Greetings, I have unfortunately fallen quite far behind on astropy development and my knowledge of the finer features of many of Python's libraries is limited because I do most of my work in C/C++. I have a use case in which I have a small (< 100k row) amount of static data which I need to both visualize and be able to easily manipulate (grouping, inclusion/exclusion based upon some criteria, application of some simple mathematics without altering the original data). I've used SQL extensively in the past and the queries which I would run are a perfect use case; however, visualizing the data with matplotlib (or similar) might be more efficiently accomplished from an astropy.table object. Is it possible to run an SQL-like query against an astropy.table object? If not with SQL-like syntax, how else could this be done? For example, say I have a table with the following columns (not the use case, but an easy demo): JD, mag, mag_error I could find all data points with an SNR > 100 within a date range by: SELECT * from table WHERE (JD > value AND JD < value AND mag/mag_err > 100); quite quickly. Conversely, implementing something similar using numpy operations appears to be quite painful (e.g. http://stackoverflow.com/questions/7169240/moving-large-sql-query-to-numpy) unless this is already built-in to astropy.table objects. Kind regards, Brian Kloppenborg From ejensen1 at swarthmore.edu Sat Feb 8 09:00:32 2014 From: ejensen1 at swarthmore.edu (Eric Jensen) Date: Sat, 8 Feb 2014 09:00:32 -0500 Subject: [AstroPy] SQL-like queries against astropy.table objects In-Reply-To: <52F5B11F.60403@gmail.com> References: <52F5B11F.60403@gmail.com> Message-ID: Hi Brian, I think that your use case is actually pretty straightforward in an astropy table, since you just want to pull out a subset of the table. It seems that what's tricky in the stackoverflow example you linked to is the JOIN and GROUPBY parts of the query, not the selection. You can easily select rows that meet multiple criteria, and use the resulting boolean array to create a new table with the subset of data you want. Here's an example, not with your variables but just using a text file of interferometry complex visibilities I had around (U, V, W baseline lengths, real and imaginary parts of the visibility): In [32]: from astropy.table import Table, Column import numpy as np In [33]: t = Table.read('test_data.txt', format='ascii.commented_header') In [34]: t.colnames Out[34]: ['U', 'V', 'W', 'Real', 'Imag', 'Weight', 'Freq'] In [35]: # Select only entries with long baselines in U and W, and Real amplitudes > 1 Jy: good_rows = np.logical_and(t['U'] > 100, t['V'] > 100, t['Real'] > 1) In [42]: # Make a new table from this selection, and see how many rows were in # new table vs. the original one: new_table = t[good_rows] new_table['U'].shape Out[42]: (120,) In [41]: t['U'].shape Out[41]: (4500,) There might be even better ways to do it (and there is almost certainly a cleaner way to get the number of rows in a table than calling 'shape' on one column), but this should give you the basic idea of the sort of selection that it seem like you're looking for. There's also a table-joining function: http://docs.astropy.org/en/latest/api/astropy.table.operations.join.html#astropy.table.operations.join For more complex operations, others who are more familiar with both astropy and SQL can chime in, but thought I'd offer what (little) I know on the selection part. Hope this helps, Eric On Feb 7, 2014, at 11:22 PM, Brian Kloppenborg wrote: > Greetings, > > I have unfortunately fallen quite far behind on astropy development and > my knowledge of the finer features of many of Python's libraries is > limited because I do most of my work in C/C++. > > I have a use case in which I have a small (< 100k row) amount of static > data which I need to both visualize and be able to easily manipulate > (grouping, inclusion/exclusion based upon some criteria, application of > some simple mathematics without altering the original data). I've used > SQL extensively in the past and the queries which I would run are a > perfect use case; however, visualizing the data with matplotlib (or > similar) might be more efficiently accomplished from an astropy.table > object. Is it possible to run an SQL-like query against an astropy.table > object? If not with SQL-like syntax, how else could this be done? > > For example, say I have a table with the following columns (not the use > case, but an easy demo): > JD, mag, mag_error > > I could find all data points with an SNR > 100 within a date range by: > SELECT * from table WHERE (JD > value AND JD < value AND > mag/mag_err > 100); > quite quickly. Conversely, implementing something similar using numpy > operations appears to be quite painful (e.g. > http://stackoverflow.com/questions/7169240/moving-large-sql-query-to-numpy) > unless this is already built-in to astropy.table objects. > > Kind regards, > Brian Kloppenborg > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Sat Feb 8 12:28:27 2014 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Sat, 8 Feb 2014 18:28:27 +0100 Subject: [AstroPy] SQL-like queries against astropy.table objects In-Reply-To: References: <52F5B11F.60403@gmail.com> Message-ID: <070AC50D-2B39-44B2-859B-D7EDA99D52E6@astro.physik.uni-goettingen.de> Hi, I know almost nothing about SQL, but I agree that for basic conditionals that can be used on numpy arrays this really is straightforward. On 08.02.2014, at 3:00PM, Eric Jensen wrote: > In [35]: > > # Select only entries with long baselines in U and W, and Real amplitudes > 1 Jy: > good_rows = np.logical_and(t['U'] > 100, t['V'] > 100, t['Real'] > 1) > In [42]: > > # Make a new table from this selection, and see how many rows were in > # new table vs. the original one: > new_table = t[good_rows] > new_table['U'].shape > This can be written even more concise, e.g. in the OP's example > For example, say I have a table with the following columns (not the use > case, but an easy demo): > JD, mag, mag_error > > I could find all data points with an SNR > 100 within a date range by: > SELECT * from table WHERE (JD > value AND JD < value AND > mag/mag_err > 100); new_table = t[[('JD']>minval)&(t['JD']100)] print(len(t), len(new_table)) directly gives you the desired selection (which works the same with any numpy structured array. Not very painful, even if it might get a bit lengthy to type. I don't think though apt.Table offers a more comfortable way like e.g. pytables does with tbl.where('JD>minval & JD References: <52F5B11F.60403@gmail.com> <070AC50D-2B39-44B2-859B-D7EDA99D52E6@astro.physik.uni-goettingen.de> Message-ID: <52F8F9B3.2050504@stsci.edu> On 02/08/2014 12:28 PM, Derek Homeier wrote: > Hi, > > I know almost nothing about SQL, but I agree that for basic conditionals that can be used > on numpy arrays this really is straightforward. > > On 08.02.2014, at 3:00PM, Eric Jensen wrote: > >> In [35]: >> >> # Select only entries with long baselines in U and W, and Real amplitudes > 1 Jy: >> good_rows = np.logical_and(t['U'] > 100, t['V'] > 100, t['Real'] > 1) >> In [42]: >> >> # Make a new table from this selection, and see how many rows were in >> # new table vs. the original one: >> new_table = t[good_rows] >> new_table['U'].shape >> ... In fairness, for a very *large* table this sort of thing could use up a lot of memory, especially considering that Numpy doesn't use bit arrays for boolean masks :/ Of course the OP said it was a small table so that's beside the point here; but an actual database should provide an efficient row-based iterator with such a selection criteria, even hypothetically over millions of rows. Of course if that's what one needs one should just loader their data into a database :) Erik B. > >> For example, say I have a table with the following columns (not the use >> case, but an easy demo): >> JD, mag, mag_error >> >> I could find all data points with an SNR > 100 within a date range by: >> SELECT * from table WHERE (JD > value AND JD < value AND >> mag/mag_err > 100); > > new_table = t[[('JD']>minval)&(t['JD']100)] > print(len(t), len(new_table)) > > directly gives you the desired selection (which works the same with any numpy structured array. > Not very painful, even if it might get a bit lengthy to type. I don't think though apt.Table offers a more > comfortable way like e.g. pytables does with tbl.where('JD>minval & JD > I am sure there's more powerful possibilities in SQL that are not as easy to reproduce; > operating on string columns is a bit more inconvenient, since numpy does not offer a lot of operations > working on arrays of str type; in this case I think you have to resort to python loops, but in this case > astropy Tables add_column[s] helps a lot over plain numpy. I have not explored the grouping methods > either. > > HTH, > Derek > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From aldcroft at head.cfa.harvard.edu Mon Feb 10 12:52:11 2014 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Mon, 10 Feb 2014 12:52:11 -0500 Subject: [AstroPy] SQL-like queries against astropy.table objects In-Reply-To: References: <52F5B11F.60403@gmail.com> Message-ID: On Sat, Feb 8, 2014 at 9:00 AM, Eric Jensen wrote: > Hi Brian, > > I think that your use case is actually pretty straightforward in an > astropy table, since you just want to pull out a subset of the table. It > seems that what's tricky in the stackoverflow example you linked to is the > JOIN and GROUPBY parts of the query, not the selection. > > You can easily select rows that meet multiple criteria, and use the > resulting boolean array to create a new table with the subset of data you > want. Here's an example, not with your variables but just using a text > file of interferometry complex visibilities I had around (U, V, W baseline > lengths, real and imaginary parts of the visibility): > > In [32]: > > from astropy.table import Table, Column > > import numpy as np > > In [33]: > > t = Table.read('test_data.txt', format='ascii.commented_header') > > In [34]: > > t.colnames > > Out[34]: > > ['U', 'V', 'W', 'Real', 'Imag', 'Weight', 'Freq'] > > In [35]: > > # Select only entries with long baselines in U and W, and Real amplitudes > 1 Jy: > > good_rows = np.logical_and(t['U'] > 100, t['V'] > 100, t['Real'] > 1) > > In [42]: > > # Make a new table from this selection, and see how many rows were in > > # new table vs. the original one: > > new_table = t[good_rows] > > new_table['U'].shape > > > Out[42]: > > (120,) > > In [41]: > > t['U'].shape > > Out[41]: > > (4500,) > > > There might be even better ways to do it (and there is almost certainly a > cleaner way to get the number of rows in a table than calling 'shape' on > one column), but this should give you the basic idea of the sort of > selection that it seem like you're looking for. > > There's also a table-joining function: > > > http://docs.astropy.org/en/latest/api/astropy.table.operations.join.html#astropy.table.operations.join > > For more complex operations, others who are more familiar with both > astropy and SQL can chime in, but thought I'd offer what (little) I know on > the selection part. > In addition to joins, astropy table has some support for group_by and table concatenation (by rows or columns): http://astropy.readthedocs.org/en/latest/table/operations.html For tables that can be manipulated entirely in memory this should be reasonably efficient. Simple selections with boolean arrays as shown earlier are quite fast and efficient. Cheers, Tom > > Hope this helps, > > Eric > > > > On Feb 7, 2014, at 11:22 PM, Brian Kloppenborg wrote: > > Greetings, > > I have unfortunately fallen quite far behind on astropy development and > my knowledge of the finer features of many of Python's libraries is > limited because I do most of my work in C/C++. > > I have a use case in which I have a small (< 100k row) amount of static > data which I need to both visualize and be able to easily manipulate > (grouping, inclusion/exclusion based upon some criteria, application of > some simple mathematics without altering the original data). I've used > SQL extensively in the past and the queries which I would run are a > perfect use case; however, visualizing the data with matplotlib (or > similar) might be more efficiently accomplished from an astropy.table > object. Is it possible to run an SQL-like query against an astropy.table > object? If not with SQL-like syntax, how else could this be done? > > For example, say I have a table with the following columns (not the use > case, but an easy demo): > JD, mag, mag_error > > I could find all data points with an SNR > 100 within a date range by: > SELECT * from table WHERE (JD > value AND JD < value AND > mag/mag_err > 100); > quite quickly. Conversely, implementing something similar using numpy > operations appears to be quite painful (e.g. > http://stackoverflow.com/questions/7169240/moving-large-sql-query-to-numpy) > > unless this is already built-in to astropy.table objects. > > Kind regards, > Brian Kloppenborg > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gb.gabrielebrambilla at gmail.com Mon Feb 10 13:25:46 2014 From: gb.gabrielebrambilla at gmail.com (Gabriele Brambilla) Date: Mon, 10 Feb 2014 13:25:46 -0500 Subject: [AstroPy] Fwd: basemap wrapimage In-Reply-To: References: Message-ID: I am trying to don't save the figures of the plot but to plot them automatically on a basemap. this the code I'm trying to run: fiq = plt.figure() axf = fiq.add_subplot(111) imh = axf.imshow(B) fiq.canvas.draw() #trying to build a map degree = list(range(360)) for d in degree: map = Basemap(projection='ortho', lat_0 = 0, lon_0 = d, resolution = 'l', area_thresh = 1000.) map.warpimage(image=imh) map.drawmapboundary() title = str(d)+'.png' map.savefig(title) but it doesn't work. it returns Traceback (most recent call last): File "building3d.py", line 103, in map.warpimage(image=imh) File "C:\Anaconda\lib\site-packages\mpl_toolkits\basemap\__init__.py", line 40 06, in warpimage if file.startswith('http'): AttributeError: 'AxesImage' object has no attribute 'startswith' And I don't find many tutorials on the web... thanks Gabriele -------------- next part -------------- An HTML attachment was scrubbed... URL: From harrold at astro.as.utexas.edu Tue Feb 11 19:46:29 2014 From: harrold at astro.as.utexas.edu (Samuel Harrold) Date: Tue, 11 Feb 2014 18:46:29 -0600 Subject: [AstroPy] astropy stability for time-series differential photometry pipeline Message-ID: Hello, My research group at UT Austin is considering migrating our data reduction pipeline from IRAF/PyRAF to astropy. We do time-series differential photometry for asteroseismology, and because we use our data for many years after we've processed it, we'd like our pipeline to be stable for 5-10 years. We're unsure if we should wait several years for the astropy collaboration to release a paper demonstrating comparisons with IRAF/PyRAF/ESO-MIDAS/other, or if we should migrate now. We have a few questions about the stability of astropy before we attempt to migrate. We greatly appreciate any advice: - In the long term, is STSCI replacing IRAF/PyRAF with astropy? - If we migrate to astropy now, how often would we need to incorporate updates to astropy, use non-astropy packages, and/or make our own work-arounds? Our key pipeline needs are: - - Operations are done on ~1000 - 5000 images at a time. (We take sequences of ~10-second exposures for many hours at a time, following the target from horizon to horizon.) - - Bias, dark, flat calibration - - Align calibrated images (to correct for bad tracking) - - Circular aperture photometry of target star and comparison stars - - Correct timestamps to BJD with accuracy to better than 0.01s. (Our instrument's timestamps are controlled by GPS. Light curves are combined with others taken over ~20 years.) Thanks for all your help, and thanks for making such a useful tool! Sam Harrold ---------- Samuel Harrold PhD Candidate Astronomy Department University of Texas at Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.vickroy at noaa.gov Wed Feb 12 11:06:41 2014 From: jim.vickroy at noaa.gov (jim vickroy) Date: Wed, 12 Feb 2014 09:06:41 -0700 Subject: [AstroPy] Solar Limb Darkening Removal? Message-ID: <52FB9C11.1050805@noaa.gov> Hi, Is there any existing Python software for the removal of Solar Limb darkening? My on-line search has failed to find anything so far. Thanks, -- jv -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuart at mumford.me.uk Wed Feb 12 12:17:38 2014 From: stuart at mumford.me.uk (Stuart Mumford) Date: Wed, 12 Feb 2014 17:17:38 +0000 Subject: [AstroPy] Solar Limb Darkening Removal? In-Reply-To: <52FB9C11.1050805@noaa.gov> References: <52FB9C11.1050805@noaa.gov> Message-ID: Hi, You might be interested in SunPy (http://sunpy.org). We don't have limb darkening code at the moment but we would be willing to help you out if you wanted to add it. Stuart On 12 February 2014 16:06, jim vickroy wrote: > Hi, > > Is there any existing Python software for the removal of Solar Limb > darkening? My on-line search has failed to find anything so far. > > Thanks, > -- jv > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.vickroy at noaa.gov Wed Feb 12 13:06:53 2014 From: jim.vickroy at noaa.gov (jim vickroy) Date: Wed, 12 Feb 2014 11:06:53 -0700 Subject: [AstroPy] {SunPy} Re: Solar Limb Darkening Removal? In-Reply-To: <184FE51A-4AB6-48B0-A1BC-747C29826341@gmail.com> References: <52FB9C11.1050805@noaa.gov> <184FE51A-4AB6-48B0-A1BC-747C29826341@gmail.com> Message-ID: <52FBB83D.6050900@noaa.gov> On 2/12/2014 10:38 AM, Steven Christe wrote: > Hi Jim, I am actually working on this problem currently and have implemented a solution in SunPy. It is currently in a branch on my fork called hmi_fix which you can find here > > https://github.com/ehsteve/sunpy/blob/hmi_fix/sunpy/map/sources/sdo.py > > under the function called _angle_of_incidence > > This function clearly belongs in a higher level than the hmi map class. Let?s discuss what your needs are, what data you are working on, and we can work how we can best help you. Hello Steven, Thanks for your interest. That is good news! I'm working on automated Solar features classification which uses mono-chromatic Solar images at various wavelengths. Presently, the classifier analyzes SDO-AIA (FITS) images, but we want to include H-alpha (FITS) images (gong2.nso.edu) to hopefully improve Sunspot and filament detection. However, the limb-darkening, of the H-alpha images, is proving problematic. When GOES-R launches, we will switch to SUVI as the UV images source. Hopefully, I've answered your questions. What is the next step here? Shall I download your code and start to play? -- jv On Feb 12, 2014, at 12:17 PM, Stuart Mumford wrote: >> Hi, >> >> You might be interested in SunPy (http://sunpy.org). >> >> We don't have limb darkening code at the moment but we would be willing to help you out if you wanted to add it. >> >> Stuart >> >> >> On 12 February 2014 16:06, jim vickroy wrote: >> Hi, >> >> Is there any existing Python software for the removal of Solar Limb darkening? My on-line search has failed to find anything so far. >> >> Thanks, >> -- jv >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups "SunPy" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to sunpy+unsubscribe at googlegroups.com. >> To post to this group, send email to sunpy at googlegroups.com. >> Visit this group at http://groups.google.com/group/sunpy. >> For more options, visit https://groups.google.com/groups/opt_out. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.vickroy at noaa.gov Wed Feb 12 13:15:50 2014 From: jim.vickroy at noaa.gov (jim vickroy) Date: Wed, 12 Feb 2014 11:15:50 -0700 Subject: [AstroPy] Solar Limb Darkening Removal? In-Reply-To: References: <52FB9C11.1050805@noaa.gov> Message-ID: <52FBBA56.7000404@noaa.gov> On 2/12/2014 10:17 AM, Stuart Mumford wrote: > Hi, > > You might be interested in SunPy (http://sunpy.org). > > We don't have limb darkening code at the moment but we would be > willing to help you out if you wanted to add it. > > Stuart Hello Stuart, Thanks for your response. I visited http://sunpy.org/help/#list but could not see how to join either of the two lists mentioned. Apologies for being dense on this one. -- jv > > > On 12 February 2014 16:06, jim vickroy > wrote: > > Hi, > > Is there any existing Python software for the removal of Solar > Limb darkening? My on-line search has failed to find anything so far. > > Thanks, > -- jv > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuart at mumford.me.uk Wed Feb 12 13:21:04 2014 From: stuart at mumford.me.uk (Stuart Mumford) Date: Wed, 12 Feb 2014 18:21:04 +0000 Subject: [AstroPy] Solar Limb Darkening Removal? In-Reply-To: <52FBBA56.7000404@noaa.gov> References: <52FB9C11.1050805@noaa.gov> <52FBBA56.7000404@noaa.gov> Message-ID: Ooops! should be links there: https://groups.google.com/forum/#!forum/sunpy https://groups.google.com/forum/#!forum/sunpy-dev On 12 February 2014 18:15, jim vickroy wrote: > On 2/12/2014 10:17 AM, Stuart Mumford wrote: > > Hi, > > You might be interested in SunPy (http://sunpy.org). > > We don't have limb darkening code at the moment but we would be willing > to help you out if you wanted to add it. > > Stuart > > > Hello Stuart, > > Thanks for your response. > > I visited http://sunpy.org/help/#list but could not see how to join > either of the two lists mentioned. Apologies for being dense on this one. > > -- jv > > > > > > On 12 February 2014 16:06, jim vickroy wrote: > >> Hi, >> >> Is there any existing Python software for the removal of Solar Limb >> darkening? My on-line search has failed to find anything so far. >> >> Thanks, >> -- jv >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shashankshalgar at unm.edu Wed Feb 12 16:03:43 2014 From: shashankshalgar at unm.edu (Shashank Shalgar) Date: Wed, 12 Feb 2014 14:03:43 -0700 Subject: [AstroPy] Query regarding astropy.cosmology Message-ID: Hello, I just found out about astropy module and it looking really useful. I am especially interested in the cosmology module. However, I did not find much documentation on the analysis of CMB anisotropies. I am not sure whether the analysis of CMB anisotropies is even possible with astropy.cosmology. I would like to know whether it is possible to get the anisotropy data as tables in the module as a numpy array so that I can wrote functions to extract information from it. For eg. correlation length, three point correlations etc. Thanks in advance. Shashank -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alexander.Conley at Colorado.EDU Wed Feb 12 16:15:08 2014 From: Alexander.Conley at Colorado.EDU (Alexander Joseph Conley) Date: Wed, 12 Feb 2014 14:15:08 -0700 Subject: [AstroPy] Query regarding astropy.cosmology In-Reply-To: References: Message-ID: <88329F01-E6E6-476A-A84E-544A4AD2073F@colorado.edu> Hi Shashank, No, astropy.cosmology is not intended to do real analysis of the CMB anisotropies. That would probably be considered too specialized for astropy (although it could make an interesting affiliated package if somebody wanted to develop such a module: http://www.astropy.org/affiliated/) Alex On 12-Feb-2014, at 2:03 PM, Shashank Shalgar > wrote: Hello, I just found out about astropy module and it looking really useful. I am especially interested in the cosmology module. However, I did not find much documentation on the analysis of CMB anisotropies. I am not sure whether the analysis of CMB anisotropies is even possible with astropy.cosmology. I would like to know whether it is possible to get the anisotropy data as tables in the module as a numpy array so that I can wrote functions to extract information from it. For eg. correlation length, three point correlations etc. Thanks in advance. Shashank _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From cf5g09 at soton.ac.uk Fri Feb 14 10:09:55 2014 From: cf5g09 at soton.ac.uk (frohmaier c. (cf5g09)) Date: Fri, 14 Feb 2014 15:09:55 +0000 Subject: [AstroPy] all_pix2world() NULL error Message-ID: Hi, When attempting to run all_pix2world() I get the following error: ERROR: RuntimeError: NULL error object in wcslib I cannot find any reference to this in the Astropy documentation. Does anyone know how I can resolve this? Cheers, Chris From stuart at mumford.me.uk Fri Feb 14 12:36:11 2014 From: stuart at mumford.me.uk (Stuart Mumford) Date: Fri, 14 Feb 2014 17:36:11 +0000 Subject: [AstroPy] SunPy 0.4.0 Released Message-ID: Hello everyone, The SunPy community is pleased to announce the release of SunPy 0.4.0. This release contains many new features, some of which are contributed by people who participated in GSOC 2013. It includes the addition of a new local database for storing and searching data, it features a HEK to VSO translator and a new HELIO module in net. As well as this major work has been undertaken on the documentation and a new website developed. To download the lastest SunPy release go to our website http://sunpy.org or download from PyPI: https://pypi.python.org/pypi/sunpy. The documentation for the latest release can now be found at http://docs.sunpy.org/en/stable/http://docs.sunpy.org/en/stable/ This release contains 1025 commits from 19 people. New Features: - *Major *documentation refactor. A far reaching re-write and restructure. - Add a SunPy Database to store and search local data. - Add beta support for querying the HELIO HEC - Add beta HEK to VSO query translation. - Add the ability to download the GOES event list. - Add support for downloading and querying the LYTAF database. - Add support for ANA data. - Updated sun.constants to use astropy.constants objects which include units, source, and error instide. For more info check out http://docs.astropy.org/en/latest/constants/index.html - Add some beta support for IRIS data products - Add a new MapCubeAnimator class with interactive widgets which is returned by mapcube.peek(). - The Glymur library is now used to read JPEG2000 files. - GOESLightCurve now supports all GOES satellites. The people who have contributed to this release are: Stuart Mumford Simon Liedtke Steven Christe Jack Ireland Andrew Inglis Nabil Freij Samuel Bennett David Perez-Suarez Pritish Chakraborty Albert Y. Shih John Evans Michael Malocha Florian Mayer Russell Hewett Jose Iv?n Campos Keith Hughitt Tiago Pereira -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Fri Feb 14 13:00:14 2014 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Fri, 14 Feb 2014 13:00:14 -0500 Subject: [AstroPy] astropy stability for time-series differential photometry pipeline In-Reply-To: References: Message-ID: Hello Sam, I'll try to answer your questions below: > - In the long term, is STSCI replacing IRAF/PyRAF with astropy? I certainly can't speak for STScI, but I've gotten the impression that is the intent (or at least a combination of astropy and more specific in-house python packages they develop). Perry certainly can make a more definitive statement about this, though. > - If we migrate to astropy now, how often would we need to incorporate updates to astropy, use non-astropy packages, and/or make our own work-arounds? As I've explained below, some of what you're looking for is in Astropy "affiliated packages". These are packages that are not part of the actual Astropy code base, but use the Astropy frameworks or are otherwise affiliated with the community. Some of it may eventually end up in the astropy core package, but we're intentionally leaving it out for now because there's a lot of development left to be done, and we want to give this stuff the opportunity to develop de-coupled from the core. They generally don't have fixed release cycles, so I can't say for sure how often you'd need updates. I think for the purposes you've described, though, the API's are/will be fairly stable, so you probably would only need to update if there's a bug fix that is important for your specific work. For the Astropy core itself, we have a release cycle every 6 months, which may introduce backwards-incompatible changes (which would require updates for you), but we are also planning "LTS" releases every 2 years. These will be supported with bugfixes for at least 2 years while remaining backwards-compatible. We're also planning to do "soft support" for even longer - e.g., if you're using an LTS that's 3 years old, and find that a crucial bug affects your work, we'd be filling to just fix that and do a bugfix release. Similarly to the affiliated packages, though, the needs you've listed in the core probably won't change substantially once it's in, so you are probably safe just upgrading to the latest version if you want a new feature. > - - Operations are done on ~1000 - 5000 images at a time. (We take sequences of ~10-second exposures for many hours at a time, following the target from horizon to horizon.) > - - Bias, dark, flat calibration > - - Align calibrated images (to correct for bad tracking) > - - Circular aperture photometry of target star and comparison stars All of this is not in the Astropy core package. Some of it is in an astropy affiliated package called `photutils`, though (https://github.com/astropy/photutils), and the rest is in a affiliated package that I believe is soon-to-be-released (something like `ccdproc`). > - - Correct timestamps to BJD with accuracy to better than 0.01s. (Our instrument's timestamps are controlled by GPS. Light curves are combined with others taken over ~20 years.) The `astropy.time.Time` object (for representing timestamps) I believe is at msec accuracy now, but I can't remember know if there's a BJD correction in there or not. Marten or Tom A probably know, though (cced here). Hopefully that helps! > Thanks for all your help, and thanks for making such a useful tool! > Sam Harrold > ---------- > Samuel Harrold > PhD Candidate > Astronomy Department > University of Texas at Austin > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Erik From mdroe at stsci.edu Fri Feb 14 15:15:30 2014 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 14 Feb 2014 15:15:30 -0500 Subject: [AstroPy] all_pix2world() NULL error In-Reply-To: References: Message-ID: <52FE7962.5070103@stsci.edu> Can you provide a standalone example and data that reproduces the error? This indicates that wcslib set an error flag on an operation but didn't actually set any information about the error. This really shouldn't happen in normal usage except if there's some latent bug in wcslib or astropy.wcs... but I'll need to be able to reproduce it to be sure. Mike On 02/14/2014 10:09 AM, frohmaier c. (cf5g09) wrote: > Hi, > > When attempting to run all_pix2world() I get the following error: > > ERROR: RuntimeError: NULL error object in wcslib > > I cannot find any reference to this in the Astropy documentation. Does anyone know how I can resolve this? > > Cheers, > Chris > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- _ |\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _ | ||(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | | http://www.droettboom.com From cf5g09 at soton.ac.uk Sun Feb 16 06:49:49 2014 From: cf5g09 at soton.ac.uk (frohmaier c. (cf5g09)) Date: Sun, 16 Feb 2014 11:49:49 +0000 Subject: [AstroPy] all_pix2world() NULL error In-Reply-To: <52FE7962.5070103@stsci.edu> References: <52FE7962.5070103@stsci.edu> Message-ID: <9D70C517-5D76-4EEE-9FDA-B33841B78190@soton.ac.uk> Hi Mike, I?m using PTF data, here is an example image: https://www.dropbox.com/s/yc7lz51ar56a68f/PTF201203051940_2_o_62108_00.w.fits My steps to reproduce the problem are: >>> hdulist=fits.open('PTF201203051940_2_o_62108_00.w.fits') >>> worldc=wcs.WCS(hdulist[0].header) >>> pixcrd=numpy.array([[536.0,894.0]],numpy.float_) >>> cords=worldc.all_pix2world(pixcrd,0) ERROR: RuntimeError: NULL error object in wcslib [astropy.wcs.wcs] Many thanks, Chris On 14 Feb 2014, at 20:15, Michael Droettboom wrote: > Can you provide a standalone example and data that reproduces the error? > > This indicates that wcslib set an error flag on an operation but didn't > actually set any information about the error. This really shouldn't > happen in normal usage except if there's some latent bug in wcslib or > astropy.wcs... but I'll need to be able to reproduce it to be sure. > > Mike > > On 02/14/2014 10:09 AM, frohmaier c. (cf5g09) wrote: >> Hi, >> >> When attempting to run all_pix2world() I get the following error: >> >> ERROR: RuntimeError: NULL error object in wcslib >> >> I cannot find any reference to this in the Astropy documentation. Does anyone know how I can resolve this? >> >> Cheers, >> Chris >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > -- > _ > |\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _ > | ||(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | | > > http://www.droettboom.com > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From ghang.naoc at gmail.com Sun Feb 16 09:51:54 2014 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Sun, 16 Feb 2014 22:51:54 +0800 Subject: [AstroPy] how to determine whether a point was observed by hst ever? Message-ID: hi, If there is just one point, we can check stsci. How about many points? I just need a yes or no. Of course an observation log is even better. Even we input a position and the output list is not null in stsci, there is not necessarily archive data covered that point because of the search radius and the shape of hst' fov. Did anybody write a similar script before? How do you check the archive data? A web crawler is necessary? Thank you. hang -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Sun Feb 16 15:16:44 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Sun, 16 Feb 2014 21:16:44 +0100 Subject: [AstroPy] how to determine whether a point was observed by hst ever? In-Reply-To: References: Message-ID: Hi Hang, You can query the HST archive from Python using the following URL: http://archive.stsci.edu/hst/search.php?RA=12.2&DEC=13.1&radius=100&max_records=1000&outputformat=CSV&action=Search modify RA, Dev, and radius as needed. I'm not sure what the latest radius and maximum number of records is, but you could always try and write a script that will run this for the whole sky by looping over RA and Dec. Cheers, Tom On 16 February 2014 15:51, gonghang.naoc wrote: > hi, > If there is just one point, we can check stsci. How about many points? > I just need a yes or no. Of course an observation log is even better. > Even we input a position and the output list is not null in stsci, there is > not necessarily archive data covered that point because of the search radius > and the shape of hst' fov. > Did anybody write a similar script before? How do you check the archive > data? A web crawler is necessary? > > Thank you. > hang > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From ghang.naoc at gmail.com Sun Feb 16 22:20:45 2014 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Mon, 17 Feb 2014 11:20:45 +0800 Subject: [AstroPy] how to determine whether a point was observed by hst ever? In-Reply-To: References: Message-ID: hi, Thank you. I tried one position with a same ra&dec&radius and it seems there is a big difference between that url output and the stsci output. The fits names are different. So I use observation time,but the time is not the same at all either. hang On Mon, Feb 17, 2014 at 4:16 AM, Thomas Robitaille < thomas.robitaille at gmail.com> wrote: > Hi Hang, > > You can query the HST archive from Python using the following URL: > > > http://archive.stsci.edu/hst/search.php?RA=12.2&DEC=13.1&radius=100&max_records=1000&outputformat=CSV&action=Search > > modify RA, Dev, and radius as needed. I'm not sure what the latest > radius and maximum number of records is, but you could always try and > write a script that will run this for the whole sky by looping over RA > and Dec. > > Cheers, > Tom > > > On 16 February 2014 15:51, gonghang.naoc wrote: > > hi, > > If there is just one point, we can check stsci. How about many points? > > I just need a yes or no. Of course an observation log is even better. > > Even we input a position and the output list is not null in stsci, there > is > > not necessarily archive data covered that point because of the search > radius > > and the shape of hst' fov. > > Did anybody write a similar script before? How do you check the archive > > data? A web crawler is necessary? > > > > Thank you. > > hang > > > > > > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Tue Feb 18 12:34:41 2014 From: mdroe at stsci.edu (Michael Droettboom) Date: Tue, 18 Feb 2014 12:34:41 -0500 Subject: [AstroPy] all_pix2world() NULL error In-Reply-To: <9D70C517-5D76-4EEE-9FDA-B33841B78190@soton.ac.uk> References: <52FE7962.5070103@stsci.edu> <9D70C517-5D76-4EEE-9FDA-B33841B78190@soton.ac.uk> Message-ID: <530399B1.3080101@stsci.edu> Indeed, there was an bug in propagating the error message back to Python in |astropy.wcs|. A fix for that is now in this pull request: https://github.com/astropy/astropy/pull/2106 As a workaround until this fix is merged into a release, you can explicitly call |.wcs.set()| on the |WCS| object after creating it (astropy 0.4 already does this implicitly anyway). That reveals the actual error in the WCS header: |astropy.wcs._wcs.InvalidTransformError: ERROR 6 in wcsset() at line 1562 of file cextern/wcslib/C/wcs.c: PV1_5 : Unrecognized coordinate transformation parameter. | This is because the WCS specifies TAN projections, which don?t have any additional PV parameters, yet this header has them. I?ve seen this in headers generated by SCAMP before, and we have a fix this technically non-standard construction in this pull request (which will make it into astropy 0.4): https://github.com/astropy/astropy/pull/1278 Mike On 02/16/2014 06:49 AM, frohmaier c. (cf5g09) wrote: > Hi Mike, > > I?m using PTF data, here is an example image: https://www.dropbox.com/s/yc7lz51ar56a68f/PTF201203051940_2_o_62108_00.w.fits > > My steps to reproduce the problem are: >>>> hdulist=fits.open('PTF201203051940_2_o_62108_00.w.fits') >>>> worldc=wcs.WCS(hdulist[0].header) >>>> pixcrd=numpy.array([[536.0,894.0]],numpy.float_) >>>> cords=worldc.all_pix2world(pixcrd,0) > ERROR: RuntimeError: NULL error object in wcslib [astropy.wcs.wcs] > > Many thanks, > > Chris > > On 14 Feb 2014, at 20:15, Michael Droettboom wrote: > >> Can you provide a standalone example and data that reproduces the error? >> >> This indicates that wcslib set an error flag on an operation but didn't >> actually set any information about the error. This really shouldn't >> happen in normal usage except if there's some latent bug in wcslib or >> astropy.wcs... but I'll need to be able to reproduce it to be sure. >> >> Mike >> >> On 02/14/2014 10:09 AM, frohmaier c. (cf5g09) wrote: >>> Hi, >>> >>> When attempting to run all_pix2world() I get the following error: >>> >>> ERROR: RuntimeError: NULL error object in wcslib >>> >>> I cannot find any reference to this in the Astropy documentation. Does anyone know how I can resolve this? >>> >>> Cheers, >>> Chris >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> -- >> _ >> |\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _ >> | ||(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | | >> >> http://www.droettboom.com >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy -- _ |\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _ | ||(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | | http://www.droettboom.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinberoiz at gmail.com Wed Feb 19 17:33:39 2014 From: martinberoiz at gmail.com (Martin Beroiz) Date: Wed, 19 Feb 2014 16:33:39 -0600 Subject: [AstroPy] example on phot class on astropysics Message-ID: Hello all, I'm trying to do simple photometry on some images using the phot clases on astropysics. I read this page: http://pythonhosted.org/Astropysics/coremods/phot.html#astropysics.phot.PhotObservation and I tried some simple calls like: >>> import astropysics.phot as ap >>> gs = ap.GaussianPointSpreadFunction() >>> cent, sig = gs.fit(sc) (Here sc is a 20x20 numpy array of float32's) Does anyone have a simple example on how to do some aperture photometry or the profile of the star? I don't seem to find any tool to do that. Also, >>> ie = ap.IsophotalEllipse(sc) >>> ie.b Gives the error: TypeError: Improper input: N=5 must not exceed M=4 ERROR: TypeError: Improper input: N=5 must not exceed M=4 [scipy.optimize.minpack] for the same image, what could that be? Thanks! M. From erik.tollerud at gmail.com Thu Feb 20 20:08:02 2014 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Thu, 20 Feb 2014 20:08:02 -0500 Subject: [AstroPy] example on phot class on astropysics In-Reply-To: References: Message-ID: Hello Martin, If you're looking for aperture photometry, I'd suggest photutils ( http://photutils.readthedocs.org/en/latest/photutils/aperture.html). That's where development will be in the future, and it's probably already better for aperture photometry than astropysics has ever been or will be (I can say that, because I'm the author ;). If PSF photometry is more what you're looking for, you might try out functionality that's due to get merge in photutils ( https://github.com/astropy/photutils/pull/26 , with a demo at http://nbviewer.ipython.org/gist/adonath/6550989/PSFPhotometrySpitzer.ipynb ). Hope that helps... On Wed, Feb 19, 2014 at 5:33 PM, Martin Beroiz wrote: > Hello all, > > I'm trying to do simple photometry on some images using the phot clases on > astropysics. > I read this page: > > http://pythonhosted.org/Astropysics/coremods/phot.html#astropysics.phot.PhotObservation > > and I tried some simple calls like: > > >>> import astropysics.phot as ap > >>> gs = ap.GaussianPointSpreadFunction() > >>> cent, sig = gs.fit(sc) > > (Here sc is a 20x20 numpy array of float32's) > > Does anyone have a simple example on how to do some aperture photometry or > the profile of the star? I don't seem to find any tool to do that. > > Also, > > >>> ie = ap.IsophotalEllipse(sc) > >>> ie.b > > Gives the error: > TypeError: Improper input: N=5 must not exceed M=4 > > ERROR: TypeError: Improper input: N=5 must not exceed M=4 > [scipy.optimize.minpack] > > for the same image, what could that be? > > > > Thanks! > M. > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Erik -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewturk at gmail.com Wed Feb 19 20:40:11 2014 From: matthewturk at gmail.com (Matthew Turk) Date: Wed, 19 Feb 2014 17:40:11 -0800 Subject: [AstroPy] ANN: SciPy 2014 Conference, July 6th - 12th, Austin, TX! Message-ID: Hello All! I am pleased to announce that SciPy 2014, the thirteenth annual Scientific Computing with Python conference, will be held this July 6th-12th in Austin, Texas. SciPy is a community dedicated to the advancement of scientific computing through open source Python software for mathematics, science, and engineering. The annual SciPy Conference allows participants from all types of organizations to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. For more information please visit our website: https://conference.scipy.org/scipy2014/ This year the conference has been extended to include an additional day of presentations. During the presentation days SciPy is proud to host the following event and talk types: Keynotes Expert Panels Short Talks Poster Presentations Birds of a Feather Sessions The full program will consist of two days of tutorials by followed by three days of presentations, and concludes with two days of developer sprints on projects of interest to attendees. This year, we are excited to present a job fair for the first time! Specialized Tracks This year we are happy to announce two specialized tracks that run in parallel to the general conference: Scientific Computing in Education Thanks to efforts such as Software Carpentry, the Hacker Within and grassroots Python Bootcamps, teaching scientific computing as a discipline is becoming more widely accepted and recognized as a crucial task in developing scientific literacy. This special track will focus on efforts to promote and develop scientific computing education, as well as related topics such as reproducibility and best practices for scientific computing. Geospatial Data in Science Python has become a core component of organiziing, understanding, and visualizing geospatial data. This track will focus on libraries, tools and techniques for processing Geospatial data of all types and for all purposes -- from low-volume to high-volume, local and global. Domain-specific Mini-symposia Introduced in 2012, mini-symposia are held to discuss scientific computing applied to a specific scientific domain/industry during a half afternoon after the general conference. Their goal is to promote industry specific libraries and tools, and gather people with similar interests for discussions. Mini-symposia on the following topics will take place this year: Astronomy and astrophysics Bioinformatics Geophysics Vision, Visualization, and Imaging Computational Social Science and Digital Humanities Engineering Tutorials Multiple interactive half-day tutorials will be taught by community experts. The tutorials provide conceptual and practical coverage of tools that have broad interest at both an introductory or advanced level. This year, a third track will be added, which target specifically programmers with no prior knowledge of scientific python. Developer Sprints A hackathon environment is setup for attendees to work on the core SciPy packages or their own personal projects. Theconference is an opportunity for developers that are usually physically separated to come together and engage in highly productive sessions. It is also an occasion for new community members to introduce themselves and receive tips from community experts. This year, some of the sprints will be scheduled and announced ahead of the conference. Birds-of-a-Feather (BOF) Sessions Birds-of-a-Feather sessions are self-organized discussions that run parallel to the main conference. The BOFs sessions cover primary, tangential, or unrelated topics in an interactive, discussion setting. This year, some of the BOF sessions will be scheduled and announced ahead of the conference. Important Dates March 14th: Presentation abstracts, poster, tutorial submission deadline. Application for sponsorship deadline. April 17th: Speakers selected April 22nd: Sponsorship acceptance deadline May 1st: Speaker schedule announced May 6th, or 150 registrants: Early-bird registration ends July 6-12th: 2 days of tutorials, 3 days of conference, 2 days of sprints We look forward to a very exciting conference and hope to see you all in Austin this summer! The SciPy2014 Organizers From martinberoiz at gmail.com Fri Feb 21 12:33:20 2014 From: martinberoiz at gmail.com (martin) Date: Fri, 21 Feb 2014 11:33:20 -0600 Subject: [AstroPy] example on phot class on astropysics In-Reply-To: References: Message-ID: <723C2AD0-191B-447D-BE06-35D9C7EB6EC0@gmail.com> Thanks! this is what I was looking for. On Feb 20, 2014, at 7:08 PM, Erik Tollerud wrote: > Hello Martin, > > If you're looking for aperture photometry, I'd suggest photutils (http://photutils.readthedocs.org/en/latest/photutils/aperture.html). That's where development will be in the future, and it's probably already better for aperture photometry than astropysics has ever been or will be (I can say that, because I'm the author ;). > > If PSF photometry is more what you're looking for, you might try out functionality that's due to get merge in photutils (https://github.com/astropy/photutils/pull/26 , with a demo at http://nbviewer.ipython.org/gist/adonath/6550989/PSFPhotometrySpitzer.ipynb). > > Hope that helps... > > > > On Wed, Feb 19, 2014 at 5:33 PM, Martin Beroiz wrote: > Hello all, > > I'm trying to do simple photometry on some images using the phot clases on astropysics. > I read this page: > http://pythonhosted.org/Astropysics/coremods/phot.html#astropysics.phot.PhotObservation > > and I tried some simple calls like: > > >>> import astropysics.phot as ap > >>> gs = ap.GaussianPointSpreadFunction() > >>> cent, sig = gs.fit(sc) > > (Here sc is a 20x20 numpy array of float32's) > > Does anyone have a simple example on how to do some aperture photometry or the profile of the star? I don't seem to find any tool to do that. > > Also, > > >>> ie = ap.IsophotalEllipse(sc) > >>> ie.b > > Gives the error: > TypeError: Improper input: N=5 must not exceed M=4 > > ERROR: TypeError: Improper input: N=5 must not exceed M=4 [scipy.optimize.minpack] > > for the same image, what could that be? > > > > Thanks! > M. > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Erik -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr.maravelias at gmail.com Tue Feb 25 09:34:20 2014 From: gr.maravelias at gmail.com (Grigoris Maravelias) Date: Tue, 25 Feb 2014 16:34:20 +0200 Subject: [AstroPy] convert equinox issue with nicmos/radio data? Message-ID: <530CA9EC.10002@gmail.com> Hello to all! A colleague of mine experiences a problem with astropy when she tries to plot a nicmos image combined with radio contours. Although the scripts runs flawlessly with most objects, when it fails the error output is this: ERROR: Exception: Cannot convert between equatorial coordinates for equinoxes j2000 and j2000 [aplpy.wcs_util] Traceback (most recent call last): File "./nicmos.py", line 98, in f.show_contour(a149, colors="black", levels=np.logspace(math.log10(sa[hrs]),math.log10(maxa[hrs]), num=10),linestyles=['--'],slices=[0,0],linewidths=1) File "", line 2, in show_contour File "/Library/Python/2.7/site-packages/aplpy/decorators.py", line 25, in _auto_refresh f(*args, **kwargs) File "/Library/Python/2.7/site-packages/aplpy/aplpy.py", line 916, in show_contour contour_util.transform(c, wcs_contour, self._wcs, filled=filled, overlap=overlap) File "/Library/Python/2.7/site-packages/aplpy/contour_util.py", line 26, in transform output=(system_out, equinox_out)) File "/Library/Python/2.7/site-packages/aplpy/wcs_util.py", line 213, in convert_coords raise Exception("Cannot convert between equatorial coordinates for equinoxes %s and %s" % (equinox_in, equinox_out)) Exception: Cannot convert between equatorial coordinates for equinoxes j2000 and j2000 Is there any idea why it fails ? Best Grigoris From thomas.robitaille at gmail.com Wed Feb 26 08:01:29 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 26 Feb 2014 14:01:29 +0100 Subject: [AstroPy] convert equinox issue with nicmos/radio data? In-Reply-To: <530CA9EC.10002@gmail.com> References: <530CA9EC.10002@gmail.com> Message-ID: Hi Grigoris, Could you send me the problematic FITS file and a short script to reproduce the problem? (off-list) Once I have the files I can look into it. Thanks, Tom (APLpy developer) On 25 February 2014 15:34, Grigoris Maravelias wrote: > Hello to all! > > A colleague of mine experiences a problem with astropy when she tries to > plot a nicmos image combined with radio contours. > Although the scripts runs flawlessly with most objects, when it fails > the error output is this: > > ERROR: Exception: Cannot convert between equatorial coordinates for > equinoxes j2000 and j2000 [aplpy.wcs_util] > Traceback (most recent call last): > File "./nicmos.py", line 98, in > f.show_contour(a149, colors="black", > levels=np.logspace(math.log10(sa[hrs]),math.log10(maxa[hrs]), > num=10),linestyles=['--'],slices=[0,0],linewidths=1) > File "", line 2, in show_contour > File "/Library/Python/2.7/site-packages/aplpy/decorators.py", line > 25, in _auto_refresh > f(*args, **kwargs) > File "/Library/Python/2.7/site-packages/aplpy/aplpy.py", line 916, in > show_contour > contour_util.transform(c, wcs_contour, self._wcs, filled=filled, > overlap=overlap) > File "/Library/Python/2.7/site-packages/aplpy/contour_util.py", line > 26, in transform > output=(system_out, equinox_out)) > File "/Library/Python/2.7/site-packages/aplpy/wcs_util.py", line 213, > in convert_coords > raise Exception("Cannot convert between equatorial coordinates for > equinoxes %s and %s" % (equinox_in, equinox_out)) > Exception: Cannot convert between equatorial coordinates for equinoxes > j2000 and j2000 > > Is there any idea why it fails ? > > Best > Grigoris > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From eebanado at uc.cl Wed Feb 26 11:23:12 2014 From: eebanado at uc.cl (=?ISO-8859-1?Q?Eduardo_Ba=F1ados_Torres?=) Date: Wed, 26 Feb 2014 17:23:12 +0100 Subject: [AstroPy] Lower(Upper)case astropy table columns Message-ID: Hi all, is there a way to lower case the columns of a table? I have tables that sometimes come with the uppercase and other lowercase keywords. By now I am just using try/except but this gets messy when the number of columns is large. I was expecting to exist something like: table.colnames.lower() Is there something like that? or how could I work this around Thanks! Eduardo Ba?ados ------------------------------------- For example from astropy.table import Table import numpy as np t = Table(arr, names=('RA', 'DEC', 'REDSHIFT')) t['RA'] # #array([ 0, 3, 6, 9, 12]) #I can access the table by t['RA'] #but I need to lowercase the columns to access them as t['ra'] #KeyError: 'ra' -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alexander.Conley at Colorado.EDU Wed Feb 26 11:33:18 2014 From: Alexander.Conley at Colorado.EDU (Alexander Joseph Conley) Date: Wed, 26 Feb 2014 09:33:18 -0700 Subject: [AstroPy] Lower(Upper)case astropy table columns In-Reply-To: References: Message-ID: <0B8BFD6F-440A-4713-A04F-AF8620F1C035@colorado.edu> for colname in t.colnames: t.rename_column(colname, colname.lower()) But if you need to be able to use -either- ?ra? or ?RA? I don?t know a way. Alex On 26-Feb-2014, at 9:23 AM, Eduardo Ba?ados Torres wrote: > Hi all, > > is there a way to lower case the columns of a table? I have tables that sometimes come with the uppercase and other lowercase keywords. By now I am just using try/except but this gets messy when the number of columns is large. I was expecting to exist something like: > > table.colnames.lower() > > Is there something like that? or how could I work this around > > Thanks! > > Eduardo Ba?ados > > ------------------------------------- > > For example > from astropy.table import Table > import numpy as np > > t = Table(arr, names=('RA', 'DEC', 'REDSHIFT')) > t['RA'] > # > #array([ 0, 3, 6, 9, 12]) > #I can access the table by t['RA'] > #but I need to lowercase the columns to access them as > t['ra'] > #KeyError: 'ra' > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From aldcroft at head.cfa.harvard.edu Wed Feb 26 12:07:49 2014 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Wed, 26 Feb 2014 12:07:49 -0500 Subject: [AstroPy] Lower(Upper)case astropy table columns In-Reply-To: <0B8BFD6F-440A-4713-A04F-AF8620F1C035@colorado.edu> References: <0B8BFD6F-440A-4713-A04F-AF8620F1C035@colorado.edu> Message-ID: On Wed, Feb 26, 2014 at 11:33 AM, Alexander Joseph Conley < Alexander.Conley at colorado.edu> wrote: > for colname in t.colnames: > t.rename_column(colname, colname.lower()) > But if you need to be able to use -either- 'ra' or 'RA' > I don't know a way. > It isn't possible with the current Table implementation. It should be possible to subclass Table (and probably Row, TableColumns) to do this, but might not be entirely trivial. - Tom > > Alex > > On 26-Feb-2014, at 9:23 AM, Eduardo Ba?ados Torres wrote: > > > Hi all, > > > > is there a way to lower case the columns of a table? I have tables that > sometimes come with the uppercase and other lowercase keywords. By now I am > just using try/except but this gets messy when the number of columns is > large. I was expecting to exist something like: > > > > table.colnames.lower() > > > > Is there something like that? or how could I work this around > > > > Thanks! > > > > Eduardo Ba?ados > > > > ------------------------------------- > > > > For example > > from astropy.table import Table > > import numpy as np > > > > t = Table(arr, names=('RA', 'DEC', 'REDSHIFT')) > > t['RA'] > > # > > #array([ 0, 3, 6, 9, 12]) > > #I can access the table by t['RA'] > > #but I need to lowercase the columns to access them as > > t['ra'] > > #KeyError: 'ra' > > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Wed Feb 26 14:24:53 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 26 Feb 2014 20:24:53 +0100 Subject: [AstroPy] convert equinox issue with nicmos/radio data? In-Reply-To: <530CA9EC.10002@gmail.com> References: <530CA9EC.10002@gmail.com> Message-ID: Hi Grigoris, (just replying to the list in case anyone has the same issue) The issue was due to the fact that one image had the first axis be the declination and the second the right ascension (as opposed to RA then Dec). I have fixed it so that this is now properly taken into account in APLpy. You can update with git clone https://github.com/aplpy/aplpy.git cd aplpy python setup.py install Let me know if this does not fix the issue. I did see another error, but I think this is unrelated. Cheers, Tom On 25 February 2014 15:34, Grigoris Maravelias wrote: > Hello to all! > > A colleague of mine experiences a problem with astropy when she tries to > plot a nicmos image combined with radio contours. > Although the scripts runs flawlessly with most objects, when it fails > the error output is this: > > ERROR: Exception: Cannot convert between equatorial coordinates for > equinoxes j2000 and j2000 [aplpy.wcs_util] > Traceback (most recent call last): > File "./nicmos.py", line 98, in > f.show_contour(a149, colors="black", > levels=np.logspace(math.log10(sa[hrs]),math.log10(maxa[hrs]), > num=10),linestyles=['--'],slices=[0,0],linewidths=1) > File "", line 2, in show_contour > File "/Library/Python/2.7/site-packages/aplpy/decorators.py", line > 25, in _auto_refresh > f(*args, **kwargs) > File "/Library/Python/2.7/site-packages/aplpy/aplpy.py", line 916, in > show_contour > contour_util.transform(c, wcs_contour, self._wcs, filled=filled, > overlap=overlap) > File "/Library/Python/2.7/site-packages/aplpy/contour_util.py", line > 26, in transform > output=(system_out, equinox_out)) > File "/Library/Python/2.7/site-packages/aplpy/wcs_util.py", line 213, > in convert_coords > raise Exception("Cannot convert between equatorial coordinates for > equinoxes %s and %s" % (equinox_in, equinox_out)) > Exception: Cannot convert between equatorial coordinates for equinoxes > j2000 and j2000 > > Is there any idea why it fails ? > > Best > Grigoris > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From asishrocks95 at gmail.com Wed Feb 26 23:58:00 2014 From: asishrocks95 at gmail.com (Asish Panda) Date: Thu, 27 Feb 2014 10:28:00 +0530 Subject: [AstroPy] Project of gsoc 2014, sunpy Message-ID: Hey all. I am planning to do the project listed in sunpy as "re-implementation of sunpy.wcs as sunpy.coordinate using astropy coordinate." https://github.com/sunpy/sunpy/wiki/Gsoc-2014-ideas They are also planning to take this new API into account that will be implemented by you all soon. https://github.com/astropy/astropy-APEs/pull/6 What would be a good place for me to start researching about it, as a complete beginner? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr.maravelias at gmail.com Fri Feb 28 07:10:46 2014 From: gr.maravelias at gmail.com (Grigoris Maravelias) Date: Fri, 28 Feb 2014 14:10:46 +0200 Subject: [AstroPy] convert equinox issue with nicmos/radio data? In-Reply-To: References: <530CA9EC.10002@gmail.com> Message-ID: <53107CC6.2040300@gmail.com> Hi Tom! Indeed this fixed the issue. Thanks for the quick response. Best Grigoris On 02/26/2014 09:24 PM, Thomas Robitaille wrote: > Hi Grigoris, > > (just replying to the list in case anyone has the same issue) > > The issue was due to the fact that one image had the first axis be the > declination and the second the right ascension (as opposed to RA then > Dec). I have fixed it so that this is now properly taken into account > in APLpy. You can update with > > git clone https://github.com/aplpy/aplpy.git > cd aplpy > python setup.py install > > Let me know if this does not fix the issue. I did see another error, > but I think this is unrelated. > > Cheers, > Tom > > > On 25 February 2014 15:34, Grigoris Maravelias wrote: >> Hello to all! >> >> A colleague of mine experiences a problem with astropy when she tries to >> plot a nicmos image combined with radio contours. >> Although the scripts runs flawlessly with most objects, when it fails >> the error output is this: >> >> ERROR: Exception: Cannot convert between equatorial coordinates for >> equinoxes j2000 and j2000 [aplpy.wcs_util] >> Traceback (most recent call last): >> File "./nicmos.py", line 98, in >> f.show_contour(a149, colors="black", >> levels=np.logspace(math.log10(sa[hrs]),math.log10(maxa[hrs]), >> num=10),linestyles=['--'],slices=[0,0],linewidths=1) >> File "", line 2, in show_contour >> File "/Library/Python/2.7/site-packages/aplpy/decorators.py", line >> 25, in _auto_refresh >> f(*args, **kwargs) >> File "/Library/Python/2.7/site-packages/aplpy/aplpy.py", line 916, in >> show_contour >> contour_util.transform(c, wcs_contour, self._wcs, filled=filled, >> overlap=overlap) >> File "/Library/Python/2.7/site-packages/aplpy/contour_util.py", line >> 26, in transform >> output=(system_out, equinox_out)) >> File "/Library/Python/2.7/site-packages/aplpy/wcs_util.py", line 213, >> in convert_coords >> raise Exception("Cannot convert between equatorial coordinates for >> equinoxes %s and %s" % (equinox_in, equinox_out)) >> Exception: Cannot convert between equatorial coordinates for equinoxes >> j2000 and j2000 >> >> Is there any idea why it fails ? >> >> Best >> Grigoris >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy From r.tanuj at gmail.com Fri Feb 28 14:21:17 2014 From: r.tanuj at gmail.com (Tanuj Rastogi) Date: Sat, 1 Mar 2014 00:51:17 +0530 Subject: [AstroPy] Introducing Myself, GSoC '14 Message-ID: Hi, I am a 3rd year CS student from Thapar University, India. I find your astropy project very interesting and have gone through the GSoC '14 ideas list. One of the ideas that I find interesting is: *Implementing uncertainties in Quantity* If possible, I would like to apply with a proposal for the same. I have forked the astropy repository and am exploring the codebase and the documentation related to this idea (mainly *nddata* and *units* modules). *This is what I have done so far:* I have gone through the code in *astropy.units *module and *astropy.nddata *module and have gained some insight into its working. I have specifically understood the* NDUncertainty* and *StdDevUncertainty*classes along with their usage in *NDData *class and the general structure of *NDData* class. ------ >From this preliminary study, my basic understanding of what I would have to do is that: 1. Implement the uncertainty attribute in the *Quantity* class ( as an object of *StdDevUncertainty* ). 2. Modify all operations on the *Quantity* class to ensure proper propagation of uncertainty. 3. Possibly, work on expanding the *Uncertainty* class, for eg., writing support for uncertainty with correlated variables. ------ Of course, I have not fully explored the *Quantity* class (or the *astropy.units* module in general) and there might be some things that I might be missing or getting wrong. Plus, I am not sure about the importance(or urgency) of this project from the organization's perspective and am open to ponder and work on other ideas if/as suggested by you. Having said this, I would appreciate if you could guide me on how to approach this idea further and building a proposal for the same. I do have some specific questions: 1. What would be your advice on approaching this idea further? Should I: * Try to fix basic bugs listed in the github isuues page. * Spend time on understanding the codebase further to get a clearer picture of what I have to do. * Focus on brushing up and reading some statistics and math related to uncertainty, to form a more concrete and clear idea of how the uncertainty would propagate and how to implement it. * Any other general advice you would like to give. 2. What should be the scope of the proposal that I should aim for. For eg., should it include expanding or modifying the present Uncertainty framework. ------ I have a good background in Python. In my 3rd semester I did an online course from Massachusetts Institute of Technology (through edx platform) on 'Introduction to Computer Science' which thought Python along with many CS concepts. I also earned a certificate for completing it. I have used Python extensively in a lot of coding contests. In addition to this, I did a course on 'Introduction to Astronomy' from Duke University (from Coursera). I would be happy to answer any questions you might have. Regards, Tanuj Rastogi -------------- next part -------------- An HTML attachment was scrubbed... URL: