From jturner at gemini.edu Mon Jun 3 14:02:48 2013 From: jturner at gemini.edu (James Turner) Date: Mon, 3 Jun 2013 14:02:48 -0400 Subject: [AstroPy] Beta release of Ureka Python + IRAF distribution Message-ID: <51ACDA48.7010504@gemini.edu> STScI and Gemini are announcing the first public beta release of the Ureka binary packaging installer for common astronomical software (primarily for the UV/Optical/IR community). The goals of the Ureka installer are to: 1) minimize the number of actions needed to install all the different software components. We are seeking a "one button install" (it's not quite one button, but not far from it). 2) permit installation without requiring system privileges. 3) make installs as problem-free as possible for the great majority of users. 4) allow users to install their own software (particularly Python-based) within this framework, or update versions of software within the framework. 5) permit different Ureka installations to coexist and to easily switch between them. 6) enable installing different versions of the same software package under a particular Ureka installation. 7) support Macs and most popular Linux variants. Ureka does not use LD_LIBRARY_PATH (or its Mac equivalent) and thus should not affect existing software after installation or use. Keep in mind that no installation system is completely foolproof (that's very nearly impossible to achieve). In particular, when users update or add software to the Ureka framework, they increase the risk of breaking something, but we feel that is an option that users should have as long as they understand the possible risks. This beta version includes IRAF and associated packages for IRAF, DS9, and a fairly full suite of Python scientific software packages (e.g., numpy, scipy, and matplotlib). The full listing of included software can be found at this link: http://ssb.stsci.edu/ureka/1.0beta5/docs/components.html Ureka can be downloaded from: http://ssb.stsci.edu/ureka/ (choose "1.0 beta 5") Instructions on the installation and use may be found on the same web page This is a beta release. We do expect to uncover some problems that testing at Gemini and STScI did not encounter. If you do find problems, please let us know (help at stsci.edu). Feedback on addition software components that would be considered generally essential for this distribution is also welcome. From ejensen1 at swarthmore.edu Mon Jun 3 22:13:40 2013 From: ejensen1 at swarthmore.edu (Eric L. N. Jensen) Date: Mon, 3 Jun 2013 22:13:40 -0400 Subject: [AstroPy] Questions re: Markov Chain Monte Carlo Message-ID: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> Hi all, I've been using python to drive the computation of some disk models, and I'm working on shifting the computation from a full (but coarse) grid search of parameter space to instead using Markov Chain Monte Carlo (via the 'emcee' package). In thinking about the best way to do this, I have a couple of questions. I should note that these aren't all pure Python questions, but instead some are more algorithmic / implementation questions; I hope it's OK to ask such things here, but feel free to tell me if it isn't. The main issue that I'm wrestling with is that each model is computationally very expensive - something like 6 minutes per set of model parameters. So, if at all possible, I'd like to avoid recalculating the same model (or a very similar model) over and over, and instead I'd like to reuse previous calculations if possible. For background, the way each model calculation works is that a subdirectory is created, whose name is based on the specific parameters, e.g. 'Mdisk_2.3_Rdisk_100_T_150' etc. This directory is then populated with input files (the density distribution, temperature distribution, etc.) and then a fortran code is called that calculates an image, ultimately producing a FITS file. So I'm imagining something like this, but I wonder if others have suggestions or alternatives: - Each time the Monte Carlo calculation makes a step in parameter space and chooses a set of parameters (i.e. a position vector in parameter space), do the following: - Check to see if the directory corresponding to that set of parameters exists; if it doesn't, then create it and proceed with the calculation. - If it does exist already, check to see if the final output image exists; if so, then use it. If not, then presumably another process is working on creating the image; sleep for some amount of time and keep checking back until the image appears. (The 'emcee' MCMC implementation uses multiple subprocesses that explore parameter space simultaneously, so it would be possible for two processes to arrive at the same place in parameter space at the same time.) One problem here is that if some process fails to finish a calculation, then other processes arriving at that space will get stuck there indefinitely while they wait; so maybe the wait needs to be finite, but if so, I'm not sure what gets returned after that finite wait; maybe just an error. So the first question is whether there's any straightforward way to avoid a race condition here, i.e. two processes both checking for the existence of a directory, both finding it non-existent, and then both trying to create it. Would something like this work? if os.path.exists(dirname): (code to use existing result in that dir, or wait for calculation to complete) else: # Directory doesn't exist, try to create it try: os.mkdir(dirname) except OSError: # Someone else got there first and created the dir since we last checked (code here to wait for completion and then return) else: # No exception was raised (code here to calculate the model) The second question involves rounding of parameter values, and its effect on a MCMC calculation. So far I've only been choosing relatively round numbers in my parameter search, since it has been a grid (e.g. exponents like 0.6, 0.7, 0.8, disk radii like 100, 150, 200). But once those values start being chosen by the MCMC routine, they won't be round numbers at all. This presents a few potential problems: first, since I'm using parameter values to name the directories, they need to be rounded to some set precision to create a reproducible directory name that I can use later to identify that model. Given how expensive each computation is, some rounding would be beneficial, since, say, a power-law exponent of 0.5999 isn't going to produce a model that's meaningfully different from an exponent of 0.6 - so if we can re-use previously-calculated results, we'll save time (especially near high-probability spots in parameter space, which should be visited more frequently). It seems like this shouldn't be too much of a problem (but I'm not that experienced with MCMC, which is why I'm asking), since: - The algorithm for moving around in parameter space is generally somewhat probabilistic anyway, so it shouldn't matter if the actual spot chosen is slightly different than what the algorithm specifies (due to rounding); - The above will only be true as long as we keep track of how we do the rounding, and apply the same rounding to the post-facto analysis of the chain as we do to the parameters for calculating chi-squared. (That is, the MCMC algorithm will be keeping track of the *unrounded* values and storing those with the chain; so we need to apply the same rounding function to the vector of parameter-space values both before calculating the probability, and when plotting the posterior distributions at the end. I can just encode any decisions about how to round each parameter in some function, and use that same function in both cases to transform a given position vector.) OK, sorry for such a long message - thanks in advance for any thoughts. Eric From mr.alex.hagen at gmail.com Mon Jun 3 23:21:18 2013 From: mr.alex.hagen at gmail.com (Alex Hagen) Date: Mon, 3 Jun 2013 23:21:18 -0400 Subject: [AstroPy] Questions re: Markov Chain Monte Carlo In-Reply-To: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> References: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> Message-ID: Hi Eric, Perhaps with so much computational complexity, you could look at precomputing a grid of parameters and then using interpolation to return results for emcee. This would be a particularly useful approach if you can use the same precomputed grid many times. The rounding that you are talking about seems dangerous because you're changing the step location and the likelihood value. Since mcmc uses these to determine the next step, rounding could be unfortunate. Hope this helps, --alex On Monday, June 3, 2013, Eric L. N. Jensen wrote: > Hi all, > > I've been using python to drive the computation of some disk models, and > I'm working on shifting the computation from a full (but coarse) grid > search of parameter space to instead using Markov Chain Monte Carlo (via > the 'emcee' package). In thinking about the best way to do this, I have a > couple of questions. I should note that these aren't all pure Python > questions, but instead some are more algorithmic / implementation > questions; I hope it's OK to ask such things here, but feel free to tell me > if it isn't. > > The main issue that I'm wrestling with is that each model is > computationally very expensive - something like 6 minutes per set of model > parameters. So, if at all possible, I'd like to avoid recalculating the > same model (or a very similar model) over and over, and instead I'd like to > reuse previous calculations if possible. > > For background, the way each model calculation works is that a > subdirectory is created, whose name is based on the specific parameters, > e.g. 'Mdisk_2.3_Rdisk_100_T_150' etc. This directory is then populated > with input files (the density distribution, temperature distribution, etc.) > and then a fortran code is called that calculates an image, ultimately > producing a FITS file. > > So I'm imagining something like this, but I wonder if others have > suggestions or alternatives: > > - Each time the Monte Carlo calculation makes a step in parameter space > and chooses a set of parameters (i.e. a position vector in parameter > space), do the following: > - Check to see if the directory corresponding to that set of > parameters exists; if it doesn't, then create it and proceed with the > calculation. > - If it does exist already, check to see if the final output image > exists; if so, then use it. If not, then presumably another process is > working on creating the image; sleep for some amount of time and keep > checking back until the image appears. (The 'emcee' MCMC implementation > uses multiple subprocesses that explore parameter space simultaneously, so > it would be possible for two processes to arrive at the same place in > parameter space at the same time.) One problem here is that if some > process fails to finish a calculation, then other processes arriving at > that space will get stuck there indefinitely while they wait; so maybe the > wait needs to be finite, but if so, I'm not sure what gets returned after > that finite wait; maybe just an error. > > So the first question is whether there's any straightforward way to avoid > a race condition here, i.e. two processes both checking for the existence > of a directory, both finding it non-existent, and then both trying to > create it. Would something like this work? > > if os.path.exists(dirname): > (code to use existing result in that dir, or wait for calculation > to complete) > else: # Directory doesn't exist, try to create it > try: > os.mkdir(dirname) > except OSError: > # Someone else got there first and created the dir since we last > checked > (code here to wait for completion and then return) > else: # No exception was raised > (code here to calculate the model) > > > The second question involves rounding of parameter values, and its effect > on a MCMC calculation. So far I've only been choosing relatively round > numbers in my parameter search, since it has been a grid (e.g. exponents > like 0.6, 0.7, 0.8, disk radii like 100, 150, 200). But once those values > start being chosen by the MCMC routine, they won't be round numbers at all. > This presents a few potential problems: first, since I'm using parameter > values to name the directories, they need to be rounded to some set > precision to create a reproducible directory name that I can use later to > identify that model. Given how expensive each computation is, some > rounding would be beneficial, since, say, a power-law exponent of 0.5999 > isn't going to produce a model that's meaningfully different from an > exponent of 0.6 - so if we can re-use previously-calculated results, we'll > save time (especially near high-probability spots in parameter space, which > should be visited more frequently). > > It seems like this shouldn't be too much of a problem (but I'm not that > experienced with MCMC, which is why I'm asking), since: > > - The algorithm for moving around in parameter space is generally > somewhat probabilistic anyway, so it shouldn't matter if the actual spot > chosen is slightly different than what the algorithm specifies (due to > rounding); > > - The above will only be true as long as we keep track of how we > do the rounding, and apply the same rounding to the post-facto analysis of > the chain as we do to the parameters for calculating chi-squared. (That > is, the MCMC algorithm will be keeping track of the *unrounded* values and > storing those with the chain; so we need to apply the same rounding > function to the vector of parameter-space values both before calculating > the probability, and when plotting the posterior distributions at the end. > I can just encode any decisions about how to round each parameter in some > function, and use that same function in both cases to transform a given > position vector.) > > OK, sorry for such a long message - thanks in advance for any thoughts. > > Eric > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- alex hagen 916.425.3581 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.p.holden at gmail.com Mon Jun 3 23:32:09 2013 From: brad.p.holden at gmail.com (Brad Holden) Date: Mon, 3 Jun 2013 20:32:09 -0700 Subject: [AstroPy] Questions re: Markov Chain Monte Carlo In-Reply-To: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> References: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> Message-ID: To prevent the race condition, you can also use lock files. The easy way would be to lock out the parent directory so if one process is making a subdirectory, no other process can. Ie., if the file is present, all other processes wait until the lock file is removed before making directories. This way you don't have to worry about exception handling (though, admittedly, in python it is pretty easy.) You could populate the grid via a tree and could have lock files at a finer grain (which is something we do). On Mon, Jun 3, 2013 at 7:13 PM, Eric L. N. Jensen wrote: > Hi all, > > I've been using python to drive the computation of some disk models, and > I'm working on shifting the computation from a full (but coarse) grid > search of parameter space to instead using Markov Chain Monte Carlo (via > the 'emcee' package). In thinking about the best way to do this, I have a > couple of questions. I should note that these aren't all pure Python > questions, but instead some are more algorithmic / implementation > questions; I hope it's OK to ask such things here, but feel free to tell me > if it isn't. > > The main issue that I'm wrestling with is that each model is > computationally very expensive - something like 6 minutes per set of model > parameters. So, if at all possible, I'd like to avoid recalculating the > same model (or a very similar model) over and over, and instead I'd like to > reuse previous calculations if possible. > > For background, the way each model calculation works is that a > subdirectory is created, whose name is based on the specific parameters, > e.g. 'Mdisk_2.3_Rdisk_100_T_150' etc. This directory is then populated > with input files (the density distribution, temperature distribution, etc.) > and then a fortran code is called that calculates an image, ultimately > producing a FITS file. > > So I'm imagining something like this, but I wonder if others have > suggestions or alternatives: > > - Each time the Monte Carlo calculation makes a step in parameter space > and chooses a set of parameters (i.e. a position vector in parameter > space), do the following: > - Check to see if the directory corresponding to that set of > parameters exists; if it doesn't, then create it and proceed with the > calculation. > - If it does exist already, check to see if the final output image > exists; if so, then use it. If not, then presumably another process is > working on creating the image; sleep for some amount of time and keep > checking back until the image appears. (The 'emcee' MCMC implementation > uses multiple subprocesses that explore parameter space simultaneously, so > it would be possible for two processes to arrive at the same place in > parameter space at the same time.) One problem here is that if some > process fails to finish a calculation, then other processes arriving at > that space will get stuck there indefinitely while they wait; so maybe the > wait needs to be finite, but if so, I'm not sure what gets returned after > that finite wait; maybe just an error. > > So the first question is whether there's any straightforward way to avoid > a race condition here, i.e. two processes both checking for the existence > of a directory, both finding it non-existent, and then both trying to > create it. Would something like this work? > > if os.path.exists(dirname): > (code to use existing result in that dir, or wait for calculation > to complete) > else: # Directory doesn't exist, try to create it > try: > os.mkdir(dirname) > except OSError: > # Someone else got there first and created the dir since we last > checked > (code here to wait for completion and then return) > else: # No exception was raised > (code here to calculate the model) > > > The second question involves rounding of parameter values, and its effect > on a MCMC calculation. So far I've only been choosing relatively round > numbers in my parameter search, since it has been a grid (e.g. exponents > like 0.6, 0.7, 0.8, disk radii like 100, 150, 200). But once those values > start being chosen by the MCMC routine, they won't be round numbers at all. > This presents a few potential problems: first, since I'm using parameter > values to name the directories, they need to be rounded to some set > precision to create a reproducible directory name that I can use later to > identify that model. Given how expensive each computation is, some > rounding would be beneficial, since, say, a power-law exponent of 0.5999 > isn't going to produce a model that's meaningfully different from an > exponent of 0.6 - so if we can re-use previously-calculated results, we'll > save time (especially near high-probability spots in parameter space, which > should be visited more frequently). > > It seems like this shouldn't be too much of a problem (but I'm not that > experienced with MCMC, which is why I'm asking), since: > > - The algorithm for moving around in parameter space is generally > somewhat probabilistic anyway, so it shouldn't matter if the actual spot > chosen is slightly different than what the algorithm specifies (due to > rounding); > > - The above will only be true as long as we keep track of how we > do the rounding, and apply the same rounding to the post-facto analysis of > the chain as we do to the parameters for calculating chi-squared. (That > is, the MCMC algorithm will be keeping track of the *unrounded* values and > storing those with the chain; so we need to apply the same rounding > function to the vector of parameter-space values both before calculating > the probability, and when plotting the posterior distributions at the end. > I can just encode any decisions about how to round each parameter in some > function, and use that same function in both cases to transform a given > position vector.) > > OK, sorry for such a long message - thanks in advance for any thoughts. > > Eric > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erwin at mpe.mpg.de Tue Jun 4 05:23:34 2013 From: erwin at mpe.mpg.de (Peter Erwin) Date: Tue, 4 Jun 2013 11:23:34 +0200 Subject: [AstroPy] Questions re: Markov Chain Monte Carlo In-Reply-To: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> References: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> Message-ID: <24850987-59A9-4E82-B9F4-0D72E2A2C4B1@mpe.mpg.de> Hi Eric, I know very little about MCMC, but my first thought is that the idea of rounding the parameter values sounds a little dodgy. On the other hand, some quick googling suggests that doing MCMC in discrete (as opposed to continuous) parameter spaces -- which is more or less what you're proposing -- isn't unknown, and there are even some codes specialized for that, e.g. http://www.hydrol-earth-syst-sci.net/15/3701/2011/hess-15-3701-2011.html It might be that using emcee (which assumes a continuous parameter space) might be somewhat less efficient than an approach built to account for discrete parameter spaces, but that may not matter much. > So I'm imagining something like this, but I wonder if others have suggestions or alternatives: > > - Each time the Monte Carlo calculation makes a step in parameter space and chooses a set of parameters (i.e. a position vector in parameter space), do the following: > - Check to see if the directory corresponding to that set of parameters exists; if it doesn't, then create it and proceed with the calculation. > - If it does exist already, check to see if the final output image exists; if so, then use it. If not, then presumably another process is working on creating the image; sleep for some amount of time and keep checking back until the image appears. (The 'emcee' MCMC implementation uses multiple subprocesses that explore parameter space simultaneously, so it would be possible for two processes to arrive at the same place in parameter space at the same time.) One problem here is that if some process fails to finish a calculation, then other processes arriving at that space will get stuck there indefinitely while they wait; so maybe the wait needs to be finite, but if so, I'm not sure what gets returned after that finite wait; maybe just an error. Just to confirm something: it sounds like you're saying that the result of starting with the same set of parameters (Mdisk, Rdisk, T, etc.) will be the exact same output FITS image; but somehow different iterations of the MC process, using the same output model FITS image, can give you different chi^2 values? Which doesn't really make sense to me -- it implies that a given set of parameters (location in the parameter space) does not uniquely determine the chi^2 value. Assuming that's not actually true -- i.e., assuming that same set of parameters --> same output model FITS image --> same chi^2 -- I think a simpler approach would be to cache the chi^2 values in a data structure which is indexed by the parameter values. Checking the cache might itself be faster than checking the disk for the existence of a given directory, since you avoid any disk I/O delays; and this way you avoid having the recalculate chi^2 for a given model image once you've done it the first time. One way of doing this in Python would be a dictionary indexed by tuples of the parameter values. So the above stage of your algorithm would look something like this: - Each time the Monte Carlo calculation makes a step in parameter space and chooses a set of parameters (i.e. a position vector in parameter space), do the following: 1. Check to see if there is already an entry in the chi^2 cache data structure corresponding to those [possibly rounded] parameter values, e.g. if (Mdisk, Rdisk, T, ...) in chi2ValuesDict: # do step 2 below else: # do step 3 below 2. If it's already there, just take the corresponding chi^2 value from the dictionary: currentChi2 = chi2ValuesDict[(Mdisk, Rdisk, T, ...)] 3. If it's *not* already done, then generate a new model. You can get a crude (though not perfect) locking approach by first placing a dummy value in the chi^2 cache: A. chi2ValuesDict[(Mdisk, Rdisk, T, ...)] = 0 # (from this point on, any other emcee subprocess which queries the dictionary will get the message that an entry already exists -- or will -- for these parameter values) B. Do the full directory setup, image calculation, and chi^2 calculation C. Store the new chi^2 in the dictionary, and return it to the emcee process: chi2ValuesDict[(Mdisk, Rdisk, T, ...)] = chi2_from_new_calculation currentChi2 = chi2_from_new_calculation This *does* require that the dictionary chi2ValuesDict be a *shared* data structure that all the emcee subprocesses can read and write to; I don't know if emcee permits that sort of thing. If not, you can probably still save *some* time by having it be a single file on disk -- i.e., you still avoid wasting time re-calculating chi^2 values for identical model images. (Naturally, this really only works if you *do* round the parameters, because otherwise the odds that any given step in the MC calculation will pick the exact same set of parameter values is (just about) infinitesimal.) cheers, Peter ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)176 2481 7713 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin From erwin at mpe.mpg.de Tue Jun 4 05:54:56 2013 From: erwin at mpe.mpg.de (Peter Erwin) Date: Tue, 4 Jun 2013 11:54:56 +0200 Subject: [AstroPy] Questions re: Markov Chain Monte Carlo In-Reply-To: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> References: <531E3BC8-DDE3-4AB7-AF2E-56D592055066@swarthmore.edu> Message-ID: Hi Eric, Whoops -- just realized that my proposed algorithm will occasionally return dummy chi^2 values rather than waiting until another subprocess has finished, which isn't what you want, so replace my "step 2" with: 2. If it's already there, just take the corresponding chi^2 value from the dictionary, or else wait for the other subprocess to finish calculations if it's working on these parameter values currentChi2 = chi2ValuesDict[(Mdisk, Rdisk, T, ...)] if chi2ValuesDict[(Mdisk, Rdisk, T, ...)] == 0: # another subprocess is working on this set of parameter values, so wait until it's done -- e.g., sleep for 1s and then re-check to see if the stored value is no longer 0... else: return currentChi2 cheers, Peter ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)176 2481 7713 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin From Nikta.Amiri at Colorado.EDU Tue Jun 11 18:09:05 2013 From: Nikta.Amiri at Colorado.EDU (Nikta Amiri) Date: Tue, 11 Jun 2013 16:09:05 -0600 Subject: [AstroPy] problem using the WCS package Message-ID: <4BA48585-C21F-40AE-8C13-5F7C0A608DF7@colorado.edu> Hello all, I am trying to use WCS package in Astropy on a fits file that has NAXIS=4 in the header, to convert some coordinates in decimal degrees to pixel values. At first, I faced this error: "RuntimeError: Input array must be 2-dimensional, where the second dimension >= 4" To get around this problem, I edited the fits header and removed NAXIS 3 and 4 as well as all corresponding CDELT/CTYPE. When running WCS_world2pix I face no error, but the pixel values that I obtain are incorrect and in some cases I even get negative values for pixels. For example, I get these values after using WCS_world2pix on a fits image with 784*282 dimension: array([[ 8.35828508, 101.42723769], [ -2.40845355, 373.76598659], [ -54.96921724, 690.64432708], [ -60.6297434 , 625.57276931], [ 75.34581951, 902.489706 ]]) Could you please let me know if there is a solution to this problem? FYI, I attach the modified fits header and the lines that I removed in the original fits header below: Thank you, Nikta SIMPLE = T / BITPIX = -32 / NAXIS = 2 NAXIS1 = 282 / NAXIS2 = 784 / EXTEND = T /Tables following main image BLOCKED = T /Tape may be blocked OBJECT = 'YY' /Source name TELESCOP= 'ZZ' / INSTRUME= ' ' / OBSERVER= 'xx ' / DATE-OBS= '1901-09-27' /Obs start date YYYY-MM-DD DATE-MAP= '2007-02-26' /Last processing date YYYY-MM-DD BSCALE = 1.00000000000E+00 /REAL = TAPE * BSCALE + BZERO BZERO = 0.00000000000E+00 / BUNIT = 'KM/S ' /Units of flux EPOCH = 1.950000000E+03 /Epoch of RA DEC BLANK = -1 / IEEE not-a-number for blanked pixels DATAMAX = -1.422322839E-01 /Maximum pixel value DATAMIN = -9.989158325E+02 /Minimum pixel value CTYPE1 = 'RA---GLS' / CRVAL1 = 1.00012502670E+01 / CDELT1 = -2.222222276E-03 / CRPIX1 = 1.360000000E+02 / CROTA1 = 0.000000000E+00 / CTYPE2 = 'DEC--GLS' / CRVAL2 = 4.10008354187E+01 / CDELT2 = 2.222222276E-03 / CRPIX2 = 3.640000000E+02 / CROTA2 = -3.770000076E+01 / And the removed lines are: NAXIS3 = 1 / NAXIS4 = 1 / CTYPE3 = 'FREQ ' / CRVAL3 = -3.50000000000E+05 / CDELT3 = -2.600757568E+03 / CRPIX3 = 1.435000000E+02 / CROTA3 = 0.000000000E+00 / CTYPE4 = ' ' / CRVAL4 = 0.00000000000E+00 / CDELT4 = 1.000000000E+00 / CRPIX4 = 0.000000000E+00 / CROTA4 = 0.000000000E+00 / From d.berry at jach.hawaii.edu Wed Jun 12 06:22:11 2013 From: d.berry at jach.hawaii.edu (David Berry) Date: Wed, 12 Jun 2013 11:22:11 +0100 Subject: [AstroPy] problem using the WCS package In-Reply-To: <4BA48585-C21F-40AE-8C13-5F7C0A608DF7@colorado.edu> References: <4BA48585-C21F-40AE-8C13-5F7C0A608DF7@colorado.edu> Message-ID: On 11 June 2013 23:09, Nikta Amiri wrote: > Hello all, > > I am trying to use WCS package in Astropy on a fits file that has NAXIS=4 in the header, to convert some coordinates in decimal degrees to pixel values. At first, I faced this error: > > "RuntimeError: Input array must be 2-dimensional, where the second dimension >= 4" > > To get around this problem, I edited the fits header and removed NAXIS 3 and 4 as well as all corresponding CDELT/CTYPE. When running WCS_world2pix I face no error, but the pixel values that I obtain are incorrect and in some cases I even get negative values for pixels. For example, I get these values after using WCS_world2pix on a fits image with 784*282 dimension: > > array([[ 8.35828508, 101.42723769], > [ -2.40845355, 373.76598659], > [ -54.96921724, 690.64432708], > [ -60.6297434 , 625.57276931], > [ 75.34581951, 902.489706 ]]) > > > Could you please let me know if there is a solution to this problem? FYI, I attach the modified fits header and the lines that I removed in the original fits header below: Your header uses a GLS projection, and has non-zero CRVAL and CROTA values. This is slightly problematic in that modern WCS software must convert this from a simple shift and rotation in pixel coordinates (as specified by the old "AIPS convention" for WCS) into an oblique projection. See section 6.1.4 in the FITS-WCS paper II ("Representations of celestial coordinates in FITS"). You say you are getting incorrect values - how are you determining the "correct" values for this comparison? Could you post a list of the world coordinates you are using with the corresponding correct pixel coordinates. BTW - what is wrong with negative pixel coordinates? It just means the position is off the edge of the array. David > Thank you, > Nikta > > > > SIMPLE = T / > BITPIX = -32 / > NAXIS = 2 > NAXIS1 = 282 / > NAXIS2 = 784 / > EXTEND = T /Tables following main image > BLOCKED = T /Tape may be blocked > OBJECT = 'YY' /Source name > TELESCOP= 'ZZ' / > INSTRUME= ' ' / > OBSERVER= 'xx ' / > DATE-OBS= '1901-09-27' /Obs start date YYYY-MM-DD > DATE-MAP= '2007-02-26' /Last processing date YYYY-MM-DD > BSCALE = 1.00000000000E+00 /REAL = TAPE * BSCALE + BZERO > BZERO = 0.00000000000E+00 / > BUNIT = 'KM/S ' /Units of flux > EPOCH = 1.950000000E+03 /Epoch of RA DEC > BLANK = -1 / IEEE not-a-number for blanked pixels > DATAMAX = -1.422322839E-01 /Maximum pixel value > DATAMIN = -9.989158325E+02 /Minimum pixel value > CTYPE1 = 'RA---GLS' / > CRVAL1 = 1.00012502670E+01 / > CDELT1 = -2.222222276E-03 / > CRPIX1 = 1.360000000E+02 / > CROTA1 = 0.000000000E+00 / > CTYPE2 = 'DEC--GLS' / > CRVAL2 = 4.10008354187E+01 / > CDELT2 = 2.222222276E-03 / > CRPIX2 = 3.640000000E+02 / > CROTA2 = -3.770000076E+01 / > > And the removed lines are: > NAXIS3 = 1 / > NAXIS4 = 1 / > > CTYPE3 = 'FREQ ' / > CRVAL3 = -3.50000000000E+05 / > CDELT3 = -2.600757568E+03 / > CRPIX3 = 1.435000000E+02 / > CROTA3 = 0.000000000E+00 / > CTYPE4 = ' ' / > CRVAL4 = 0.00000000000E+00 / > CDELT4 = 1.000000000E+00 / > CRPIX4 = 0.000000000E+00 / > CROTA4 = 0.000000000E+00 / > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From ejensen1 at swarthmore.edu Wed Jun 12 12:47:50 2013 From: ejensen1 at swarthmore.edu (Eric L. N. Jensen) Date: Wed, 12 Jun 2013 12:47:50 -0400 Subject: [AstroPy] Rotating a FITS image (or just its WCS) Message-ID: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> Hi all, I'm creating some model disk images which are computationally intensive. It occurs to me that, for one of my parameters, the position angle of the disk projected on the sky, there is no need to recompute an image each time, as the image at different PAs is just a rotated version of the same image at zero PA. So I'd like to be able to take an existing FITS image, with WCS info in the header, and either keep the WCS the same while rotating the image pixels, or simply modify the image WCS to indicate a rotated reference frame for the image, to be interpreted by CASA when it reads the image. My first approach was to simply add a rotation matrix to the FITS header, using the PC_i_j keywords that specify the image's rotation; ds9 viewed the image OK, but it appears that this is not supported by CASA. So, I'm wondering about rotating the image itself and leaving the WCS unchanged. So, is there a straightforward way to rotate an image by an arbitrary angle? (Ideally with handling things like padding the image so it remains rectangular.) Thanks in advance for your help with this, Eric From crawford at saao.ac.za Wed Jun 12 13:02:45 2013 From: crawford at saao.ac.za (Steve Crawford) Date: Wed, 12 Jun 2013 19:02:45 +0200 (SAST) Subject: [AstroPy] Rotating a FITS image (or just its WCS) In-Reply-To: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> Message-ID: <2f48923b-fa61-4f77-836c-d00a741a626d@mailcape> Hi Eric, For transforming 2D arrays, there are some useful tools under scipy.ndimage.interpoloate: http://docs.scipy.org/doc/scipy/reference/ndimage.html#module-scipy.ndimage.interpolation There is a rotate task as well as more complex transformations, but they would probably meet your needs for rotating the image. Cheers Steve ----- Original Message ----- > From: "Eric L. N. Jensen" > To: "astropy" > Sent: Wednesday, 12 June, 2013 6:47:50 PM > Subject: [AstroPy] Rotating a FITS image (or just its WCS) > > Hi all, > > I'm creating some model disk images which are computationally > intensive. It occurs to me that, for one of my parameters, the > position angle of the disk projected on the sky, there is no need to > recompute an image each time, as the image at different PAs is just > a rotated version of the same image at zero PA. > > So I'd like to be able to take an existing FITS image, with WCS info > in the header, and either keep the WCS the same while rotating the > image pixels, or simply modify the image WCS to indicate a rotated > reference frame for the image, to be interpreted by CASA when it > reads the image. > > My first approach was to simply add a rotation matrix to the FITS > header, using the PC_i_j keywords that specify the image's rotation; > ds9 viewed the image OK, but it appears that this is not supported > by CASA. > > So, I'm wondering about rotating the image itself and leaving the WCS > unchanged. So, is there a straightforward way to rotate an image by > an arbitrary angle? (Ideally with handling things like padding the > image so it remains rectangular.) > > > > Thanks in advance for your help with this, > > Eric > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > !DSPAM:51b8a65431151747570720! > > > From rowen at uw.edu Wed Jun 12 13:06:00 2013 From: rowen at uw.edu (Russell Owen) Date: Wed, 12 Jun 2013 10:06:00 -0700 Subject: [AstroPy] Rotating a FITS image (or just its WCS) In-Reply-To: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> References: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> Message-ID: I don't know if you'd call it "straightforward", but the usual technique is resampling the image. Resampling correlates noise and either smears the image (if you use a low-order resampling kernel) or adds ringing around sharp features (if you use a high-order kernel). SExtractor can do this easily and quickly (so can the LSST software stack, and I'm sure there are other options). You will need to generate the desired new WCS. -- Russell On Jun 12, 2013, at 9:47 AM, "Eric L. N. Jensen" wrote: > Hi all, > > I'm creating some model disk images which are computationally intensive. It occurs to me that, for one of my parameters, the position angle of the disk projected on the sky, there is no need to recompute an image each time, as the image at different PAs is just a rotated version of the same image at zero PA. > > So I'd like to be able to take an existing FITS image, with WCS info in the header, and either keep the WCS the same while rotating the image pixels, or simply modify the image WCS to indicate a rotated reference frame for the image, to be interpreted by CASA when it reads the image. > > My first approach was to simply add a rotation matrix to the FITS header, using the PC_i_j keywords that specify the image's rotation; ds9 viewed the image OK, but it appears that this is not supported by CASA. > > So, I'm wondering about rotating the image itself and leaving the WCS unchanged. So, is there a straightforward way to rotate an image by an arbitrary angle? (Ideally with handling things like padding the image so it remains rectangular.) > > > > Thanks in advance for your help with this, From mperrin at stsci.edu Wed Jun 12 13:06:04 2013 From: mperrin at stsci.edu (Marshall Perrin) Date: Wed, 12 Jun 2013 17:06:04 +0000 Subject: [AstroPy] Rotating a FITS image (or just its WCS) In-Reply-To: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> References: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> Message-ID: <8E38540F-C873-4869-AFAB-3B441468A63B@stsci.edu> On Jun 12, 2013, at 12:47 PM, "Eric L. N. Jensen" wrote: > So, I'm wondering about rotating the image itself and leaving the WCS unchanged. So, is there a straightforward way to rotate an image by an arbitrary angle? (Ideally with handling things like padding the image so it remains rectangular.) scipy.ndimage.interpolation.rotate http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.interpolation.rotate.html The 'reshape' keyword will do the padding you mention. One caveat with this is that it can only do rotations about the exact center point of the array. That is often but not always what you want? - Marshall > > > > Thanks in advance for your help with this, > > Eric > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From d.berry at jach.hawaii.edu Wed Jun 12 13:09:48 2013 From: d.berry at jach.hawaii.edu (David Berry) Date: Wed, 12 Jun 2013 18:09:48 +0100 Subject: [AstroPy] Rotating a FITS image (or just its WCS) In-Reply-To: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> References: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> Message-ID: On 12 June 2013 17:47, Eric L. N. Jensen wrote: > Hi all, > > I'm creating some model disk images which are computationally intensive. It occurs to me that, for one of my parameters, the position angle of the disk projected on the sky, there is no need to recompute an image each time, as the image at different PAs is just a rotated version of the same image at zero PA. > > So I'd like to be able to take an existing FITS image, with WCS info in the header, and either keep the WCS the same while rotating the image pixels, or simply modify the image WCS to indicate a rotated reference frame for the image, to be interpreted by CASA when it reads the image. > > My first approach was to simply add a rotation matrix to the FITS header, using the PC_i_j keywords that specify the image's rotation; ds9 viewed the image OK, but it appears that this is not supported by CASA. Have you tried using CDi_j instead of PCi_j? Or even good ol' CROTAi ? CASA must surely upport some way of rating an image, so it's got to be PCi_j, CDi_j or CROTA... David > > So, I'm wondering about rotating the image itself and leaving the WCS unchanged. So, is there a straightforward way to rotate an image by an arbitrary angle? (Ideally with handling things like padding the image so it remains rectangular.) > > > > Thanks in advance for your help with this, > > Eric > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From ejensen1 at swarthmore.edu Wed Jun 12 15:23:34 2013 From: ejensen1 at swarthmore.edu (Eric L. N. Jensen) Date: Wed, 12 Jun 2013 15:23:34 -0400 Subject: [AstroPy] Rotating a FITS image (or just its WCS) In-Reply-To: References: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> Message-ID: <030E2DCA-E888-494C-8332-750F899478B9@swarthmore.edu> Thanks to everyone for the quick and helpful responses, as usual! On Jun 12, 2013, at 1:06 PM, Marshall Perrin wrote: > scipy.ndimage.interpolation.rotate > http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.interpolation.rotate.html This did the trick - thanks! On Jun 12, 2013, at 1:09 PM, David Berry wrote: > > Have you tried using CDi_j instead of PCi_j? Or even good ol' CROTAi ? > > CASA must surely upport some way of rating an image, so it's got to be > PCi_j, CDi_j or CROTA... I looked at the CASA code, and it appears that rotation of the input file WCS is not yet supported by the 'simobserve' task I'm using. Thanks again, Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Jun 13 04:03:42 2013 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 13 Jun 2013 10:03:42 +0200 Subject: [AstroPy] Rotating a FITS image (or just its WCS) In-Reply-To: <030E2DCA-E888-494C-8332-750F899478B9@swarthmore.edu> References: <75C28EEB-F679-40F0-BDEF-5360A04C487C@swarthmore.edu> <030E2DCA-E888-494C-8332-750F899478B9@swarthmore.edu> Message-ID: Just for the record, there's also the option to use Montage which I wrote an Astropy affiliated package wrapper for: http://www.astropy.org/montage-wrapper/ For example, Montage has an mRotate command that might come in handy in cases like this. Cheers, Tom On 12 June 2013 21:23, Eric L. N. Jensen wrote: > Thanks to everyone for the quick and helpful responses, as usual! > > On Jun 12, 2013, at 1:06 PM, Marshall Perrin wrote: > > scipy.ndimage.interpolation.rotate > http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.interpolation.rotate.html > > > This did the trick - thanks! > > > > On Jun 12, 2013, at 1:09 PM, David Berry wrote: > > > Have you tried using CDi_j instead of PCi_j? Or even good ol' CROTAi ? > > CASA must surely upport some way of rating an image, so it's got to be > PCi_j, CDi_j or CROTA... > > > I looked at the CASA code, and it appears that rotation of the input file > WCS is not yet supported by the 'simobserve' task I'm using. > > > Thanks again, > > Eric > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From sergio.pasra at gmail.com Fri Jun 14 12:06:23 2013 From: sergio.pasra at gmail.com (Sergio Pascual) Date: Fri, 14 Jun 2013 18:06:23 +0200 Subject: [AstroPy] message from SOFA chair In-Reply-To: References: <5188D8A3.9050707@nrao.edu> <43A19EA6-232C-43E1-8B59-26D360085E21@gmail.com> Message-ID: Hello, is there any progress in this topic? I have checked today and the license text in the SOFA webpage remains the same. Sergio 2013/5/7 Thomas Robitaille > Just for information, we are currently working towards a solution > off-list with the SOFA board, and will inform the list once things > have been settled. So no need to propose a license text at this stage. > > Tom > > On 7 May 2013 17:15, Wolfgang Kerzendorf wrote: > > I think what Mario and Tim are proposing is very good as it would > alleviate > > the problems mentioned by Catherine. To speed up the process, we should > > propose a license text and they could then see if that meets their > criteria > > (make it as hard as possible for them to delay or refuse). > > > > Cheers > > Wolfgang > > On 2013-05-07, at 10:45 AM, Tim Jenness wrote: > > > > Thanks. > > > > > > On Tue, May 7, 2013 at 3:34 AM, Scott Ransom wrote: > >> > >> > >> On the subject of the SOFA license conditions, our Board member Patrick > >> Wallace is continuing discussions he had on this topic last year, which > >> resulted in changes to the license that enabled the Debian release to > >> proceed. Recent emails, which unfortunately were delayed by the spam > >> filter problems, show that further discussion is needed, and Patrick is > >> now in touch with Perry Greenfield, STScI Science Software Branch lead. > >> The nub of the problem is that SOFA software has to address two > >> conflicting requirements: (i) the insistence by free software groups > >> that users should not be constrained in any way and (ii) the need to > >> prevent "counterfeit" versions coming into circulation. The second > >> point is vital because SOFA software represents IAU standards and indeed > >> is cited in other standards such as IERS Conventions. > >> > >> > > > > Regarding the counterfeit argument, isn't this very similar to encryption > > libraries where you want to make sure that you are using the official > > library and not one you found on the internet that happens to have a back > > door? OpenSSL and other libraries deal with this. People are far more > > concerned about using the proper OpenSSL than using SOFA but the > underlying > > principal is the same. OpenSSL has a very straightforward licence > > (http://www.openssl.org/source/license.html) which has clearly been > approved > > for distribution. > > > > -- > > Tim Jenness > > CCAT Software Manager > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Fri Jun 14 12:12:22 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Fri, 14 Jun 2013 12:12:22 -0400 Subject: [AstroPy] message from SOFA chair In-Reply-To: References: <5188D8A3.9050707@nrao.edu> <43A19EA6-232C-43E1-8B59-26D360085E21@gmail.com> Message-ID: Yes, as a matter of fact, today we got explicit permission from the Board to relicense SOFA. That will require us to release it under a different name, and using a different prefix for the routine names. You should be hearing about it soon. But they will not be changing their license. Perry On Jun 14, 2013, at 12:06 PM, Sergio Pascual wrote: > Hello, > > is there any progress in this topic? I have checked today and the license text in the SOFA webpage remains the same. > > Sergio > > > 2013/5/7 Thomas Robitaille > Just for information, we are currently working towards a solution > off-list with the SOFA board, and will inform the list once things > have been settled. So no need to propose a license text at this stage. > > Tom > > On 7 May 2013 17:15, Wolfgang Kerzendorf wrote: > > I think what Mario and Tim are proposing is very good as it would alleviate > > the problems mentioned by Catherine. To speed up the process, we should > > propose a license text and they could then see if that meets their criteria > > (make it as hard as possible for them to delay or refuse). > > > > Cheers > > Wolfgang > > On 2013-05-07, at 10:45 AM, Tim Jenness wrote: > > > > Thanks. > > > > > > On Tue, May 7, 2013 at 3:34 AM, Scott Ransom wrote: > >> > >> > >> On the subject of the SOFA license conditions, our Board member Patrick > >> Wallace is continuing discussions he had on this topic last year, which > >> resulted in changes to the license that enabled the Debian release to > >> proceed. Recent emails, which unfortunately were delayed by the spam > >> filter problems, show that further discussion is needed, and Patrick is > >> now in touch with Perry Greenfield, STScI Science Software Branch lead. > >> The nub of the problem is that SOFA software has to address two > >> conflicting requirements: (i) the insistence by free software groups > >> that users should not be constrained in any way and (ii) the need to > >> prevent "counterfeit" versions coming into circulation. The second > >> point is vital because SOFA software represents IAU standards and indeed > >> is cited in other standards such as IERS Conventions. > >> > >> > > > > Regarding the counterfeit argument, isn't this very similar to encryption > > libraries where you want to make sure that you are using the official > > library and not one you found on the internet that happens to have a back > > door? OpenSSL and other libraries deal with this. People are far more > > concerned about using the proper OpenSSL than using SOFA but the underlying > > principal is the same. OpenSSL has a very straightforward licence > > (http://www.openssl.org/source/license.html) which has clearly been approved > > for distribution. > > > > -- > > Tim Jenness > > CCAT Software Manager > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From sransom at nrao.edu Fri Jun 14 12:59:40 2013 From: sransom at nrao.edu (Scott Ransom) Date: Fri, 14 Jun 2013 12:59:40 -0400 Subject: [AstroPy] message from SOFA chair In-Reply-To: References: <5188D8A3.9050707@nrao.edu> <43A19EA6-232C-43E1-8B59-26D360085E21@gmail.com> Message-ID: <51BB4BFC.4090400@nrao.edu> Yeah. They won't budge on the license issue -- and I tried. They are absolutely convinced that they need an IAU "reference" implementation of many of these algorithms that noone can change. But they are fine taking the same code, stripping "SOFA" and/or "IAU" off the routine names, and re-licensing it in a free fashion even if that means that noone uses their (original) code. I have to admit that I don't understand the logic.... Scott On 06/14/2013 12:12 PM, Perry Greenfield wrote: > Yes, as a matter of fact, today we got explicit permission from the Board to relicense SOFA. That will require us to release it under a different name, and using a different prefix for the routine names. You should be hearing about it soon. But they will not be changing their license. > > Perry > > On Jun 14, 2013, at 12:06 PM, Sergio Pascual wrote: > >> Hello, >> >> is there any progress in this topic? I have checked today and the license text in the SOFA webpage remains the same. >> >> Sergio >> >> >> 2013/5/7 Thomas Robitaille >> Just for information, we are currently working towards a solution >> off-list with the SOFA board, and will inform the list once things >> have been settled. So no need to propose a license text at this stage. >> >> Tom >> >> On 7 May 2013 17:15, Wolfgang Kerzendorf wrote: >>> I think what Mario and Tim are proposing is very good as it would alleviate >>> the problems mentioned by Catherine. To speed up the process, we should >>> propose a license text and they could then see if that meets their criteria >>> (make it as hard as possible for them to delay or refuse). >>> >>> Cheers >>> Wolfgang >>> On 2013-05-07, at 10:45 AM, Tim Jenness wrote: >>> >>> Thanks. >>> >>> >>> On Tue, May 7, 2013 at 3:34 AM, Scott Ransom wrote: >>>> >>>> >>>> On the subject of the SOFA license conditions, our Board member Patrick >>>> Wallace is continuing discussions he had on this topic last year, which >>>> resulted in changes to the license that enabled the Debian release to >>>> proceed. Recent emails, which unfortunately were delayed by the spam >>>> filter problems, show that further discussion is needed, and Patrick is >>>> now in touch with Perry Greenfield, STScI Science Software Branch lead. >>>> The nub of the problem is that SOFA software has to address two >>>> conflicting requirements: (i) the insistence by free software groups >>>> that users should not be constrained in any way and (ii) the need to >>>> prevent "counterfeit" versions coming into circulation. The second >>>> point is vital because SOFA software represents IAU standards and indeed >>>> is cited in other standards such as IERS Conventions. >>>> >>>> >>> >>> Regarding the counterfeit argument, isn't this very similar to encryption >>> libraries where you want to make sure that you are using the official >>> library and not one you found on the internet that happens to have a back >>> door? OpenSSL and other libraries deal with this. People are far more >>> concerned about using the proper OpenSSL than using SOFA but the underlying >>> principal is the same. OpenSSL has a very straightforward licence >>> (http://www.openssl.org/source/license.html) which has clearly been approved >>> for distribution. >>> >>> -- >>> Tim Jenness >>> CCAT Software Manager >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: sransom at nrao.edu Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 From stsci.perry at gmail.com Fri Jun 14 13:17:12 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Fri, 14 Jun 2013 13:17:12 -0400 Subject: [AstroPy] message from SOFA chair In-Reply-To: <51BB4BFC.4090400@nrao.edu> References: <5188D8A3.9050707@nrao.edu> <43A19EA6-232C-43E1-8B59-26D360085E21@gmail.com> <51BB4BFC.4090400@nrao.edu> Message-ID: <166957AD-E3B4-43FE-AA98-D3746A041A1C@gmail.com> Neither do I, neither do I. On Jun 14, 2013, at 12:59 PM, Scott Ransom wrote: > Yeah. They won't budge on the license issue -- and I tried. > > They are absolutely convinced that they need an IAU "reference" > implementation of many of these algorithms that noone can change. But > they are fine taking the same code, stripping "SOFA" and/or "IAU" off > the routine names, and re-licensing it in a free fashion even if that > means that noone uses their (original) code. I have to admit that I > don't understand the logic.... > > Scott > > On 06/14/2013 12:12 PM, Perry Greenfield wrote: >> Yes, as a matter of fact, today we got explicit permission from the Board to relicense SOFA. That will require us to release it under a different name, and using a different prefix for the routine names. You should be hearing about it soon. But they will not be changing their license. >> >> Perry >> >> On Jun 14, 2013, at 12:06 PM, Sergio Pascual wrote: >> >>> Hello, >>> >>> is there any progress in this topic? I have checked today and the license text in the SOFA webpage remains the same. >>> >>> Sergio >>> >>> >>> 2013/5/7 Thomas Robitaille >>> Just for information, we are currently working towards a solution >>> off-list with the SOFA board, and will inform the list once things >>> have been settled. So no need to propose a license text at this stage. >>> >>> Tom >>> >>> On 7 May 2013 17:15, Wolfgang Kerzendorf wrote: >>>> I think what Mario and Tim are proposing is very good as it would alleviate >>>> the problems mentioned by Catherine. To speed up the process, we should >>>> propose a license text and they could then see if that meets their criteria >>>> (make it as hard as possible for them to delay or refuse). >>>> >>>> Cheers >>>> Wolfgang >>>> On 2013-05-07, at 10:45 AM, Tim Jenness wrote: >>>> >>>> Thanks. >>>> >>>> >>>> On Tue, May 7, 2013 at 3:34 AM, Scott Ransom wrote: >>>>> >>>>> >>>>> On the subject of the SOFA license conditions, our Board member Patrick >>>>> Wallace is continuing discussions he had on this topic last year, which >>>>> resulted in changes to the license that enabled the Debian release to >>>>> proceed. Recent emails, which unfortunately were delayed by the spam >>>>> filter problems, show that further discussion is needed, and Patrick is >>>>> now in touch with Perry Greenfield, STScI Science Software Branch lead. >>>>> The nub of the problem is that SOFA software has to address two >>>>> conflicting requirements: (i) the insistence by free software groups >>>>> that users should not be constrained in any way and (ii) the need to >>>>> prevent "counterfeit" versions coming into circulation. The second >>>>> point is vital because SOFA software represents IAU standards and indeed >>>>> is cited in other standards such as IERS Conventions. >>>>> >>>>> >>>> >>>> Regarding the counterfeit argument, isn't this very similar to encryption >>>> libraries where you want to make sure that you are using the official >>>> library and not one you found on the internet that happens to have a back >>>> door? OpenSSL and other libraries deal with this. People are far more >>>> concerned about using the proper OpenSSL than using SOFA but the underlying >>>> principal is the same. OpenSSL has a very straightforward licence >>>> (http://www.openssl.org/source/license.html) which has clearly been approved >>>> for distribution. >>>> >>>> -- >>>> Tim Jenness >>>> CCAT Software Manager >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>>> >>>> >>>> >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > -- > Scott M. Ransom Address: NRAO > Phone: (434) 296-0320 520 Edgemont Rd. > email: sransom at nrao.edu Charlottesville, VA 22903 USA > GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From nchlsearl at gmail.com Sat Jun 15 00:51:09 2013 From: nchlsearl at gmail.com (Nicholas Earl) Date: Fri, 14 Jun 2013 21:51:09 -0700 Subject: [AstroPy] PyFITS Too many open files Message-ID: Hello everyone, I'm having an issue with opening and retrieving information for a large amount of fits files. Mainly, I keep running into a mmap.error: [Errno 24] Too many open files error. The code I'm using is: import pyfits for filename in os.listdir(directory): kep_id = filename[4:13] with pyfits.open(directory + "/" + filename, ignore_missing_end=True) as fits: time = fits[1].data["TIME"] energy = fits[1].data["ENERGY"] Any ideas would be greatly appreciated! Nicholas Earl Graduate Research Assistant San Diego State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From nico.nell at gmail.com Sat Jun 15 01:08:42 2013 From: nico.nell at gmail.com (Nico Nell) Date: Fri, 14 Jun 2013 23:08:42 -0600 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: You need to close the files once you have retrieved the data... See here: http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close ~Nick On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: > Hello everyone, > > I'm having an issue with opening and retrieving information for a large > amount of fits files. Mainly, I keep running into a mmap.error: [Errno > 24] Too many open files error. The code I'm using is: > > import pyfits > > for filename in os.listdir(directory): > kep_id = filename[4:13] > > with pyfits.open(directory + "/" + filename, ignore_missing_end=True) as > fits: > time = fits[1].data["TIME"] > energy = fits[1].data["ENERGY"] > > Any ideas would be greatly appreciated! > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jjk at uvic.ca Sat Jun 15 10:19:20 2013 From: jjk at uvic.ca (JJ Kavelaars) Date: Sat, 15 Jun 2013 07:19:20 -0700 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: Shouldn't the 'with' achieve that? JJ On 2013-06-14, at 10:08 PM, Nico Nell wrote: > You need to close the files once you have retrieved the data... > > See here: http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close > > ~Nick > > > On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: >> Hello everyone, >> >> I'm having an issue with opening and retrieving information for a large amount of fits files. Mainly, I keep running into a mmap.error: [Errno 24] Too many open files error. The code I'm using is: >> >> import pyfits >> >> for filename in os.listdir(directory): >> kep_id = filename[4:13] >> >> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) as fits: >> time = fits[1].data["TIME"] >> energy = fits[1].data["ENERGY"] >> >> Any ideas would be greatly appreciated! >> >> Nicholas Earl >> Graduate Research Assistant >> San Diego State University >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From nchlsearl at gmail.com Sat Jun 15 13:29:24 2013 From: nchlsearl at gmail.com (Nicholas Earl) Date: Sat, 15 Jun 2013 10:29:24 -0700 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: Sorry, but it seems I sent this back to only Nice Nell, and not the entire list! ----- Thanks for the reply. I was under the impression that the with command automatically closed the file? Regardless, I went ahead and wrote the script up like this import pyfits for filename in os.listdir(directory): kep_id = filename[4:13] fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) time = fits[1].data["TIME"] energy = fits[1].data["ENERGY"] fits.close() Unfortunately, I get the same error. Nicholas Earl Graduate Research Assistant San Diego State University On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell wrote: > You need to close the files once you have retrieved the data... > > See here: > http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close > > ~Nick > > > On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: > >> Hello everyone, >> >> I'm having an issue with opening and retrieving information for a large >> amount of fits files. Mainly, I keep running into a mmap.error: [Errno >> 24] Too many open files error. The code I'm using is: >> >> import pyfits >> >> for filename in os.listdir(directory): >> kep_id = filename[4:13] >> >> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) as >> fits: >> time = fits[1].data["TIME"] >> energy = fits[1].data["ENERGY"] >> >> Any ideas would be greatly appreciated! >> >> Nicholas Earl >> Graduate Research Assistant >> San Diego State University >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From beaumont at hawaii.edu Sat Jun 15 14:00:33 2013 From: beaumont at hawaii.edu (Chris Beaumont) Date: Sat, 15 Jun 2013 14:00:33 -0400 Subject: [AstroPy] Fwd: PyFITS Too many open files In-Reply-To: References: Message-ID: (Forgot to reply to list as well!) Just a guess, but I think the problem is that pyfits is opening files in "memory map" mode -- that is, the array isn't actually loaded into RAM, but rather kept on disk as long as possible. This is nice when working with big files, but probably involves numpy keeping an internal open fille handle, even after you close the fits file handle (either explicitly or by exiting the with block). If that's the case, then passing "memmap=False" to pyfits.open should solve the problem (but watch out for increased RAM usage!) chris On Sat, Jun 15, 2013 at 1:29 PM, Nicholas Earl wrote: > Sorry, but it seems I sent this back to only Nice Nell, and not the entire > list! > > ----- > > Thanks for the reply. I was under the impression that the with command > automatically closed the file? Regardless, I went ahead and wrote the > script up like this > > import pyfits > > for filename in os.listdir(directory): > kep_id = filename[4:13] > fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) > time = fits[1].data["TIME"] > energy = fits[1].data["ENERGY"] > fits.close() > > Unfortunately, I get the same error. > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > > On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell wrote: > >> You need to close the files once you have retrieved the data... >> >> See here: >> http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close >> >> ~Nick >> >> >> On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: >> >>> Hello everyone, >>> >>> I'm having an issue with opening and retrieving information for a large >>> amount of fits files. Mainly, I keep running into a mmap.error: [Errno >>> 24] Too many open files error. The code I'm using is: >>> >>> import pyfits >>> >>> for filename in os.listdir(directory): >>> kep_id = filename[4:13] >>> >>> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) as >>> fits: >>> time = fits[1].data["TIME"] >>> energy = fits[1].data["ENERGY"] >>> >>> Any ideas would be greatly appreciated! >>> >>> Nicholas Earl >>> Graduate Research Assistant >>> San Diego State University >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- ************************************ Chris Beaumont Graduate Student Institute for Astronomy University of Hawaii at Manoa 2680 Woodlawn Drive Honolulu, HI 96822 www.ifa.hawaii.edu/~beaumont ************************************ -- ************************************ Chris Beaumont Graduate Student Institute for Astronomy University of Hawaii at Manoa 2680 Woodlawn Drive Honolulu, HI 96822 www.ifa.hawaii.edu/~beaumont ************************************ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nico.nell at gmail.com Sat Jun 15 14:01:38 2013 From: nico.nell at gmail.com (Nico Nell) Date: Sat, 15 Jun 2013 12:01:38 -0600 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: Sorry -- I should have read the blip of code better. So, what version of python and pyfits are you using? How many files are you opening? What happens if you add 'memmap=False' in each of your pyfits open() lines. The pyfits mmap stuff is relatively new so maybe there is an issue. ~Nick On Sat, Jun 15, 2013 at 11:29 AM, Nicholas Earl wrote: > Sorry, but it seems I sent this back to only Nice Nell, and not the entire > list! > > ----- > > Thanks for the reply. I was under the impression that the with command > automatically closed the file? Regardless, I went ahead and wrote the > script up like this > > import pyfits > > for filename in os.listdir(directory): > kep_id = filename[4:13] > fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) > time = fits[1].data["TIME"] > energy = fits[1].data["ENERGY"] > fits.close() > > Unfortunately, I get the same error. > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > > On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell wrote: > >> You need to close the files once you have retrieved the data... >> >> See here: >> http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close >> >> ~Nick >> >> >> On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: >> >>> Hello everyone, >>> >>> I'm having an issue with opening and retrieving information for a large >>> amount of fits files. Mainly, I keep running into a mmap.error: [Errno >>> 24] Too many open files error. The code I'm using is: >>> >>> import pyfits >>> >>> for filename in os.listdir(directory): >>> kep_id = filename[4:13] >>> >>> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) as >>> fits: >>> time = fits[1].data["TIME"] >>> energy = fits[1].data["ENERGY"] >>> >>> Any ideas would be greatly appreciated! >>> >>> Nicholas Earl >>> Graduate Research Assistant >>> San Diego State University >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nchlsearl at gmail.com Sat Jun 15 18:21:37 2013 From: nchlsearl at gmail.com (Nicholas Earl) Date: Sat, 15 Jun 2013 15:21:37 -0700 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: Thanks for the ideas! It seems that memmap=False solves the problem, although I can see that it does begin using a large percentage of the memory -- I can only imagine how much if I let it run on the entire set of directories... Is there not any way to flush it all after I close the file? I am using python 2.7.5, and PyFITs 3.1.2. Thanks again, everyone, you're awesome. Nicholas Earl Graduate Research Assistant San Diego State University On Sat, Jun 15, 2013 at 11:01 AM, Nico Nell wrote: > Sorry -- I should have read the blip of code better. > > So, what version of python and pyfits are you using? How many files are > you opening? > > What happens if you add 'memmap=False' in each of your pyfits open() > lines. The pyfits mmap stuff is relatively new so maybe there is an issue. > > ~Nick > > > On Sat, Jun 15, 2013 at 11:29 AM, Nicholas Earl wrote: > >> Sorry, but it seems I sent this back to only Nice Nell, and not the >> entire list! >> >> ----- >> >> Thanks for the reply. I was under the impression that the with command >> automatically closed the file? Regardless, I went ahead and wrote the >> script up like this >> >> import pyfits >> >> for filename in os.listdir(directory): >> kep_id = filename[4:13] >> fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) >> time = fits[1].data["TIME"] >> energy = fits[1].data["ENERGY"] >> fits.close() >> >> Unfortunately, I get the same error. >> >> Nicholas Earl >> Graduate Research Assistant >> San Diego State University >> >> >> On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell wrote: >> >>> You need to close the files once you have retrieved the data... >>> >>> See here: >>> http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close >>> >>> ~Nick >>> >>> >>> On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: >>> >>>> Hello everyone, >>>> >>>> I'm having an issue with opening and retrieving information for a large >>>> amount of fits files. Mainly, I keep running into a mmap.error: [Errno >>>> 24] Too many open files error. The code I'm using is: >>>> >>>> import pyfits >>>> >>>> for filename in os.listdir(directory): >>>> kep_id = filename[4:13] >>>> >>>> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) >>>> as fits: >>>> time = fits[1].data["TIME"] >>>> energy = fits[1].data["ENERGY"] >>>> >>>> Any ideas would be greatly appreciated! >>>> >>>> Nicholas Earl >>>> Graduate Research Assistant >>>> San Diego State University >>>> >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nico.nell at gmail.com Sat Jun 15 18:42:53 2013 From: nico.nell at gmail.com (Nico Nell) Date: Sat, 15 Jun 2013 16:42:53 -0600 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: You can try running the python garbage collector after each file close with gc.collect(). For the memmap issue consider submitting a bug report if you have time: https://github.com/spacetelescope/PyFITS/issues?state=open ~Nick On Sat, Jun 15, 2013 at 4:21 PM, Nicholas Earl wrote: > Thanks for the ideas! It seems that memmap=False solves the problem, > although I can see that it does begin using a large percentage of the > memory -- I can only imagine how much if I let it run on the entire set of > directories... Is there not any way to flush it all after I close the file? > > I am using python 2.7.5, and PyFITs 3.1.2. > > Thanks again, everyone, you're awesome. > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > > On Sat, Jun 15, 2013 at 11:01 AM, Nico Nell wrote: > >> Sorry -- I should have read the blip of code better. >> >> So, what version of python and pyfits are you using? How many files are >> you opening? >> >> What happens if you add 'memmap=False' in each of your pyfits open() >> lines. The pyfits mmap stuff is relatively new so maybe there is an issue. >> >> ~Nick >> >> >> On Sat, Jun 15, 2013 at 11:29 AM, Nicholas Earl wrote: >> >>> Sorry, but it seems I sent this back to only Nice Nell, and not the >>> entire list! >>> >>> ----- >>> >>> Thanks for the reply. I was under the impression that the with command >>> automatically closed the file? Regardless, I went ahead and wrote the >>> script up like this >>> >>> import pyfits >>> >>> for filename in os.listdir(directory): >>> kep_id = filename[4:13] >>> fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) >>> time = fits[1].data["TIME"] >>> energy = fits[1].data["ENERGY"] >>> fits.close() >>> >>> Unfortunately, I get the same error. >>> >>> Nicholas Earl >>> Graduate Research Assistant >>> San Diego State University >>> >>> >>> On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell wrote: >>> >>>> You need to close the files once you have retrieved the data... >>>> >>>> See here: >>>> http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close >>>> >>>> ~Nick >>>> >>>> >>>> On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: >>>> >>>>> Hello everyone, >>>>> >>>>> I'm having an issue with opening and retrieving information for a >>>>> large amount of fits files. Mainly, I keep running into a mmap.error: >>>>> [Errno 24] Too many open files error. The code I'm using is: >>>>> >>>>> import pyfits >>>>> >>>>> for filename in os.listdir(directory): >>>>> kep_id = filename[4:13] >>>>> >>>>> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) >>>>> as fits: >>>>> time = fits[1].data["TIME"] >>>>> energy = fits[1].data["ENERGY"] >>>>> >>>>> Any ideas would be greatly appreciated! >>>>> >>>>> Nicholas Earl >>>>> Graduate Research Assistant >>>>> San Diego State University >>>>> >>>>> _______________________________________________ >>>>> AstroPy mailing list >>>>> AstroPy at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/astropy >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonysmith80 at gmail.com Mon Jun 17 15:31:18 2013 From: anthonysmith80 at gmail.com (Anthony Smith) Date: Mon, 17 Jun 2013 20:31:18 +0100 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: Hi all, I haven't been looking into this properly, but if you copy the arrays, then surely it frees the files to be closed? E.g., time = fits[1].data["TIME"].copy() rather than time = fits[1].data["TIME"] Worth a try... Anthony 2013/6/15 Nico Nell > You can try running the python garbage collector after each file close > with gc.collect(). > > For the memmap issue consider submitting a bug report if you have time: > https://github.com/spacetelescope/PyFITS/issues?state=open > > ~Nick > > > On Sat, Jun 15, 2013 at 4:21 PM, Nicholas Earl wrote: > >> Thanks for the ideas! It seems that memmap=False solves the problem, >> although I can see that it does begin using a large percentage of the >> memory -- I can only imagine how much if I let it run on the entire set of >> directories... Is there not any way to flush it all after I close the file? >> >> I am using python 2.7.5, and PyFITs 3.1.2. >> >> Thanks again, everyone, you're awesome. >> >> Nicholas Earl >> Graduate Research Assistant >> San Diego State University >> >> >> On Sat, Jun 15, 2013 at 11:01 AM, Nico Nell wrote: >> >>> Sorry -- I should have read the blip of code better. >>> >>> So, what version of python and pyfits are you using? How many files are >>> you opening? >>> >>> What happens if you add 'memmap=False' in each of your pyfits open() >>> lines. The pyfits mmap stuff is relatively new so maybe there is an issue. >>> >>> ~Nick >>> >>> >>> On Sat, Jun 15, 2013 at 11:29 AM, Nicholas Earl wrote: >>> >>>> Sorry, but it seems I sent this back to only Nice Nell, and not the >>>> entire list! >>>> >>>> ----- >>>> >>>> Thanks for the reply. I was under the impression that the with command >>>> automatically closed the file? Regardless, I went ahead and wrote the >>>> script up like this >>>> >>>> import pyfits >>>> >>>> for filename in os.listdir(directory): >>>> kep_id = filename[4:13] >>>> fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) >>>> time = fits[1].data["TIME"] >>>> energy = fits[1].data["ENERGY"] >>>> fits.close() >>>> >>>> Unfortunately, I get the same error. >>>> >>>> Nicholas Earl >>>> Graduate Research Assistant >>>> San Diego State University >>>> >>>> >>>> On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell wrote: >>>> >>>>> You need to close the files once you have retrieved the data... >>>>> >>>>> See here: >>>>> http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close >>>>> >>>>> ~Nick >>>>> >>>>> >>>>> On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl wrote: >>>>> >>>>>> Hello everyone, >>>>>> >>>>>> I'm having an issue with opening and retrieving information for a >>>>>> large amount of fits files. Mainly, I keep running into a mmap.error: >>>>>> [Errno 24] Too many open files error. The code I'm using is: >>>>>> >>>>>> import pyfits >>>>>> >>>>>> for filename in os.listdir(directory): >>>>>> kep_id = filename[4:13] >>>>>> >>>>>> with pyfits.open(directory + "/" + filename, ignore_missing_end=True) >>>>>> as fits: >>>>>> time = fits[1].data["TIME"] >>>>>> energy = fits[1].data["ENERGY"] >>>>>> >>>>>> Any ideas would be greatly appreciated! >>>>>> >>>>>> Nicholas Earl >>>>>> Graduate Research Assistant >>>>>> San Diego State University >>>>>> >>>>>> _______________________________________________ >>>>>> AstroPy mailing list >>>>>> AstroPy at scipy.org >>>>>> http://mail.scipy.org/mailman/listinfo/astropy >>>>>> >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kellecruz at gmail.com Mon Jun 17 16:56:59 2013 From: kellecruz at gmail.com (Kelle Cruz) Date: Mon, 17 Jun 2013 16:56:59 -0400 Subject: [AstroPy] Status of Specutils? Message-ID: Hi Astropy peeps, I'm planning on having my summer students work on analysis of spectra. Pretty basic stuff like measuring equivalent widths and spectral indices (aka, ratios)?things that IRAF SPLOT does interactively. What's the status of the specutils package? Can we volunteer to be beta testers? Or should we be contributing code somewhere? Last summer we tried to use Sherpa, and that was a disaster (sorry Sherpa people who might be on this list). thanks for any updates or advice. kelle, in constant pursuit of avoiding re-inventing the wheel -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkerzendorf at gmail.com Mon Jun 17 17:13:10 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Mon, 17 Jun 2013 17:13:10 -0400 Subject: [AstroPy] Status of Specutils? In-Reply-To: References: Message-ID: <69167E41-C2CE-4CE4-A3A3-6820639ADE5F@gmail.com> Hi Kelle, Specutils does not have much in it currently. We have a extinction class (for optical extinction) and we have (the beginnings of) a 1d spectrum class. The 1d Spectrum class needs some love as the WCS it currently uses needs to be ported over to the new astropy models class (which is supposedly going to be the base class for all). I haven't gotten around to do that yet, however that should not keep you or your students to provide stuff for the specutils package. I imagine building a equivalent widths model (using the existing 1d gauss model class, https://github.com/astropy/astropy/blob/master/astropy/modeling/functional_models.py) as a first step and playing around with using it with a fitter. The modelling package currently is in the late design phase so lots of people trying to build something with it, definitely is a good thing. your thoughts? Hope that helps, Wolfgang On 2013-06-17, at 4:56 PM, Kelle Cruz wrote: > Hi Astropy peeps, > > I'm planning on having my summer students work on analysis of spectra. Pretty basic stuff like measuring equivalent widths and spectral indices (aka, ratios)?things that IRAF SPLOT does interactively. What's the status of the specutils package? Can we volunteer to be beta testers? Or should we be contributing code somewhere? Last summer we tried to use Sherpa, and that was a disaster (sorry Sherpa people who might be on this list). > > thanks for any updates or advice. > > kelle, in constant pursuit of avoiding re-inventing the wheel > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From thomas.robitaille at gmail.com Tue Jun 18 12:15:55 2013 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Tue, 18 Jun 2013 18:15:55 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 Message-ID: Hi everyone, The deadline for mentoring organizations for the ESA Summer of Code in Space (SOCIS) 2013 is this coming Thursday. http://sophia.estec.esa.int/socis2013/ Since we were successful with the Google Summer of Code, the Astropy project will not be participating as a whole, but if you are leading the development of a Python package for astronomy, and are interested in having a student work on your project, you should consider applying! As part of this program, students are paid 4000 euros. Cheers, Tom From jslavin at cfa.harvard.edu Tue Jun 18 13:54:03 2013 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Tue, 18 Jun 2013 13:54:03 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 Message-ID: Hi all, Tom's post reminded me of what I was thinking after Kelle's post earlier today: a really good project for someone would be to create a matplotlib-based spectral line fitting package. Maybe this ESA opportunity would work for that. Like many others I have my own routines that I've written to do this, but they are neither general enough nor user-friendly enough to be widely usable. I think that a focused effort on this could produce something that would be very helpful to many people. The requirements would be (just off the top of my head): - gui interface, for example for placement of line centers - general methods for inputting data (fits, hdf5, ascii table) - robust fitting routines that return error bars on fitted parameters - a range of possible line shape functions (e.g. gaussian, lorentz, etc.) and continuum fitting - able to deal with both emission and absorption lines I hope someone out there is interested in doing something like that. Jon ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ On Tue, Jun 18, 2013 at 1:00 PM, wrote: > > Date: Tue, 18 Jun 2013 18:15:55 +0200 > From: Thomas Robitaille > Subject: [AstroPy] ESA Summer of Code in Space 2013 > To: astropy mailing list , astropy-dev mailing > list > Message-ID: > < > CAGMHX_3b6NfY-L-OHhZqjcNFSARf3K0EMOjE0u1QWnb4nYX_+Q at mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Hi everyone, > > The deadline for mentoring organizations for the ESA Summer of Code in > Space (SOCIS) 2013 is this coming Thursday. > > http://sophia.estec.esa.int/socis2013/ > > Since we were successful with the Google Summer of Code, the Astropy > project will not be participating as a whole, but if you are leading > the development of a Python package for astronomy, and are interested > in having a student work on your project, you should consider > applying! As part of this program, students are paid 4000 euros. > > Cheers, > Tom > > > ------------------------------ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > End of AstroPy Digest, Vol 81, Issue 10 > *************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkerzendorf at gmail.com Tue Jun 18 14:01:24 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Tue, 18 Jun 2013 14:01:24 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: Hi Jonathan, I think that's a fantastic idea! Are you raising your hand to put the application in ? I'm happy to help with some of the sections. Maybe Kelle is interested as well to join this effort. The workload divided by three might be worth it. What do you guys think? Cheers Wolfgang On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" wrote: > Hi all, > > Tom's post reminded me of what I was thinking after Kelle's post earlier today: a really good project for someone would be to create a matplotlib-based spectral line fitting package. Maybe this ESA opportunity would work for that. Like many others I have my own routines that I've written to do this, but they are neither general enough nor user-friendly enough to be widely usable. I think that a focused effort on this could produce something that would be very helpful to many people. The requirements would be (just off the top of my head): > - gui interface, for example for placement of line centers > - general methods for inputting data (fits, hdf5, ascii table) > - robust fitting routines that return error bars on fitted parameters > - a range of possible line shape functions (e.g. gaussian, lorentz, etc.) and continuum fitting > - able to deal with both emission and absorption lines > > I hope someone out there is interested in doing something like that. > > Jon > > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > On Tue, Jun 18, 2013 at 1:00 PM, wrote: > Date: Tue, 18 Jun 2013 18:15:55 +0200 > From: Thomas Robitaille > Subject: [AstroPy] ESA Summer of Code in Space 2013 > To: astropy mailing list , astropy-dev mailing > list > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi everyone, > > The deadline for mentoring organizations for the ESA Summer of Code in > Space (SOCIS) 2013 is this coming Thursday. > > http://sophia.estec.esa.int/socis2013/ > > Since we were successful with the Google Summer of Code, the Astropy > project will not be participating as a whole, but if you are leading > the development of a Python package for astronomy, and are interested > in having a student work on your project, you should consider > applying! As part of this program, students are paid 4000 euros. > > Cheers, > Tom > > > ------------------------------ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > End of AstroPy Digest, Vol 81, Issue 10 > *************************************** > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From jslavin at cfa.harvard.edu Tue Jun 18 14:07:26 2013 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Tue, 18 Jun 2013 14:07:26 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 Message-ID: Hi Wolfgang, Well, I'm far from being a student and am not a European. So I think that disqualifies me. And I have my hands full just doing the work that I get paid for. I'd like to help in some way, though. I'd be willing to provide feedback and suggestions on the code. Jon ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ On Tue, Jun 18, 2013 at 2:01 PM, Wolfgang Kerzendorf wrote: > Hi Jonathan, > > I think that's a fantastic idea! Are you raising your hand to put the > application in ? I'm happy to help with some of the sections. Maybe Kelle > is interested as well to join this effort. The workload divided by three > might be worth it. > > What do you guys think? > > Cheers > Wolfgang > > On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" > wrote: > > Hi all, > > Tom's post reminded me of what I was thinking after Kelle's post earlier > today: a really good project for someone would be to create a > matplotlib-based spectral line fitting package. Maybe this ESA opportunity > would work for that. Like many others I have my own routines that I've > written to do this, but they are neither general enough nor user-friendly > enough to be widely usable. I think that a focused effort on this could > produce something that would be very helpful to many people. The > requirements would be (just off the top of my head): > - gui interface, for example for placement of line centers > - general methods for inputting data (fits, hdf5, ascii table) > - robust fitting routines that return error bars on fitted parameters > - a range of possible line shape functions (e.g. gaussian, lorentz, etc.) > and continuum fitting > - able to deal with both emission and absorption lines > > I hope someone out there is interested in doing something like that. > > Jon > > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > On Tue, Jun 18, 2013 at 1:00 PM, wrote: >> >> Date: Tue, 18 Jun 2013 18:15:55 +0200 >> From: Thomas Robitaille >> Subject: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy mailing list , astropy-dev mailing >> list >> Message-ID: >> < >> CAGMHX_3b6NfY-L-OHhZqjcNFSARf3K0EMOjE0u1QWnb4nYX_+Q at mail.gmail.com> >> Content-Type: text/plain; charset=ISO-8859-1 >> >> Hi everyone, >> >> The deadline for mentoring organizations for the ESA Summer of Code in >> Space (SOCIS) 2013 is this coming Thursday. >> >> http://sophia.estec.esa.int/socis2013/ >> >> Since we were successful with the Google Summer of Code, the Astropy >> project will not be participating as a whole, but if you are leading >> the development of a Python package for astronomy, and are interested >> in having a student work on your project, you should consider >> applying! As part of this program, students are paid 4000 euros. >> >> Cheers, >> Tom >> >> >> ------------------------------ >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> End of AstroPy Digest, Vol 81, Issue 10 >> *************************************** >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkerzendorf at gmail.com Tue Jun 18 14:14:34 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Tue, 18 Jun 2013 14:14:34 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: Hi Jonathan, I'm aware of you not being a student - I meant to put an application in for astropy/specutils as a mentoring organization. That's the first step. I'm going to take this offline, whoever is interested in a discussion for applying as a mentoring organization to ESA please contact me privately. Cheers, Wolfgang On 2013-06-18, at 2:07 PM, "Slavin, Jonathan" wrote: > Hi Wolfgang, > > Well, I'm far from being a student and am not a European. So I think that disqualifies me. And I have my hands full just doing the work that I get paid for. I'd like to help in some way, though. I'd be willing to provide feedback and suggestions on the code. > > Jon > > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > On Tue, Jun 18, 2013 at 2:01 PM, Wolfgang Kerzendorf wrote: > Hi Jonathan, > > I think that's a fantastic idea! Are you raising your hand to put the application in ? I'm happy to help with some of the sections. Maybe Kelle is interested as well to join this effort. The workload divided by three might be worth it. > > What do you guys think? > > Cheers > Wolfgang > > On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" wrote: > >> Hi all, >> >> Tom's post reminded me of what I was thinking after Kelle's post earlier today: a really good project for someone would be to create a matplotlib-based spectral line fitting package. Maybe this ESA opportunity would work for that. Like many others I have my own routines that I've written to do this, but they are neither general enough nor user-friendly enough to be widely usable. I think that a focused effort on this could produce something that would be very helpful to many people. The requirements would be (just off the top of my head): >> - gui interface, for example for placement of line centers >> - general methods for inputting data (fits, hdf5, ascii table) >> - robust fitting routines that return error bars on fitted parameters >> - a range of possible line shape functions (e.g. gaussian, lorentz, etc.) and continuum fitting >> - able to deal with both emission and absorption lines >> >> I hope someone out there is interested in doing something like that. >> >> Jon >> >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >> phone: (617) 496-7981 Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> On Tue, Jun 18, 2013 at 1:00 PM, wrote: >> Date: Tue, 18 Jun 2013 18:15:55 +0200 >> From: Thomas Robitaille >> Subject: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy mailing list , astropy-dev mailing >> list >> Message-ID: >> >> Content-Type: text/plain; charset=ISO-8859-1 >> >> Hi everyone, >> >> The deadline for mentoring organizations for the ESA Summer of Code in >> Space (SOCIS) 2013 is this coming Thursday. >> >> http://sophia.estec.esa.int/socis2013/ >> >> Since we were successful with the Google Summer of Code, the Astropy >> project will not be participating as a whole, but if you are leading >> the development of a Python package for astronomy, and are interested >> in having a student work on your project, you should consider >> applying! As part of this program, students are paid 4000 euros. >> >> Cheers, >> Tom >> >> >> ------------------------------ >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> End of AstroPy Digest, Vol 81, Issue 10 >> *************************************** >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From thoger.emil at gmail.com Tue Jun 18 14:39:55 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Tue, 18 Jun 2013 20:39:55 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: <51C0A97B.8090703@gmail.com> I have been working on a fitting GUI for a while, although it is made with a specific task in mind. However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and Pandas. It is made for a specific projhect I'm working and as such not yet usable for more general cases, but it could be a starting point, if the dependencies don't conflict with astropy politics. Especially, I am happy about the choice of Pandas for managing a quite complex data structure (the fitted and/or guessed values of an arbitrary number of transitions for an arbitrary number of rows or collapsed rows of a spatially resolved spectrum) of a), but also with the Traits-based interactive interface to build complex line profiles from single gaussians, good for fitting-by-eye and giving good initial guesses for fitting of complex line profiles. It hooks directly up to a wrapper I've made for lmfit, but given the modularity, it should be relatively easy to change to other backends. It's still a work-in-progress, but there are some screenshots here: http://flic.kr/s/aHsjGaEMGg . I know the choice and number of dependencies may be prohibitive but it saved a lot of work on the GUI, and Pandas means the difference between sanity and madness when it comes to keeping track of so many parameters. Cheers, Emil On 2013-06-18 20:01, Wolfgang Kerzendorf wrote: > Hi Jonathan, > > I think that's a fantastic idea! Are you raising your hand to put the > application in ? I'm happy to help with some of the sections. Maybe > Kelle is interested as well to join this effort. The workload divided > by three might be worth it. > > What do you guys think? > > Cheers > Wolfgang > On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" > wrote: > >> Hi all, >> >> Tom's post reminded me of what I was thinking after Kelle's post >> earlier today: a really good project for someone would be to create a >> matplotlib-based spectral line fitting package. Maybe this ESA >> opportunity would work for that. Like many others I have my own >> routines that I've written to do this, but they are neither general >> enough nor user-friendly enough to be widely usable. I think that a >> focused effort on this could produce something that would be very >> helpful to many people. The requirements would be (just off the top >> of my head): >> - gui interface, for example for placement of line centers >> - general methods for inputting data (fits, hdf5, ascii table) >> - robust fitting routines that return error bars on fitted parameters >> - a range of possible line shape functions (e.g. gaussian, lorentz, >> etc.) and continuum fitting >> - able to deal with both emission and absorption lines >> >> I hope someone out there is interested in doing something like that. >> >> Jon >> >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 Garden >> Street, MS 83 >> phone: (617) 496-7981 Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> On Tue, Jun 18, 2013 at 1:00 PM, > > wrote: >> >> Date: Tue, 18 Jun 2013 18:15:55 +0200 >> From: Thomas Robitaille > > >> Subject: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy mailing list > >, astropy-dev mailing >> list > > >> Message-ID: >> >> > > >> Content-Type: text/plain; charset=ISO-8859-1 >> >> Hi everyone, >> >> The deadline for mentoring organizations for the ESA Summer of >> Code in >> Space (SOCIS) 2013 is this coming Thursday. >> >> http://sophia.estec.esa.int/socis2013/ >> >> Since we were successful with the Google Summer of Code, the Astropy >> project will not be participating as a whole, but if you are leading >> the development of a Python package for astronomy, and are interested >> in having a student work on your project, you should consider >> applying! As part of this program, students are paid 4000 euros. >> >> Cheers, >> Tom >> >> >> ------------------------------ >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> End of AstroPy Digest, Vol 81, Issue 10 >> *************************************** >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkerzendorf at gmail.com Tue Jun 18 14:50:01 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Tue, 18 Jun 2013 14:50:01 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: <51C0A97B.8090703@gmail.com> References: <51C0A97B.8090703@gmail.com> Message-ID: <4A8F7017-BA52-451F-A23C-AD728F847B70@gmail.com> Hi Emil, I fully agree that PANDAS is the choice between sanity and madness as I've discovered a couple of months ago! I think as there are some other tools being developed now in matplotlib (thinking of Tom R's WCSAxes), it would be nice to stick to that. I'm not arguing that traits is a bad choice, just from a unifying standpoint matplotlib makes sense. It would be great to get some of your expertise on this mentor application. Interested? Cheers, Wolfgang On 2013-06-18, at 2:39 PM, Th?ger Rivera-Thorsen wrote: > I have been working on a fitting GUI for a while, although it is made with a specific task in mind. > However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and Pandas. It is made for a specific projhect I'm working and as such not yet usable for more general cases, but it could be a starting point, if the dependencies don't conflict with astropy politics. > > Especially, I am happy about the choice of Pandas for managing a quite complex data structure (the fitted and/or guessed values of an arbitrary number of transitions for an arbitrary number of rows or collapsed rows of a spatially resolved spectrum) of a), but also with the Traits-based interactive interface to build complex line profiles from single gaussians, good for fitting-by-eye and giving good initial guesses for fitting of complex line profiles. It hooks directly up to a wrapper I've made for lmfit, but given the modularity, it should be relatively easy to change to other backends. > > It's still a work-in-progress, but there are some screenshots here: http://flic.kr/s/aHsjGaEMGg . > I know the choice and number of dependencies may be prohibitive but it saved a lot of work on the GUI, and Pandas means the difference between sanity and madness when it comes to keeping track of so many parameters. > > Cheers, > Emil > > > On 2013-06-18 20:01, Wolfgang Kerzendorf wrote: >> Hi Jonathan, >> >> I think that's a fantastic idea! Are you raising your hand to put the application in ? I'm happy to help with some of the sections. Maybe Kelle is interested as well to join this effort. The workload divided by three might be worth it. >> >> What do you guys think? >> >> Cheers >> Wolfgang >> On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" wrote: >> >>> Hi all, >>> >>> Tom's post reminded me of what I was thinking after Kelle's post earlier today: a really good project for someone would be to create a matplotlib-based spectral line fitting package. Maybe this ESA opportunity would work for that. Like many others I have my own routines that I've written to do this, but they are neither general enough nor user-friendly enough to be widely usable. I think that a focused effort on this could produce something that would be very helpful to many people. The requirements would be (just off the top of my head): >>> - gui interface, for example for placement of line centers >>> - general methods for inputting data (fits, hdf5, ascii table) >>> - robust fitting routines that return error bars on fitted parameters >>> - a range of possible line shape functions (e.g. gaussian, lorentz, etc.) and continuum fitting >>> - able to deal with both emission and absorption lines >>> >>> I hope someone out there is interested in doing something like that. >>> >>> Jon >>> >>> ________________________________________________________ >>> Jonathan D. Slavin Harvard-Smithsonian CfA >>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >>> phone: (617) 496-7981 Cambridge, MA 02138-1516 >>> fax: (617) 496-7577 USA >>> ________________________________________________________ >>> >>> >>> >>> On Tue, Jun 18, 2013 at 1:00 PM, wrote: >>> Date: Tue, 18 Jun 2013 18:15:55 +0200 >>> From: Thomas Robitaille >>> Subject: [AstroPy] ESA Summer of Code in Space 2013 >>> To: astropy mailing list , astropy-dev mailing >>> list >>> Message-ID: >>> >>> Content-Type: text/plain; charset=ISO-8859-1 >>> >>> Hi everyone, >>> >>> The deadline for mentoring organizations for the ESA Summer of Code in >>> Space (SOCIS) 2013 is this coming Thursday. >>> >>> http://sophia.estec.esa.int/socis2013/ >>> >>> Since we were successful with the Google Summer of Code, the Astropy >>> project will not be participating as a whole, but if you are leading >>> the development of a Python package for astronomy, and are interested >>> in having a student work on your project, you should consider >>> applying! As part of this program, students are paid 4000 euros. >>> >>> Cheers, >>> Tom >>> >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >>> End of AstroPy Digest, Vol 81, Issue 10 >>> *************************************** >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From jslavin at cfa.harvard.edu Tue Jun 18 14:53:56 2013 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Tue, 18 Jun 2013 14:53:56 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: Message-ID: Hi Emil, That looks very nice! I don't see Pandas as a big issue in terms of dependencies. I don't know that much about traits, etc. My thought about the gui was just based on my experience with matplotlib, and the fact that it is widely used -- though I would agree that too many dependencies can be a deterrent to people using something. Are you offering your code as a starting point for the project? It strikes me that many have gotten some sort of fitting package to a point of personal usability but no one has the time/interest/motivation to make a more generally usable package. Jon On Tue, Jun 18, 2013 at 2:34 PM, wrote: > Date: Tue, 18 Jun 2013 20:39:55 +0200 > From: Th?ger Rivera-Thorsen > Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > To: astropy at scipy.org > Message-ID: <51C0A97B.8090703 at gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > I have been working on a fitting GUI for a while, although it is made > with a specific task in mind. > However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and > Pandas. It is made for a specific projhect I'm working and as such not > yet usable for more general cases, but it could be a starting point, if > the dependencies don't conflict with astropy politics. > > Especially, I am happy about the choice of Pandas for managing a quite > complex data structure (the fitted and/or guessed values of an arbitrary > number of transitions for an arbitrary number of rows or collapsed rows > of a spatially resolved spectrum) of a), but also with the Traits-based > interactive interface to build complex line profiles from single > gaussians, good for fitting-by-eye and giving good initial guesses for > fitting of complex line profiles. It hooks directly up to a wrapper I've > made for lmfit, but given the modularity, it should be relatively easy > to change to other backends. > > It's still a work-in-progress, but there are some screenshots here: > http://flic.kr/s/aHsjGaEMGg . > I know the choice and number of dependencies may be prohibitive but it > saved a lot of work on the GUI, and Pandas means the difference between > sanity and madness when it comes to keeping track of so many parameters. > > Cheers, > Emil > ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoger.emil at gmail.com Tue Jun 18 15:04:27 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Tue, 18 Jun 2013 21:04:27 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: <51C0AF3B.9040604@gmail.com> Btw., if that is interestting, I have also written a scipy-only-dependency implementation of Tepper-Garcia (2006) Voigt profile approximation, if that could be of interest for e.g. the modeling module or maybe a spectools module. The upside to it is it is precise and is expressed in terms of physical units - column density and broadening parameter - and hence can be fed directly to software like e.g. VPfit, and it is easdily interpreted. The downside to this is ithat it needs information about the line's resonator strength and rest wavelength, the former found by aid of the latter. For ths I am so far for my personal use depending on the list from VPfit, but any list of similar format would of course do. /Emil On 2013-06-18 20:01, Wolfgang Kerzendorf wrote: > Hi Jonathan, > > I think that's a fantastic idea! Are you raising your hand to put the > application in ? I'm happy to help with some of the sections. Maybe > Kelle is interested as well to join this effort. The workload divided > by three might be worth it. > > What do you guys think? > > Cheers > Wolfgang > On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" > wrote: > >> Hi all, >> >> Tom's post reminded me of what I was thinking after Kelle's post >> earlier today: a really good project for someone would be to create a >> matplotlib-based spectral line fitting package. Maybe this ESA >> opportunity would work for that. Like many others I have my own >> routines that I've written to do this, but they are neither general >> enough nor user-friendly enough to be widely usable. I think that a >> focused effort on this could produce something that would be very >> helpful to many people. The requirements would be (just off the top >> of my head): >> - gui interface, for example for placement of line centers >> - general methods for inputting data (fits, hdf5, ascii table) >> - robust fitting routines that return error bars on fitted parameters >> - a range of possible line shape functions (e.g. gaussian, lorentz, >> etc.) and continuum fitting >> - able to deal with both emission and absorption lines >> >> I hope someone out there is interested in doing something like that. >> >> Jon >> >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 Garden >> Street, MS 83 >> phone: (617) 496-7981 Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> On Tue, Jun 18, 2013 at 1:00 PM, > > wrote: >> >> Date: Tue, 18 Jun 2013 18:15:55 +0200 >> From: Thomas Robitaille > > >> Subject: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy mailing list > >, astropy-dev mailing >> list > > >> Message-ID: >> >> > > >> Content-Type: text/plain; charset=ISO-8859-1 >> >> Hi everyone, >> >> The deadline for mentoring organizations for the ESA Summer of >> Code in >> Space (SOCIS) 2013 is this coming Thursday. >> >> http://sophia.estec.esa.int/socis2013/ >> >> Since we were successful with the Google Summer of Code, the Astropy >> project will not be participating as a whole, but if you are leading >> the development of a Python package for astronomy, and are interested >> in having a student work on your project, you should consider >> applying! As part of this program, students are paid 4000 euros. >> >> Cheers, >> Tom >> >> >> ------------------------------ >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> End of AstroPy Digest, Vol 81, Issue 10 >> *************************************** >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From trive at astro.su.se Tue Jun 18 15:29:33 2013 From: trive at astro.su.se (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Tue, 18 Jun 2013 21:29:33 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: <4A8F7017-BA52-451F-A23C-AD728F847B70@gmail.com> References: <51C0A97B.8090703@gmail.com> <4A8F7017-BA52-451F-A23C-AD728F847B70@gmail.com> Message-ID: <51C0B51D.4070407@astro.su.se> I'm not exatly sure I understand your comments here, so a few questions: On 2013-06-18 20:50, Wolfgang Kerzendorf wrote: > Hi Emil, > > I fully agree that PANDAS is the choice between sanity and madness as > I've discovered a couple of months ago! I think as there are some > other tools being developed now in matplotlib (thinking of Tom R's > WCSAxes), it would be nice to stick to that. You mean it would be nice to sick with Matplotlib (but not to ditch pandas as the backend, is that it?). I am sure that these tools are nice, my choicce pof chaco was from a GUI-techjnical point of view. I use Traits for all the quantities that can be manipulated in the GUI, because it is a huge simplification compared to the "grown-up" toolkits; and Chaco is well integrated with and aware of Traits. Of course, it the goal is many features and tight integration with astropy, matplotlib can possibly be a better tool, but it is not a GUI toolkit and should not be used as such, and it requires quite a bit more work to use, I think. > I'm not arguing that traits is a bad choice, just from a unifying > standpoint matplotlib makes sense. It should be stressed that matplotlib and traits fill two completely different functions. Traits simplifies the automatic updating of parameters that depend on other parameters, and provides a way to easily represent a quantity with an appropriate GUI element (boolean - tick box, integer as either drop-down box or range slider, etc.). It is not a plotting library - that is Chaco. Matplotlib can be integrated with Traits, but it is a bit hairy to do, I quickly gave up and went with Chaco instead. > > It would be great to get some of your expertise on this mentor > application. Interested? Yes, but I'm not sure exactly what you are asking me about :) /Emil > > Cheers, > Wolfgang > > On 2013-06-18, at 2:39 PM, Th?ger Rivera-Thorsen > > wrote: > >> I have been working on a fitting GUI for a while, although it is made >> with a specific task in mind. >> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco >> and Pandas. It is made for a specific projhect I'm working and as >> such not yet usable for more general cases, but it could be a >> starting point, if the dependencies don't conflict with astropy politics. >> >> Especially, I am happy about the choice of Pandas for managing a >> quite complex data structure (the fitted and/or guessed values of an >> arbitrary number of transitions for an arbitrary number of rows or >> collapsed rows of a spatially resolved spectrum) of a), but also with >> the Traits-based interactive interface to build complex line profiles >> from single gaussians, good for fitting-by-eye and giving good >> initial guesses for fitting of complex line profiles. It hooks >> directly up to a wrapper I've made for lmfit, but given the >> modularity, it should be relatively easy to change to other backends. >> >> It's still a work-in-progress, but there are some screenshots here: >> http://flic.kr/s/aHsjGaEMGg . >> I know the choice and number of dependencies may be prohibitive but >> it saved a lot of work on the GUI, and Pandas means the difference >> between sanity and madness when it comes to keeping track of so many >> parameters. >> >> Cheers, >> Emil >> >> >> On 2013-06-18 20:01, Wolfgang Kerzendorf wrote: >>> Hi Jonathan, >>> >>> I think that's a fantastic idea! Are you raising your hand to put >>> the application in ? I'm happy to help with some of the sections. >>> Maybe Kelle is interested as well to join this effort. The workload >>> divided by three might be worth it. >>> >>> What do you guys think? >>> >>> Cheers >>> Wolfgang >>> On 2013-06-18, at 1:54 PM, "Slavin, Jonathan" >>> > wrote: >>> >>>> Hi all, >>>> >>>> Tom's post reminded me of what I was thinking after Kelle's post >>>> earlier today: a really good project for someone would be to create >>>> a matplotlib-based spectral line fitting package. Maybe this ESA >>>> opportunity would work for that. Like many others I have my own >>>> routines that I've written to do this, but they are neither general >>>> enough nor user-friendly enough to be widely usable. I think that >>>> a focused effort on this could produce something that would be very >>>> helpful to many people. The requirements would be (just off the >>>> top of my head): >>>> - gui interface, for example for placement of line centers >>>> - general methods for inputting data (fits, hdf5, ascii table) >>>> - robust fitting routines that return error bars on fitted parameters >>>> - a range of possible line shape functions (e.g. gaussian, lorentz, >>>> etc.) and continuum fitting >>>> - able to deal with both emission and absorption lines >>>> >>>> I hope someone out there is interested in doing something like that. >>>> >>>> Jon >>>> >>>> ________________________________________________________ >>>> Jonathan D. Slavin Harvard-Smithsonian CfA >>>> jslavin at cfa.harvard.edu 60 >>>> Garden Street, MS 83 >>>> phone: (617) 496-7981 Cambridge, MA 02138-1516 >>>> fax: (617) 496-7577 USA >>>> ________________________________________________________ >>>> >>>> >>>> >>>> On Tue, Jun 18, 2013 at 1:00 PM, >>> > wrote: >>>> >>>> Date: Tue, 18 Jun 2013 18:15:55 +0200 >>>> From: Thomas Robitaille >>> > >>>> Subject: [AstroPy] ESA Summer of Code in Space 2013 >>>> To: astropy mailing list >>> >, astropy-dev mailing >>>> list >>> > >>>> Message-ID: >>>> >>>> >>> > >>>> Content-Type: text/plain; charset=ISO-8859-1 >>>> >>>> Hi everyone, >>>> >>>> The deadline for mentoring organizations for the ESA Summer of >>>> Code in >>>> Space (SOCIS) 2013 is this coming Thursday. >>>> >>>> http://sophia.estec.esa.int/socis2013/ >>>> >>>> Since we were successful with the Google Summer of Code, the >>>> Astropy >>>> project will not be participating as a whole, but if you are >>>> leading >>>> the development of a Python package for astronomy, and are >>>> interested >>>> in having a student work on your project, you should consider >>>> applying! As part of this program, students are paid 4000 euros. >>>> >>>> Cheers, >>>> Tom >>>> >>>> >>>> ------------------------------ >>>> >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>>> >>>> >>>> End of AstroPy Digest, Vol 81, Issue 10 >>>> *************************************** >>>> >>>> >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From trive at astro.su.se Tue Jun 18 15:35:54 2013 From: trive at astro.su.se (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Tue, 18 Jun 2013 21:35:54 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: Message-ID: <51C0B69A.6020402@astro.su.se> Pandas is a part of the newly-defined SciPy stack, after all, so that would be part of any science-oriented distribution worth its salt. In fact, I think it could be a good idea for astropy in general to use under the hood, but again, could clash with the philosophy of the project and possibly also maintainabillity. As for offering my code or just my experience, I'll have to square it with my supervisor first, and I also think it depends on what direction the project in question will take. I'm positive about the idea (which is why I wrote in the first place), but supervisor might think it is a better idea to actually get my paper in the project wrapped up before sending the code out there. Will get back about that one! /Emil On 2013-06-18 20:53, Slavin, Jonathan wrote: > Hi Emil, > > That looks very nice! I don't see Pandas as a big issue in terms of > dependencies. I don't know that much about traits, etc. My thought > about the gui was just based on my experience with matplotlib, and the > fact that it is widely used -- though I would agree that too many > dependencies can be a deterrent to people using something. Are you > offering your code as a starting point for the project? It strikes me > that many have gotten some sort of fitting package to a point of > personal usability but no one has the time/interest/motivation to make > a more generally usable package. > > Jon > > On Tue, Jun 18, 2013 at 2:34 PM, > wrote: > > Date: Tue, 18 Jun 2013 20:39:55 +0200 > From: Th?ger Rivera-Thorsen > > Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > To: astropy at scipy.org > Message-ID: <51C0A97B.8090703 at gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > I have been working on a fitting GUI for a while, although it is made > with a specific task in mind. > However, it is not based on Matplotlib but on > Traits/Traitsui/Chaco and > Pandas. It is made for a specific projhect I'm working and as such not > yet usable for more general cases, but it could be a starting > point, if > the dependencies don't conflict with astropy politics. > > Especially, I am happy about the choice of Pandas for managing a quite > complex data structure (the fitted and/or guessed values of an > arbitrary > number of transitions for an arbitrary number of rows or collapsed > rows > of a spatially resolved spectrum) of a), but also with the > Traits-based > interactive interface to build complex line profiles from single > gaussians, good for fitting-by-eye and giving good initial guesses for > fitting of complex line profiles. It hooks directly up to a > wrapper I've > made for lmfit, but given the modularity, it should be relatively easy > to change to other backends. > > It's still a work-in-progress, but there are some screenshots here: > http://flic.kr/s/aHsjGaEMGg . > I know the choice and number of dependencies may be prohibitive but it > saved a lot of work on the GUI, and Pandas means the difference > between > sanity and madness when it comes to keeping track of so many > parameters. > > Cheers, > Emil > > > > > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden > Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at physics.ucf.edu Tue Jun 18 16:53:38 2013 From: jh at physics.ucf.edu (Joe Harrington) Date: Tue, 18 Jun 2013 22:53:38 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: (astropy-request@scipy.org) Message-ID: Just to interject a few quick thoughts into this discussion... Knowing little of the situation on the ground WRT spectral line fitting, it does seem to me that it's been done a million times. If any of those implementations is decent, OSS, and compiled, why not just wrap it? Probably the GUI would want to be in Python, but there's a lot of other stuff under the hood that would be time-consuming and likely slower to re-(re-re-re-)write. At the very least, you might save some time by studying what existing codes do and what people like about them. Jon Slavin> - robust fitting routines that return error bars on fitted parameters Regarding the reporting of fits with errors, as many of us have painfully learned, minimizers don't give these, they only pretend to. Since that's often not good enough, it would be nice to see a relatively plug-n-play MCMC (e.g., DE-) put forward as a fitting package. It would have to evaluate its own distribution, gave errors, and otherwise behaved a bit like lmfit. Yes, there are subtleties to Markov Chains, but this is also true of minimizers. Getting something out there and inviting people to improve it could produce something usable in a few iterations. My group can contribute some DE-MCMC code that someone could adapt. To Jon's list of requirements, I'd add: - Able to use a GUI for user-cue input OR take such input from a text file - Able to write such a text file from the GUI user-cue input, for subsequent runs (and add to it from a second run, etc.) One thing that isn't clear to me from the discussion is whether the scope is merely to identify the center and width vs. pixel number to get a wavelength solution or calculate Doppler shifts, or whether to do the whole job of reading multiple line lists, broadening the lines, and fitting all the lines to the data, returning column densities and temperatures vs. depth. In other words, will it calibrate the spectrum or reproduce it with a model? --jh-- From thoger.emil at gmail.com Tue Jun 18 19:26:57 2013 From: thoger.emil at gmail.com (=?windows-1252?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Wed, 19 Jun 2013 01:26:57 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: <51C0ECC1.5030307@gmail.com> On 18-06-2013 22:53, Joe Harrington wrote: > Just to interject a few quick thoughts into this discussion... > > Knowing little of the situation on the ground WRT spectral line fitting, > it does seem to me that it's been done a million times. If any of those > implementations is decent, OSS, and compiled, why not just wrap it? It's been done a million times, but I don't know of any one solution that people are generally happy with. Sherpa is maybe the best shot, but it comes either as part of the enormous CXC software package, or in a good but poorly supported stand-alone version with long-standing installation bugs etc. The GUI is limited to the interactive capabilities of Matplotlib, the procedural CLI interface is very easy to use from a console but not well fit for scripting/programming, and the OO interface is not too well documented and seemed generally obscure to me. I've personally used it, but given the problems in connection to installing new versions or on new computers, I have finally gone over to LMfit. I have to do more myself, but I know that it works. IRAF/PyRAF has finally come in a manageable package, but by IRAF standards, "manageable" means that it comes embedded in its own Python distribution; and Iraf/pyraf is not exactly pythonic in its usage and interface Another problem is that when you "just" wrap compiled routines in e.g. C or Fortran, compiler and library dependency issues usually show up in great numbers. The hard number crunching can sometimes be wrappped - and has been, in e.g. mpmath - but anything more than that often relies on archaic libraries hard-coded to need old versions not in the standard package manager of the major OSs, etc. I think the right way ahead is to build it on well-tried and well-supported, actively developed packages and libraries, that should protect against the old dependency walls and endless install failures. > Probably the GUI would want to be in Python, but there's a lot of other > stuff under the hood that would be time-consuming and likely slower to > re-(re-re-re-)write. At the very least, you might save some time by > studying what existing codes do and what people like about them. I agree as far as if something reliable already exists, by all means, use it. But "sometimes frugality cheats wisdom", as the Swedish saying goes: puttig in extra effort now can save a lot of work later. As far as I can see, what makes sense to reuse is number crunching code. On top of this comes another layer of infrastructure, and yet another layer of (G)UI. The infrastructure is the model classes that keep track of free, tied and bound parameters, initial and best-fit values etc., and the way they keep track of each other. This is where I think an extra effort could really pay off in the long run, this is the hard part and if Done Right?, it can really make a difference. This means that the classes for this should be either already existing astropy classes or inherit heavily from them, and in general, the astropy framework should be used with as little modification as possible. > Jon Slavin> - robust fitting routines that return error bars on fitted parameters > > Regarding the reporting of fits with errors, as many of us have > painfully learned, minimizers don't give these, they only pretend to. > Since that's often not good enough, it would be nice to see a relatively > plug-n-play MCMC (e.g., DE-) put forward as a fitting package. It would > have to evaluate its own distribution, gave errors, and otherwise > behaved a bit like lmfit. Yes, there are subtleties to Markov Chains, > but this is also true of minimizers. Getting something out there and > inviting people to improve it could produce something usable in a few > iterations. My group can contribute some DE-MCMC code that someone > could adapt. Sounds absolutely great. Sherpa has similar routines and they are really strong work horses, so possibly, some of their code could be studied too. > To Jon's list of requirements, I'd add: > > - Able to use a GUI for user-cue input OR take such input from a text > file > - Able to write such a text file from the GUI user-cue input, for > subsequent runs (and add to it from a second run, etc.) I personally think that the ideal GUI at any (or most) moment can take both command line inputs and point-and-click inputs, as I have tried to do in the GUI I wrote about earlier. And when it can be given commands during runtime, of course it can also be scripted. > One thing that isn't clear to me from the discussion is whether the > scope is merely to identify the center and width vs. pixel number to get > a wavelength solution or calculate Doppler shifts, or whether to do the > whole job of reading multiple line lists, broadening the lines, and > fitting all the lines to the data, returning column densities and > temperatures vs. depth. In other words, will it calibrate the spectrum > or reproduce it with a model? I don't think that there is a general consensus on this, but in my view, the question must be which tasks make sense to do from the command line and which ones will be made significantly easier by means of a GUI, and implementing them in that order. I always shun from software that makes me set everything through point-and-click, file choosing dialogs etc., when it could be done easier by CLI. Reading a line list from a file should be a no-brainer from the command line, and possibly, a good fitting package would provide a few convenience functions for loading data and line lists into the appropriate data structures. I think that the package should start with the simpler models first, and then move on to the more complicated stuff later. Which is another reason why I think that a well designed infrastructure layer is very important: once it has been made properly modular, it is easy to write more different modules and plug them in. In that way, if someone comes up with a smart way to implement a very sophisticated model, we can plug it in to the existing infrastructure and give it as an option. One example where this is relevant is in ISM absorption lines: does one just want to fit them to quick but not too accurate gaussians or lorentzians, or does one want to fit them to a "geometrically" (width, depth, etc) defined Voigt profile, or to a full-fledged voigt profile in terms of N and b and z? Proper modularity could mean that the latter could be relatively simply plugged in to existing infrastructure once this is built (I have made a half,and not yet succesful, attempt at this myself). Something I have implemented in my half-cooked GUI so far is: - Load FITS or ascii file into a 2d-spectrum class (as CLI function) - extract any continuous group of rows into properly weighted 1D spectrum (interactive, can easily get a CLI convenience function). - interactively model line profiles of up to 10 components each, and immediately fit them to data using lmfit as backend (have a half finished sherpa backend too). - Saves the model for all transitions for all rows of potentially several spectra in one big, flat, human-readable ascii file which during runtime is handed as a pandas DataFrame, which of course also means that all pandas and numpy operations are available for it, and the graphical representation gets updated accordingly. - Assign one-letter labels (represeted as colors in the plot) to each peak. Can be done both CLI and GUI way. - Set/edit wavelength range(s) for inclusion in fit and which ones to ignore, CLI and GUI. Things on my current wish list: - Proper continuum modeling (only takes an additive constant right now). - instrument convolution of model - more different line profiles (lorentz, voigt, etc.). The tricky part is the GUI. - more convenience functions and CLI options. - seamlessly interchangeable velocity and wavelength representation on the diffraction axis - more sophisticated parameter handling (freeze/thaw, tie etc.) On the other hand, I think that things like e.g. stacking, flux calibration, and other clearly calibration phase tasks are better left to other kinds of software. Cheers, Emil > --jh-- > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From blanc at apc.univ-paris7.fr Wed Jun 19 11:50:36 2013 From: blanc at apc.univ-paris7.fr (Guillaume Blanc) Date: Wed, 19 Jun 2013 17:50:36 +0200 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? Message-ID: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> Hi, I'm looking in a python library to plot a mollweide projection of the celestial sphere with some points and lines between two points... I tried using healpy, but the projplot function doesn't work well, and the plotting options look limited (well, it's not the main goal of that library, actually). To try to be clearer, I'd like to plot some objects in a projection of the celestial sphere in galactic coordinates with the standard mollweide projection, and plot as well some "lines" such as the celestial equator and so on... I'm sure something exist somewhere to do that! Many thanks in advance, Guillaume Blanc From marquett at iap.fr Wed Jun 19 11:55:19 2013 From: marquett at iap.fr (Jean-Baptiste Marquette) Date: Wed, 19 Jun 2013 17:55:19 +0200 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? In-Reply-To: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> References: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> Message-ID: <29AF5804-4C95-4953-A700-A5222B75283D@iap.fr> Le 19 juin 2013 ? 17:50, Guillaume Blanc a ?crit : > I'm looking in a python library to plot a mollweide projection of the celestial sphere with some points and lines between two points... I tried using healpy, but the projplot function doesn't work well, and the plotting options look limited (well, it's not the main goal of that library, actually). APLpy? http://aplpy.github.io/ From jakevdp at cs.washington.edu Wed Jun 19 12:04:41 2013 From: jakevdp at cs.washington.edu (Jacob Vanderplas) Date: Wed, 19 Jun 2013 09:04:41 -0700 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord)? Message-ID: Here's a quick example of plotting various spherical projections using matplotlib http://www.astroml.org/book_figures/chapter1/fig_projections.html Here's an example of a script which uses matplotlib to scatter-plot SDSS data using a mollweide projection: https://github.com/astroML/astroML/blob/master/book_figures/chapter1/fig_NASA_atlas.py Hope that helps, Jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From mperrin at stsci.edu Wed Jun 19 12:21:29 2013 From: mperrin at stsci.edu (Marshall Perrin) Date: Wed, 19 Jun 2013 16:21:29 +0000 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? In-Reply-To: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> References: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> Message-ID: On Jun 19, 2013, at 11:50 AM, Guillaume Blanc wrote: > I'm looking in a python library to plot a mollweide projection of the celestial sphere with some points and lines between two points... I tried using healpy, but the projplot function doesn't work well, and the plotting options look limited (well, it's not the main goal of that library, actually). > > To try to be clearer, I'd like to plot some objects in a projection of the celestial sphere in galactic coordinates with the standard mollweide projection, and plot as well some "lines" such as the celestial equator and so on... > > I'm sure something exist somewhere to do that! This can actually all be done in plain matplotlib, just by setting the 'projection' keyword when setting up the plot axes. See http://matplotlib.org/examples/pylab_examples/geo_demo.html - Marshall From kellecruz at gmail.com Wed Jun 19 13:10:50 2013 From: kellecruz at gmail.com (Kelle Cruz) Date: Wed, 19 Jun 2013 13:10:50 -0400 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? In-Reply-To: References: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> Message-ID: this is also included in one of the tutorials Adrian and I worked on: https://github.com/astropy/astropy-tutorials/blob/master/tutorials/Plot-Catalog/plot-catalog.py On Wed, Jun 19, 2013 at 12:21 PM, Marshall Perrin wrote: > On Jun 19, 2013, at 11:50 AM, Guillaume Blanc wrote: > > I'm looking in a python library to plot a mollweide projection of the > celestial sphere with some points and lines between two points... I tried > using healpy, but the projplot function doesn't work well, and the plotting > options look limited (well, it's not the main goal of that library, > actually). > > > > To try to be clearer, I'd like to plot some objects in a projection of > the celestial sphere in galactic coordinates with the standard mollweide > projection, and plot as well some "lines" such as the celestial equator and > so on... > > > > I'm sure something exist somewhere to do that! > > > This can actually all be done in plain matplotlib, just by setting the > 'projection' keyword when setting up the plot axes. > > See http://matplotlib.org/examples/pylab_examples/geo_demo.html > > > - Marshall > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rahman at astro.utoronto.ca Wed Jun 19 13:24:14 2013 From: rahman at astro.utoronto.ca (Mubdi Rahman) Date: Wed, 19 Jun 2013 18:24:14 +0100 (BST) Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? In-Reply-To: References: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> Message-ID: <1371662654.72022.YahooMailNeo@web28706.mail.ir2.yahoo.com> The built-in matplotlib way is fairly easy (using the "basemap" package):?http://matplotlib.org/basemap/users/moll.html One note that I'd make is that when you define your basemap, make sure you use the flag "celestial=True" to get coordinates going in the right direction for astronomy. >________________________________ > From: Kelle Cruz >To: Marshall Perrin >Cc: "" >Sent: Wednesday, June 19, 2013 1:10:50 PM >Subject: Re: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? > > > >this is also included in one of the tutorials Adrian and I worked on: >https://github.com/astropy/astropy-tutorials/blob/master/tutorials/Plot-Catalog/plot-catalog.py > > > > >On Wed, Jun 19, 2013 at 12:21 PM, Marshall Perrin wrote: > >On Jun 19, 2013, at 11:50 AM, Guillaume Blanc wrote: >>> I'm looking in a python library to plot a mollweide projection of the celestial sphere with some points and lines between two points... I tried using healpy, but the projplot function doesn't work well, and the plotting options look limited (well, it's not the main goal of that library, actually). >>> >>> To try to be clearer, I'd like to plot some objects in a projection of the celestial sphere in galactic coordinates with the standard mollweide projection, and plot as well some "lines" such as the celestial equator and so on... >>> >>> I'm sure something exist somewhere to do that! >> >> >>This can actually all be done in plain matplotlib, just by setting the 'projection' keyword when setting up the plot axes. >> >>See http://matplotlib.org/examples/pylab_examples/geo_demo.html >> >> >>?- Marshall >> >>_______________________________________________ >>AstroPy mailing list >>AstroPy at scipy.org >>http://mail.scipy.org/mailman/listinfo/astropy >> > > > >-- >Kelle Cruz, PhD ??http://kellecruz.com/ >917.725.1334 ??Hunter: x16486 ? AMNH: x3404 >_______________________________________________ >AstroPy mailing list >AstroPy at scipy.org >http://mail.scipy.org/mailman/listinfo/astropy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From keflavich at gmail.com Wed Jun 19 16:32:31 2013 From: keflavich at gmail.com (Adam Ginsburg) Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: <51C0AF3B.9040604@gmail.com> References: <51C0AF3B.9040604@gmail.com> Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5@googlegroups.com> Emil and Jon - there is a package that fits exactly those requirements, called pyspeckit: pyspeckit.bitbucket.org One of my major goals over the next year is to incorporate it into astropy bit by bit, i.e. make it work with astropy.models, astropy.units, and specutils. Only a little progress in that direction has been made (in part because my coding efforts have been focused more on astroquery). Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. You can check that it agrees with yours: http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt I'm bringing this up on this particular thread because I had considered submitting pyspeckit as a SOCIS project, but I think focusing on specutils instead will be better for the community. Nonetheless, there are some specific project ideas in place that may help support the case for a student working on specutils via SOCIS: pyspeckit.bitbucket.org/html/sphinx/projects.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From npkuin at gmail.com Wed Jun 19 17:12:20 2013 From: npkuin at gmail.com (Paul Kuin) Date: Wed, 19 Jun 2013 22:12:20 +0100 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: <44536fa7-b34e-4f05-941b-2cc05535bcc5@googlegroups.com> References: <51C0AF3B.9040604@gmail.com> <44536fa7-b34e-4f05-941b-2cc05535bcc5@googlegroups.com> Message-ID: Just a brief comment. At first sight it looks like a good start. However, how do you deal with bad data/ Suppose I flag a part of the spectrum as bad. Is your formalism capable of handling that? It would be nice if there were s spectral library that could be used to fit to. For example a stellar one (Pickles) and a galaxies one (forgot the reference). Similarly, if you have models for a Star+disk, or binary, that would be nice to have to fit to. When measuring lines, it would be nice to have line ratios of (effectively) optically thin lines from the same upper level to work with. That is useful both for seeing if the line identification is good and estimating optical depth effects. But so far, this is the best I've seen pythonic wise. (and that may just be because I don't know much).... Paul Kuin On Wed, Jun 19, 2013 at 9:32 PM, Adam Ginsburg wrote: > Emil and Jon - there is a package that fits exactly those requirements, > called pyspeckit: > pyspeckit.bitbucket.org > > One of my major goals over the next year is to incorporate it into astropy > bit by bit, i.e. make it work with astropy.models, astropy.units, and > specutils. Only a little progress in that direction has been made (in part > because my coding efforts have been focused more on astroquery). > > Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. > You can check that it agrees with yours: > > http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt > > I'm bringing this up on this particular thread because I had considered > submitting pyspeckit as a SOCIS project, but I think focusing on specutils > instead will be better for the community. Nonetheless, there are some > specific project ideas in place that may help support the case for a > student working on specutils via SOCIS: > pyspeckit.bitbucket.org/html/sphinx/projects.html > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) phone +44-(0)1483 (prefix) -204927 (work) -276110 (home) mobile +44(0)7806985366 skype ID: npkuin Mullard Space Science Laboratory ? University College London ? Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eebanado at uc.cl Wed Jun 19 17:26:45 2013 From: eebanado at uc.cl (=?ISO-8859-1?Q?Eduardo_Ba=F1ados_Torres?=) Date: Wed, 19 Jun 2013 23:26:45 +0200 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? In-Reply-To: <1371662654.72022.YahooMailNeo@web28706.mail.ir2.yahoo.com> References: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> <1371662654.72022.YahooMailNeo@web28706.mail.ir2.yahoo.com> Message-ID: Some time ago I asked a related question here and in stackoverflow... I think here is the answer you are looking for (you can do it all just in matplotlib) http://stackoverflow.com/questions/7355497/curious-bad-behavior-creating-all-sky-projections-with-matplotlib/7408280#7408280 Cheers, Eduardo On Wed, Jun 19, 2013 at 7:24 PM, Mubdi Rahman wrote: > The built-in matplotlib way is fairly easy (using the "basemap" package): > http://matplotlib.org/basemap/users/moll.html > > One note that I'd make is that when you define your basemap, make sure you > use the flag "celestial=True" to get coordinates going in the right > direction for astronomy. > > ------------------------------ > *From:* Kelle Cruz > *To:* Marshall Perrin > *Cc:* "" > *Sent:* Wednesday, June 19, 2013 1:10:50 PM > *Subject:* Re: [AstroPy] mollweide projection plotting (celestial sphere > in Galactic coord) ? > > this is also included in one of the tutorials Adrian and I worked on: > > https://github.com/astropy/astropy-tutorials/blob/master/tutorials/Plot-Catalog/plot-catalog.py > > > On Wed, Jun 19, 2013 at 12:21 PM, Marshall Perrin wrote: > > On Jun 19, 2013, at 11:50 AM, Guillaume Blanc wrote: > > I'm looking in a python library to plot a mollweide projection of the > celestial sphere with some points and lines between two points... I tried > using healpy, but the projplot function doesn't work well, and the plotting > options look limited (well, it's not the main goal of that library, > actually). > > > > To try to be clearer, I'd like to plot some objects in a projection of > the celestial sphere in galactic coordinates with the standard mollweide > projection, and plot as well some "lines" such as the celestial equator and > so on... > > > > I'm sure something exist somewhere to do that! > > > This can actually all be done in plain matplotlib, just by setting the > 'projection' keyword when setting up the plot axes. > > See http://matplotlib.org/examples/pylab_examples/geo_demo.html > > > - Marshall > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- Eduardo Ba?ados Torres -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at physics.ucf.edu Thu Jun 20 05:31:37 2013 From: jh at physics.ucf.edu (Joe Harrington) Date: Thu, 20 Jun 2013 11:31:37 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: (astropy-request@scipy.org) Message-ID: It sounds like we're on the same wavelength (sorry) regarding the interface, and yes, from the state you describe writing something to last from the ground up or from some basic workhorse routines sounds like a good idea. >> temperatures vs. depth. In other words, will it calibrate the spectrum >> or reproduce it with a model? > > I don't think that there is a general consensus on this, but in my view, > the question > must be which tasks make sense to do from the command line and which > ones will be made significantly easier > by means of a GUI, and implementing them in that order. I always shun > from software that makes me set everything through > point-and-click, file choosing dialogs etc., when it could be done > easier by CLI. > Reading a line list from a file should be a no-brainer from the command > line, and possibly, a good > fitting package would provide a few convenience functions for loading > data and line lists into the appropriate data structures. So, I think we're misconnecting here. To me a line list has some hundreds of millions of lines, and you'd never put it in a text file. What kind of spectra are you thinking of fitting? Is each feature a single transition, like stellar UV atomic lines, or does each feature integrate huge numbers of transitions, like in NIR planetary/brown dwarf spectra? It sounds like you are separately fitting each feature to a Voigt, rather than broadening and integrating, say, all the methane lines, then scaling the result to the data to derive a methane column abundance. Or are you thinking of both applications? The reason it matters now is that some designs that do the simple situation well might not be easily adapted to the complex one. FWIW, my group is working on a radiative transfer program for exoplanetary emission and absorption, along with a DE-MCMC fitting routine. The basic RT code and MCMC are working and are OSS, though not yet on github. If you do go to the point of fitting molecular spectra to get abundances, the RT code (written in C and depending on just the GSL, I think) might be what you need. The DE-MCMC is Python (it might have some low-level C, I'm not sure, but if so it doesn't depend on anything beyond libc). Let me know if you want these. --jh-- From blanc at apc.univ-paris7.fr Thu Jun 20 07:15:27 2013 From: blanc at apc.univ-paris7.fr (Guillaume Blanc) Date: Thu, 20 Jun 2013 13:15:27 +0200 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord) ? In-Reply-To: References: <339AD428-9816-404D-B362-7525BCC5997E@apc.univ-paris7.fr> Message-ID: <9B3255E6-4F1B-4B26-9A80-2889775C405F@apc.univ-paris7.fr> Hi, Thanks a lot. I was able to do what I wanted using matplotlib. Here is the code, and below the corresponding figure, if someone is interested. Guillaume Blanc --------------- import numpy as np import pylab as pl import math # the spot to plot x = np.array([-37, 88, -137, -139, -136, -44]) # longitude y = np.array([27, -60, -1.4, -50, -77, -46]) # latitude lab = ['TF0.1', 'TF0.2', 'TF0.3', 'TF0.4', 'TF0.5', 'TF0.6' ] # To plot the celestial equator in galactic coordinates degtorad = math.pi/180. alpha = np.arange(-180,180.,1.) alpha *= degtorad # From Meeus, Astronomical algorithms (with delta = 0) x1 = np.sin(192.25*degtorad - alpha) x2 = np.cos(192.25*degtorad - alpha)*np.sin(27.4*degtorad) yy = np.arctan2(x1, x2) longitude = 303*degtorad - yy x3 = np.cos(27.4*degtorad) * np.cos(192.25*degtorad - alpha) latitude = np.arcsin(x3) # We put the angles in the right direction for i in range(0,len(alpha)): if longitude[i] > 2.*math.pi: longitude[i] -= 2.*math.pi longitude[i] -= math.pi latitude[i] = -latitude[i] # To avoid a line in the middle of the plot (the curve must not loop) for i in range(0,len(longitude)-1): if (longitude[i] * longitude[i+1] < 0 and longitude[i] > 170*degtorad and longitude[i+1] < -170.*degtorad): indice = i break # The array is put in increasing longitude longitude2 = np.zeros(len(longitude)) latitude2 = np.zeros(len(latitude)) longitude2[0:len(longitude)-1-indice] = longitude[indice+1:len(longitude)] longitude2[len(longitude)-indice-1:len(longitude)] = longitude[0:indice+1] latitude2[0:len(longitude)-1-indice] = latitude[indice+1:len(longitude)] latitude2[len(longitude)-indice-1:len(longitude)] = latitude[0:indice+1] xrad = x * degtorad yrad = y * degtorad fig2 = pl.figure(2) ax1 = fig2.add_subplot(111, projection="mollweide") ax1.scatter(xrad,yrad) ax1.plot([-math.pi, math.pi], [0,0],'r-') ax1.plot([0,0],[-math.pi, math.pi], 'r-') # plot celestial equator ax1.plot(longitude2,latitude2,'g-') for i in range(0,6): ax1.text(xrad[i], yrad[i], lab[i]) pl.title("Test Fields") pl.grid(True) pl.show() ------------ Le 19 juin 2013 ? 19:10, Kelle Cruz a ?crit : > this is also included in one of the tutorials Adrian and I worked on: > https://github.com/astropy/astropy-tutorials/blob/master/tutorials/Plot-Catalog/plot-catalog.py > > > On Wed, Jun 19, 2013 at 12:21 PM, Marshall Perrin wrote: > On Jun 19, 2013, at 11:50 AM, Guillaume Blanc wrote: > > I'm looking in a python library to plot a mollweide projection of the celestial sphere with some points and lines between two points... I tried using healpy, but the projplot function doesn't work well, and the plotting options look limited (well, it's not the main goal of that library, actually). > > > > To try to be clearer, I'd like to plot some objects in a projection of the celestial sphere in galactic coordinates with the standard mollweide projection, and plot as well some "lines" such as the celestial equator and so on... > > > > I'm sure something exist somewhere to do that! > > > This can actually all be done in plain matplotlib, just by setting the 'projection' keyword when setting up the plot axes. > > See http://matplotlib.org/examples/pylab_examples/geo_demo.html > > > - Marshall > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: figure_2.png Type: image/png Size: 67690 bytes Desc: not available URL: From trive at astro.su.se Thu Jun 20 07:17:46 2013 From: trive at astro.su.se (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Thu, 20 Jun 2013 13:17:46 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: <51C2E4DA.5010600@astro.su.se> On 2013-06-20 11:31, Joe Harrington wrote: > It sounds like we're on the same wavelength (sorry) regarding the > interface, and yes, from the state you describe writing something to > last from the ground up or from some basic workhorse routines sounds > like a good idea. Nice pun ;) As you may have seen from Adam Ginsburg, his PySpecKit actually has something that could provide either a starrting point or some detailed design inspiration (depending on how well it complies with the general astropy design philosophy, about which I have no idea). >>> temperatures vs. depth. In other words, will it calibrate the spectrum >>> or reproduce it with a model? >> I don't think that there is a general consensus on this, but in my view, >> the question >> must be which tasks make sense to do from the command line and which >> ones will be made significantly easier >> by means of a GUI, and implementing them in that order. I always shun >> from software that makes me set everything through >> point-and-click, file choosing dialogs etc., when it could be done >> easier by CLI. >> Reading a line list from a file should be a no-brainer from the command >> line, and possibly, a good >> fitting package would provide a few convenience functions for loading >> data and line lists into the appropriate data structures. > So, I think we're misconnecting here. To me a line list has some > hundreds of millions of lines, and you'd never put it in a text file. > What kind of spectra are you thinking of fitting? Is each feature a > single transition, like stellar UV atomic lines, or does each feature > integrate huge numbers of transitions, like in NIR planetary/brown dwarf > spectra? It sounds like you are separately fitting each feature to a > Voigt, rather than broadening and integrating, say, all the methane > lines, then scaling the result to the data to derive a methane column > abundance. Or are you thinking of both applications? The reason it > matters now is that some designs that do the simple situation well might > not be easily adapted to the complex one. Yes, seems like very different kinds of tasks we are talking about here - I'm doing some relatively simple galactic stellar and nebular emission and absorption lines, which usually are not more than maybe a couple of atomic transitions blended. They can show some quite complicated dynamics and other effects, though. But you are right, it looks like these tasks really call for different approaches. Again, I think the wise thing to do is to saplit up the workflow in smaller tasks and evaluate one for one what would be gained by GUI-ifying it, vs. what would be the cost in gained complexity of the interface and in sheer working hours for coding. > FWIW, my group is working on a radiative transfer program for > exoplanetary emission and absorption, along with a DE-MCMC fitting > routine. The basic RT code and MCMC are working and are OSS, though not > yet on github. If you do go to the point of fitting molecular spectra > to get abundances, the RT code (written in C and depending on just the > GSL, I think) might be what you need. The DE-MCMC is Python (it might > have some low-level C, I'm not sure, but if so it doesn't depend on > anything beyond libc). Let me know if you want these. Did you mean the specutils project here, or me personally? My work in the foreseeable future is probably not going to involve any detailed molecular fitting; we are mainly focusing on features that could at least in priciple be detected and distinguiished at high redshifts, too. > > --jh-- > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From jslavin at cfa.harvard.edu Thu Jun 20 08:42:54 2013 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Thu, 20 Jun 2013 08:42:54 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 Message-ID: Hi Adam, I have used pyspeckit. There are parts of it that are very useful, but I found overall that it didn't do what I wanted. For example, I wanted to give it my spectrum, have it display it, allow me to manually (in the gui) place my guesses for line centers (would probably need guesses for widths as well) and then fit the lines (and continuum) and return the fitted line centers, widths, and amplitudes, etc. along with error estimates. It's been a while now since I used it, but as I recall, pyspeckit did not do all of those things. There may have been other reasons as well, but in the end I wrote my own fitting routines to work with the data. Am I wrong about the capabilities of pyspeckit? Jon ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ On Wed, Jun 19, 2013 at 5:21 PM, wrote: > > Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) > From: Adam Ginsburg > Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > To: astropy-dev at googlegroups.com > Cc: astropy at scipy.org > Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5 at googlegroups.com> > Content-Type: text/plain; charset="utf-8" > > Emil and Jon - there is a package that fits exactly those requirements, > called pyspeckit: > pyspeckit.bitbucket.org > > One of my major goals over the next year is to incorporate it into astropy > bit by bit, i.e. make it work with astropy.models, astropy.units, and > specutils. Only a little progress in that direction has been made (in part > because my coding efforts have been focused more on astroquery). > > Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. > You can check that it agrees with yours: > > http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt > > I'm bringing this up on this particular thread because I had considered > submitting pyspeckit as a SOCIS project, but I think focusing on specutils > instead will be better for the community. Nonetheless, there are some > specific project ideas in place that may help support the case for a > student working on specutils via SOCIS: > pyspeckit.bitbucket.org/html/sphinx/projects.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jslavin at cfa.harvard.edu Thu Jun 20 09:17:17 2013 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Thu, 20 Jun 2013 09:17:17 -0400 Subject: [AstroPy] mollweide projection plotting (celestial sphere in Galactic coord)? Message-ID: Hi Guillaume, This code makes a nice plot, though the longitude is backward as compared with a normal sky map (i.e. longitude generally runs right-to-left). A couple little things you might want to know. Numpy has a pi value, so you don't need to import math -- just use numpy.pi. Are you familiar with this trick:for reversing arrays: y = y[::-1]? Also instead of x = np.zeros(len(y)) you can do x = np.zeros_like(y). Just thought I'd pass these along. Jon On Thu, Jun 20, 2013 at 7:10 AM, wrote: > Date: Thu, 20 Jun 2013 13:15:27 +0200 > From: Guillaume Blanc > Subject: Re: [AstroPy] mollweide projection plotting (celestial sphere > in Galactic coord) ? > To: Kelle Cruz > Cc: "" > Message-ID: <9B3255E6-4F1B-4B26-9A80-2889775C405F at apc.univ-paris7.fr> > Content-Type: text/plain; charset="windows-1252" > > Hi, > > Thanks a lot. I was able to do what I wanted using matplotlib. Here is the > code, and below the corresponding figure, if someone is interested. > > Guillaume Blanc > ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Thu Jun 20 09:28:44 2013 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Thu, 20 Jun 2013 09:28:44 -0400 Subject: [AstroPy] Doodle poll for Astropy Coordination Meeting Message-ID: Hello all, As you may or may not know, there have been coordination meetings the last two years (at the CfA and STScI) to discuss progress and future plans for the Astropy project. These meetings were crucial in helping organize the initial stages and direction for the project. Since then, Astropy has grown significantly and there are a number of quite different subpackages, each with somewhat different items to address and consider. This has prompted discussion in the coordinating committee that we might consider holding in-person coordination meetings less often (every 2nd year perhaps?). Instead, we might start scheduling regular google hangouts to deal with items that might otherwise have been targeted for the coordination meeting. Or, we might continue the yearly schedule, but focus the coordination meetings on coding sprints, rather than planning discussions. E.g., a day or half-day of progress overviews, followed by 1 or 2 days of sprinting to focus on specific issues that are better addressed in-person. (Perhaps we can even dabble in pair coding.) So to help in making such a decision, I have created a doodle poll at http://www.doodle.com/t9em8iaqm7f6hrx7 This poll contains a number of potential dates this fall for the coordination meeting. The Yale Astronomy department is willing to host such an event for this year (host=provide a space and pay for coffee and lunch), so it will most likely be in New Haven, CT, USA. *If* you would be interested in attending such a meeting this year, please add your name to the poll and indicate which dates you are available. Please aim to respond by July 4 (2 weeks). If at that point enough people express interest that it seems worthwhile to hold a meeting, we will schedule a time based on the responses. If not, we will skip the meeting this year, and instead hold one in the fall of 2014. (And if anyone has ideas for other approaches to coordination meetings, of course feel free to respond to this post.) Thanks! -- Erik T -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Thu Jun 20 12:27:27 2013 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Thu, 20 Jun 2013 12:27:27 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: I want to issue a generic plea on this issue: lets do our best to join forces. There are already a number of tools that do part of spectrum plotting and line-fitting, and if we'd all work together, with less effort we'd have a tool that does all of those and more. That's the main point of Astropy, after all: encouraging collobaration on and re-use of python tools for astronomy. I'm partly guilty here, as I have a similar fitting tool in astropysics - http://pythonhosted.org/Astropysics/gui/spylot.html, but I'd rather encourage people to use a tool that we can all get behind. I'll call this the "Astropy spirit" :) That said, I see the virtue of designing a new tool (ideally an astropy affiliated package) to combine the shared wisdom (perhaps with SOCIS, if it goes through). But the *worst* thing to have happen is to have the other efforts continue in parallel, with everyone designing similar but slightly incompatible tools. Along similar lines, Emil and Adam (although I think Adam already knows about it), I'd *strongly* encourage you to take a look at the astropy modeling framework (http://docs.astropy.org/en/latest/modeling/index.html) - this will be in the next major release, and I'd hate to see you re-implementing all of that in an incompatible way. Or, if you don't like something about how that works for your needs, please help us make it better! On Thu, Jun 20, 2013 at 8:42 AM, Slavin, Jonathan wrote: > Hi Adam, > > I have used pyspeckit. There are parts of it that are very useful, but I > found overall that it didn't do what I wanted. For example, I wanted to > give it my spectrum, have it display it, allow me to manually (in the gui) > place my guesses for line centers (would probably need guesses for widths as > well) and then fit the lines (and continuum) and return the fitted line > centers, widths, and amplitudes, etc. along with error estimates. It's been > a while now since I used it, but as I recall, pyspeckit did not do all of > those things. There may have been other reasons as well, but in the end I > wrote my own fitting routines to work with the data. Am I wrong about the > capabilities of pyspeckit? > > Jon > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > On Wed, Jun 19, 2013 at 5:21 PM, wrote: >> >> Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) >> From: Adam Ginsburg >> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy-dev at googlegroups.com >> Cc: astropy at scipy.org >> Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5 at googlegroups.com> >> Content-Type: text/plain; charset="utf-8" >> >> >> Emil and Jon - there is a package that fits exactly those requirements, >> called pyspeckit: >> pyspeckit.bitbucket.org >> >> One of my major goals over the next year is to incorporate it into astropy >> bit by bit, i.e. make it work with astropy.models, astropy.units, and >> specutils. Only a little progress in that direction has been made (in >> part >> because my coding efforts have been focused more on astroquery). >> >> Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. >> You can check that it agrees with yours: >> >> http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt >> >> I'm bringing this up on this particular thread because I had considered >> submitting pyspeckit as a SOCIS project, but I think focusing on specutils >> instead will be better for the community. Nonetheless, there are some >> specific project ideas in place that may help support the case for a >> student working on specutils via SOCIS: >> pyspeckit.bitbucket.org/html/sphinx/projects.html >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Erik From erik.tollerud at gmail.com Thu Jun 20 12:34:26 2013 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Thu, 20 Jun 2013 12:34:26 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <51C0B69A.6020402@astro.su.se> References: <51C0B69A.6020402@astro.su.se> Message-ID: I'm of mixed minds about traits UI because once you know it you can make great GUIs with it, but I've spent a lot of time troubleshooting people's python installations to get traits to work. That is, in general it can be tricky to get installed because of all the dependencies. Maybe this has improved recently with Enthought's Canopy (or other new python distros), but that's been my past experience. More generally, the view in the astropy core package is that we don't want to put GUIs in the core because GUIs always carry lots of dependencies, which we don't want to be forced to deal with. But part of the whole reason for affiliated packages was to get around this, so we're happy to see GUI-based affiliated packages. As for Pandas, to be totally honest, I don't see a huge amount to be gained from adding a Pandas dependency Astropy. It's honestly not clear what it gives the astronomy community that numpy does not already have. The following quote from the Pandas web site has guided me to that conclusion: " *pandas* helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R." I have been carrying out my entire data analysis workflow for some time now in python without using Pandas. It looks to me like Pandas is a tool that was written by and for statisticians who use R. While we can take lessons from this, it's not clear we get much out of it in an astronomy context. For example, how would it make astropy's NDData, Quantity, or Table better to use a Pandas DataFrame vs. a numpy array? Most of what we are doing is building astronomy-convenient interfaces, and I'm not sure what Pandas adds there, at the cost of a pretty heavy-weight dependency. It could just be that I don't know enough about Pandas, though. So if someone who knows Pandas better can speak to this, I'm all ears. On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen wrote: > Pandas is a part of the newly-defined SciPy stack, after all, so that > would be part of any science-oriented distribution worth its salt. In fact, > I think it could be a good idea for astropy in general to use under the > hood, but again, could clash with the philosophy of the project and > possibly also maintainabillity. > > As for offering my code or just my experience, I'll have to square it with > my supervisor first, and I also think it depends on what direction the > project in question will take. I'm positive about the idea (which is why I > wrote in the first place), but supervisor might think it is a better idea > to actually get my paper in the project wrapped up before sending the code > out there. Will get back about that one! > > /Emil > > > > > > On 2013-06-18 20:53, Slavin, Jonathan wrote: > > Hi Emil, > > That looks very nice! I don't see Pandas as a big issue in terms of > dependencies. I don't know that much about traits, etc. My thought about > the gui was just based on my experience with matplotlib, and the fact that > it is widely used -- though I would agree that too many dependencies can be > a deterrent to people using something. Are you offering your code as a > starting point for the project? It strikes me that many have gotten some > sort of fitting package to a point of personal usability but no one has the > time/interest/motivation to make a more generally usable package. > > Jon > > On Tue, Jun 18, 2013 at 2:34 PM, wrote: > >> Date: Tue, 18 Jun 2013 20:39:55 +0200 >> From: Th?ger Rivera-Thorsen >> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy at scipy.org >> Message-ID: <51C0A97B.8090703 at gmail.com> >> Content-Type: text/plain; charset="iso-8859-1" >> >> I have been working on a fitting GUI for a while, although it is made >> with a specific task in mind. >> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and >> Pandas. It is made for a specific projhect I'm working and as such not >> yet usable for more general cases, but it could be a starting point, if >> the dependencies don't conflict with astropy politics. >> >> Especially, I am happy about the choice of Pandas for managing a quite >> complex data structure (the fitted and/or guessed values of an arbitrary >> number of transitions for an arbitrary number of rows or collapsed rows >> of a spatially resolved spectrum) of a), but also with the Traits-based >> interactive interface to build complex line profiles from single >> gaussians, good for fitting-by-eye and giving good initial guesses for >> fitting of complex line profiles. It hooks directly up to a wrapper I've >> made for lmfit, but given the modularity, it should be relatively easy >> to change to other backends. >> >> It's still a work-in-progress, but there are some screenshots here: >> http://flic.kr/s/aHsjGaEMGg . >> I know the choice and number of dependencies may be prohibitive but it >> saved a lot of work on the GUI, and Pandas means the difference between >> sanity and madness when it comes to keeping track of so many parameters. >> >> Cheers, >> Emil >> > > > > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > > _______________________________________________ > AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- Erik -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at depagne.org Thu Jun 20 12:41:56 2013 From: eric at depagne.org (=?iso-8859-1?q?=C9ric_Depagne?=) Date: Thu, 20 Jun 2013 18:41:56 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: <201306201841.56490.eric@depagne.org> Hi all, > Along similar lines, Emil and Adam (although I think Adam already > knows about it), I'd *strongly* encourage you to take a look at the > astropy modeling framework > (http://docs.astropy.org/en/latest/modeling/index.html) - this will be > in the next major release, and I'd hate to see you re-implementing all > of that in an incompatible way. Or, if you don't like something about > how that works for your needs, please help us make it better! And for me, whose talent in coding won't be enough to be of real help in the coding, I'd be happy to help otherwise : providing things I'd like to see, features that may help (that's the obvious and easy part), but aslo, I'd be happy to help test and debug. ?ric. > > > On Thu, Jun 20, 2013 at 8:42 AM, Slavin, Jonathan > > wrote: > > Hi Adam, > > > > I have used pyspeckit. There are parts of it that are very useful, but I > > found overall that it didn't do what I wanted. For example, I wanted to > > give it my spectrum, have it display it, allow me to manually (in the > > gui) place my guesses for line centers (would probably need guesses for > > widths as well) and then fit the lines (and continuum) and return the > > fitted line centers, widths, and amplitudes, etc. along with error > > estimates. It's been a while now since I used it, but as I recall, > > pyspeckit did not do all of those things. There may have been other > > reasons as well, but in the end I wrote my own fitting routines to work > > with the data. Am I wrong about the capabilities of pyspeckit? > > > > Jon > > ________________________________________________________ > > Jonathan D. Slavin Harvard-Smithsonian CfA > > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > > phone: (617) 496-7981 Cambridge, MA 02138-1516 > > fax: (617) 496-7577 USA > > ________________________________________________________ > > > > On Wed, Jun 19, 2013 at 5:21 PM, wrote: > >> Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) > >> From: Adam Ginsburg > >> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > >> To: astropy-dev at googlegroups.com > >> Cc: astropy at scipy.org > >> Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5 at googlegroups.com> > >> Content-Type: text/plain; charset="utf-8" > >> > >> > >> Emil and Jon - there is a package that fits exactly those requirements, > >> called pyspeckit: > >> pyspeckit.bitbucket.org > >> > >> One of my major goals over the next year is to incorporate it into > >> astropy bit by bit, i.e. make it work with astropy.models, > >> astropy.units, and specutils. Only a little progress in that direction > >> has been made (in part > >> because my coding efforts have been focused more on astroquery). > >> > >> Emil - there is a scipy-dependent voigt profile implemented in > >> pyspeckit. > >> > >> You can check that it agrees with yours: > >> http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectru > >> m.models.inherited_voigtfitter.voigt > >> > >> I'm bringing this up on this particular thread because I had considered > >> submitting pyspeckit as a SOCIS project, but I think focusing on > >> specutils instead will be better for the community. Nonetheless, there > >> are some specific project ideas in place that may help support the case > >> for a student working on specutils via SOCIS: > >> pyspeckit.bitbucket.org/html/sphinx/projects.html > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy Un clavier azerty en vaut deux ---------------------------------------------------------- ?ric Depagne eric at depagne.org From beaumont at hawaii.edu Thu Jun 20 12:50:52 2013 From: beaumont at hawaii.edu (Chris Beaumont) Date: Thu, 20 Jun 2013 12:50:52 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: I thought I'd chime in on the pandas discussion :) I'm starting to use pandas a bit more in my day-to-day work. The two features most useful to me are: 1) Its file parsers are pretty robust and fast. I always try parsing CSV with pandas first 2) For tables tables with lots of categorical data, the grouping functionality is very nice. For example, calculations like "the mean age of each spectral type of star in my catalog" are usually one liners like df.groupby(['spectral_type']).age.mean. I spend a lot of time on the "split-apply-combine" page on the pandas docs ( http://pandas.pydata.org/pandas-docs/stable/groupby.html). I won't speculate about whether that's enough an asset to warrant a dependency in astropy. I do agree that lots of other pandas features don't translate as well into astronomy use. On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud wrote: > I'm of mixed minds about traits UI because once you know it you can make > great GUIs with it, but I've spent a lot of time troubleshooting people's > python installations to get traits to work. That is, in general it can be > tricky to get installed because of all the dependencies. Maybe this has > improved recently with Enthought's Canopy (or other new python distros), > but that's been my past experience. > > More generally, the view in the astropy core package is that we don't want > to put GUIs in the core because GUIs always carry lots of dependencies, > which we don't want to be forced to deal with. But part of the whole > reason for affiliated packages was to get around this, so we're happy to > see GUI-based affiliated packages. > > > As for Pandas, to be totally honest, I don't see a huge amount to be > gained from adding a Pandas dependency Astropy. It's honestly not clear > what it gives the astronomy community that numpy does not already have. > The following quote from the Pandas web site has guided me to that > conclusion: "*pandas* helps fill this gap, enabling you to carry out your > entire data analysis workflow in Python without having to switch to a more > domain specific language like R." > > I have been carrying out my entire data analysis workflow for some time > now in python without using Pandas. It looks to me like Pandas is a tool > that was written by and for statisticians who use R. While we can take > lessons from this, it's not clear we get much out of it in an astronomy > context. For example, how would it make astropy's NDData, Quantity, or > Table better to use a Pandas DataFrame vs. a numpy array? Most of what we > are doing is building astronomy-convenient interfaces, and I'm not sure > what Pandas adds there, at the cost of a pretty heavy-weight dependency. > > It could just be that I don't know enough about Pandas, though. So if > someone who knows Pandas better can speak to this, I'm all ears. > > > > > On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen wrote: > >> Pandas is a part of the newly-defined SciPy stack, after all, so that >> would be part of any science-oriented distribution worth its salt. In fact, >> I think it could be a good idea for astropy in general to use under the >> hood, but again, could clash with the philosophy of the project and >> possibly also maintainabillity. >> >> As for offering my code or just my experience, I'll have to square it >> with my supervisor first, and I also think it depends on what direction the >> project in question will take. I'm positive about the idea (which is why I >> wrote in the first place), but supervisor might think it is a better idea >> to actually get my paper in the project wrapped up before sending the code >> out there. Will get back about that one! >> >> /Emil >> >> >> >> >> >> On 2013-06-18 20:53, Slavin, Jonathan wrote: >> >> Hi Emil, >> >> That looks very nice! I don't see Pandas as a big issue in terms of >> dependencies. I don't know that much about traits, etc. My thought about >> the gui was just based on my experience with matplotlib, and the fact that >> it is widely used -- though I would agree that too many dependencies can be >> a deterrent to people using something. Are you offering your code as a >> starting point for the project? It strikes me that many have gotten some >> sort of fitting package to a point of personal usability but no one has the >> time/interest/motivation to make a more generally usable package. >> >> Jon >> >> On Tue, Jun 18, 2013 at 2:34 PM, wrote: >> >>> Date: Tue, 18 Jun 2013 20:39:55 +0200 >>> From: Th?ger Rivera-Thorsen >>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>> To: astropy at scipy.org >>> Message-ID: <51C0A97B.8090703 at gmail.com> >>> Content-Type: text/plain; charset="iso-8859-1" >>> >>> I have been working on a fitting GUI for a while, although it is made >>> with a specific task in mind. >>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and >>> Pandas. It is made for a specific projhect I'm working and as such not >>> yet usable for more general cases, but it could be a starting point, if >>> the dependencies don't conflict with astropy politics. >>> >>> Especially, I am happy about the choice of Pandas for managing a quite >>> complex data structure (the fitted and/or guessed values of an arbitrary >>> number of transitions for an arbitrary number of rows or collapsed rows >>> of a spatially resolved spectrum) of a), but also with the Traits-based >>> interactive interface to build complex line profiles from single >>> gaussians, good for fitting-by-eye and giving good initial guesses for >>> fitting of complex line profiles. It hooks directly up to a wrapper I've >>> made for lmfit, but given the modularity, it should be relatively easy >>> to change to other backends. >>> >>> It's still a work-in-progress, but there are some screenshots here: >>> http://flic.kr/s/aHsjGaEMGg . >>> I know the choice and number of dependencies may be prohibitive but it >>> saved a lot of work on the GUI, and Pandas means the difference between >>> sanity and madness when it comes to keeping track of so many parameters. >>> >>> Cheers, >>> Emil >>> >> >> >> >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >> phone: (617) 496-7981 Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> _______________________________________________ >> AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > > -- > Erik > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- ************************************ Chris Beaumont Graduate Student Institute for Astronomy University of Hawaii at Manoa 2680 Woodlawn Drive Honolulu, HI 96822 www.ifa.hawaii.edu/~beaumont ************************************ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Thu Jun 20 13:17:20 2013 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Thu, 20 Jun 2013 13:17:20 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: Hmm, ok, those seem like use-cases that overlap a lot with astropy.table and astropy.io.ascii. Do you find pandas is noticeably faster than astropy.tables? On Thu, Jun 20, 2013 at 12:50 PM, Chris Beaumont wrote: > I thought I'd chime in on the pandas discussion :) > > I'm starting to use pandas a bit more in my day-to-day work. The two > features most useful to me are: > > 1) Its file parsers are pretty robust and fast. I always try parsing CSV > with pandas first > > 2) For tables tables with lots of categorical data, the grouping > functionality is very nice. For example, calculations like "the mean age of > each spectral type of star in my catalog" are usually one liners like > df.groupby(['spectral_type']).age.mean. I spend a lot of time on the > "split-apply-combine" page on the pandas docs > (http://pandas.pydata.org/pandas-docs/stable/groupby.html). > > I won't speculate about whether that's enough an asset to warrant a > dependency in astropy. I do agree that lots of other pandas features don't > translate as well into astronomy use. > > > > On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud > wrote: >> >> I'm of mixed minds about traits UI because once you know it you can make >> great GUIs with it, but I've spent a lot of time troubleshooting people's >> python installations to get traits to work. That is, in general it can be >> tricky to get installed because of all the dependencies. Maybe this has >> improved recently with Enthought's Canopy (or other new python distros), but >> that's been my past experience. >> >> More generally, the view in the astropy core package is that we don't want >> to put GUIs in the core because GUIs always carry lots of dependencies, >> which we don't want to be forced to deal with. But part of the whole reason >> for affiliated packages was to get around this, so we're happy to see >> GUI-based affiliated packages. >> >> >> As for Pandas, to be totally honest, I don't see a huge amount to be >> gained from adding a Pandas dependency Astropy. It's honestly not clear >> what it gives the astronomy community that numpy does not already have. The >> following quote from the Pandas web site has guided me to that conclusion: >> "pandas helps fill this gap, enabling you to carry out your entire data >> analysis workflow in Python without having to switch to a more domain >> specific language like R." >> >> I have been carrying out my entire data analysis workflow for some time >> now in python without using Pandas. It looks to me like Pandas is a tool >> that was written by and for statisticians who use R. While we can take >> lessons from this, it's not clear we get much out of it in an astronomy >> context. For example, how would it make astropy's NDData, Quantity, or Table >> better to use a Pandas DataFrame vs. a numpy array? Most of what we are >> doing is building astronomy-convenient interfaces, and I'm not sure what >> Pandas adds there, at the cost of a pretty heavy-weight dependency. >> >> It could just be that I don't know enough about Pandas, though. So if >> someone who knows Pandas better can speak to this, I'm all ears. >> >> >> >> >> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen >> wrote: >>> >>> Pandas is a part of the newly-defined SciPy stack, after all, so that >>> would be part of any science-oriented distribution worth its salt. In fact, >>> I think it could be a good idea for astropy in general to use under the >>> hood, but again, could clash with the philosophy of the project and possibly >>> also maintainabillity. >>> >>> As for offering my code or just my experience, I'll have to square it >>> with my supervisor first, and I also think it depends on what direction the >>> project in question will take. I'm positive about the idea (which is why I >>> wrote in the first place), but supervisor might think it is a better idea to >>> actually get my paper in the project wrapped up before sending the code out >>> there. Will get back about that one! >>> >>> /Emil >>> >>> >>> >>> >>> >>> On 2013-06-18 20:53, Slavin, Jonathan wrote: >>> >>> Hi Emil, >>> >>> That looks very nice! I don't see Pandas as a big issue in terms of >>> dependencies. I don't know that much about traits, etc. My thought about >>> the gui was just based on my experience with matplotlib, and the fact that >>> it is widely used -- though I would agree that too many dependencies can be >>> a deterrent to people using something. Are you offering your code as a >>> starting point for the project? It strikes me that many have gotten some >>> sort of fitting package to a point of personal usability but no one has the >>> time/interest/motivation to make a more generally usable package. >>> >>> Jon >>> >>> On Tue, Jun 18, 2013 at 2:34 PM, wrote: >>>> >>>> Date: Tue, 18 Jun 2013 20:39:55 +0200 >>>> From: Th?ger Rivera-Thorsen >>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>>> To: astropy at scipy.org >>>> Message-ID: <51C0A97B.8090703 at gmail.com> >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> >>>> I have been working on a fitting GUI for a while, although it is made >>>> with a specific task in mind. >>>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and >>>> Pandas. It is made for a specific projhect I'm working and as such not >>>> yet usable for more general cases, but it could be a starting point, if >>>> the dependencies don't conflict with astropy politics. >>>> >>>> Especially, I am happy about the choice of Pandas for managing a quite >>>> complex data structure (the fitted and/or guessed values of an arbitrary >>>> number of transitions for an arbitrary number of rows or collapsed rows >>>> of a spatially resolved spectrum) of a), but also with the Traits-based >>>> interactive interface to build complex line profiles from single >>>> gaussians, good for fitting-by-eye and giving good initial guesses for >>>> fitting of complex line profiles. It hooks directly up to a wrapper I've >>>> made for lmfit, but given the modularity, it should be relatively easy >>>> to change to other backends. >>>> >>>> It's still a work-in-progress, but there are some screenshots here: >>>> http://flic.kr/s/aHsjGaEMGg . >>>> I know the choice and number of dependencies may be prohibitive but it >>>> saved a lot of work on the GUI, and Pandas means the difference between >>>> sanity and madness when it comes to keeping track of so many parameters. >>>> >>>> Cheers, >>>> Emil >>> >>> >>> >>> >>> ________________________________________________________ >>> Jonathan D. Slavin Harvard-Smithsonian CfA >>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >>> phone: (617) 496-7981 Cambridge, MA 02138-1516 >>> fax: (617) 496-7577 USA >>> ________________________________________________________ >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >> >> >> >> -- >> Erik >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > > -- > ************************************ > Chris Beaumont > Graduate Student > Institute for Astronomy > University of Hawaii at Manoa > 2680 Woodlawn Drive > Honolulu, HI 96822 > www.ifa.hawaii.edu/~beaumont > ************************************ -- Erik From te.pickering at gmail.com Thu Jun 20 13:16:48 2013 From: te.pickering at gmail.com (Timothy Pickering) Date: Thu, 20 Jun 2013 19:16:48 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <5990E89E-217D-4E71-8775-F625E9D84BAA@gmail.com> > I thought I'd chime in on the pandas discussion :) > > I'm starting to use pandas a bit more in my day-to-day work. The two features most useful to me are: > > 1) Its file parsers are pretty robust and fast. I always try parsing CSV with pandas first > > 2) For tables tables with lots of categorical data, the grouping functionality is very nice. For example, calculations like "the mean age of each spectral type of star in my catalog" are usually one liners like df.groupby(['spectral_type']).age.mean. I spend a lot of time on the "split-apply-combine" page on the pandas docs (http://pandas.pydata.org/pandas-docs/stable/groupby.html). > > I won't speculate about whether that's enough an asset to warrant a dependency in astropy. I do agree that lots of other pandas features don't translate as well into astronomy use. i'll add my R0.02 and second the two points given above. what i find pandas most useful for is combining different sets of data taken with different timestamps and intervals. e.g., weather data from different stations, telescope telemetry, seeing monitor telemetry, etc. can all be combined into one table in a sane fashion with proper handling of missing/bad data. i'm not sure pandas should be an astropy dependency, but they're definitely complementary. it's pretty well integrated with matplotlib and numpy so you can pull data out of pandas tables and manipulate it in the usual numpy way. what pandas gives you is a higher level interface for handling and managing the data. i've successfully chunked through multi-GB datasets with it tim -- +-------------------------------------------------------------------+ | T. E. Pickering, Ph.D. | Southern African Large Telescope | | SALT Astronomer | SAAO | | tim at saao.ac.za (520) 305-9823 | Observatory Road | | tim at salt.ac.za +27-71-551-8281 | 7925 Observatory, South Africa | +-------------------------------------------------------------------+ overflow error in /dev/null From te.pickering at gmail.com Thu Jun 20 13:37:10 2013 From: te.pickering at gmail.com (Timothy Pickering) Date: Thu, 20 Jun 2013 19:37:10 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: > Hmm, ok, those seem like use-cases that overlap a lot with > astropy.table and astropy.io.ascii. Do you find pandas is noticeably > faster than astropy.tables? haven't directly compared. a pandas data frame is a higher level construct than what astropy.table provides. it's more than just a table. as far as the IO stuff, pandas can be finicky and doesn't provide the breadth of format support like astropy does (e.g. FITS or latex). it should be possible to relatively easily convert between astropy.table.Table and pandas.DataFrame. been meaning to look into that, but haven't gotten around to it. tim -- +-------------------------------------------------------------------+ | T. E. Pickering, Ph.D. | Southern African Large Telescope | | SALT Astronomer | SAAO | | tim at saao.ac.za (520) 305-9823 | Observatory Road | | tim at salt.ac.za +27-71-551-8281 | 7925 Observatory, South Africa | +-------------------------------------------------------------------+ overflow error in /dev/null From aldcroft at head.cfa.harvard.edu Thu Jun 20 13:38:18 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 20 Jun 2013 13:38:18 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: On Thu, Jun 20, 2013 at 12:50 PM, Chris Beaumont wrote: > I thought I'd chime in on the pandas discussion :) > > I'm starting to use pandas a bit more in my day-to-day work. The two > features most useful to me are: > > 1) Its file parsers are pretty robust and fast. I always try parsing CSV > with pandas first > I've wondered how hard it would be to incorporate some of the pandas CSV fast reading functions for the easy cases. I'm assuming it is licensed so that would be an option. > > 2) For tables tables with lots of categorical data, the grouping > functionality is very nice. For example, calculations like "the mean age of > each spectral type of star in my catalog" are usually one liners like > df.groupby(['spectral_type']).age.mean. I spend a lot of time on the > "split-apply-combine" page on the pandas docs ( > http://pandas.pydata.org/pandas-docs/stable/groupby.html). > Group-by and related functionality is top on my list of priorities for astropy.table (in fact I see it every day on my google keep app...). Join and merging are in master now. In my tests the astropy table join is within a factor of 2 to 3 in speed relative to pandas, so in most use cases it should be good enough. It's probably worth pointing out to the community that it was not a lightly-taken decision to reject pandas for use as the base data storage container. For the case of tables there is one show-stopper which is that pandas DataFrame does not support arbitrary multi-dimensional columns, i.e. column where each element is itself an N-d array. These occur enough in astronomy and are supported by FITS and VO standards, so the astropy Table must be able to represent that. The lack of support for table and column metadata is a smaller but still important issue. Having said that, there is no question pandas has a ton of highly-efficient and useful machinery and we are working on ways to improve inter-operability. This includes being able convert between Table and DataFrame easily. Suggestions and (especially) pull requests welcome. > > I won't speculate about whether that's enough an asset to warrant a > dependency in astropy. I do agree that lots of other pandas features don't > translate as well into astronomy use. > > > > On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud wrote: > >> I'm of mixed minds about traits UI because once you know it you can make >> great GUIs with it, but I've spent a lot of time troubleshooting people's >> python installations to get traits to work. That is, in general it can be >> tricky to get installed because of all the dependencies. Maybe this has >> improved recently with Enthought's Canopy (or other new python distros), >> but that's been my past experience. >> >> More generally, the view in the astropy core package is that we don't >> want to put GUIs in the core because GUIs always carry lots of >> dependencies, which we don't want to be forced to deal with. But part of >> the whole reason for affiliated packages was to get around this, so we're >> happy to see GUI-based affiliated packages. >> >> >> As for Pandas, to be totally honest, I don't see a huge amount to be >> gained from adding a Pandas dependency Astropy. It's honestly not clear >> what it gives the astronomy community that numpy does not already have. >> The following quote from the Pandas web site has guided me to that >> conclusion: "*pandas* helps fill this gap, enabling you to carry out >> your entire data analysis workflow in Python without having to switch to a >> more domain specific language like R." >> >> I have been carrying out my entire data analysis workflow for some time >> now in python without using Pandas. It looks to me like Pandas is a tool >> that was written by and for statisticians who use R. While we can take >> lessons from this, it's not clear we get much out of it in an astronomy >> context. For example, how would it make astropy's NDData, Quantity, or >> Table better to use a Pandas DataFrame vs. a numpy array? Most of what we >> are doing is building astronomy-convenient interfaces, and I'm not sure >> what Pandas adds there, at the cost of a pretty heavy-weight dependency. >> >> It could just be that I don't know enough about Pandas, though. So if >> someone who knows Pandas better can speak to this, I'm all ears. >> >> >> >> >> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen > > wrote: >> >>> Pandas is a part of the newly-defined SciPy stack, after all, so that >>> would be part of any science-oriented distribution worth its salt. In fact, >>> I think it could be a good idea for astropy in general to use under the >>> hood, but again, could clash with the philosophy of the project and >>> possibly also maintainabillity. >>> >>> As for offering my code or just my experience, I'll have to square it >>> with my supervisor first, and I also think it depends on what direction the >>> project in question will take. I'm positive about the idea (which is why I >>> wrote in the first place), but supervisor might think it is a better idea >>> to actually get my paper in the project wrapped up before sending the code >>> out there. Will get back about that one! >>> >>> /Emil >>> >>> >>> >>> >>> >>> On 2013-06-18 20:53, Slavin, Jonathan wrote: >>> >>> Hi Emil, >>> >>> That looks very nice! I don't see Pandas as a big issue in terms of >>> dependencies. I don't know that much about traits, etc. My thought about >>> the gui was just based on my experience with matplotlib, and the fact that >>> it is widely used -- though I would agree that too many dependencies can be >>> a deterrent to people using something. Are you offering your code as a >>> starting point for the project? It strikes me that many have gotten some >>> sort of fitting package to a point of personal usability but no one has the >>> time/interest/motivation to make a more generally usable package. >>> >>> Jon >>> >>> On Tue, Jun 18, 2013 at 2:34 PM, wrote: >>> >>>> Date: Tue, 18 Jun 2013 20:39:55 +0200 >>>> From: Th?ger Rivera-Thorsen >>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>>> To: astropy at scipy.org >>>> Message-ID: <51C0A97B.8090703 at gmail.com> >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> >>>> I have been working on a fitting GUI for a while, although it is made >>>> with a specific task in mind. >>>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and >>>> Pandas. It is made for a specific projhect I'm working and as such not >>>> yet usable for more general cases, but it could be a starting point, if >>>> the dependencies don't conflict with astropy politics. >>>> >>>> Especially, I am happy about the choice of Pandas for managing a quite >>>> complex data structure (the fitted and/or guessed values of an arbitrary >>>> number of transitions for an arbitrary number of rows or collapsed rows >>>> of a spatially resolved spectrum) of a), but also with the Traits-based >>>> interactive interface to build complex line profiles from single >>>> gaussians, good for fitting-by-eye and giving good initial guesses for >>>> fitting of complex line profiles. It hooks directly up to a wrapper I've >>>> made for lmfit, but given the modularity, it should be relatively easy >>>> to change to other backends. >>>> >>>> It's still a work-in-progress, but there are some screenshots here: >>>> http://flic.kr/s/aHsjGaEMGg . >>>> I know the choice and number of dependencies may be prohibitive but it >>>> saved a lot of work on the GUI, and Pandas means the difference between >>>> sanity and madness when it comes to keeping track of so many parameters. >>>> >>>> Cheers, >>>> Emil >>>> >>> >>> >>> >>> ________________________________________________________ >>> Jonathan D. Slavin Harvard-Smithsonian CfA >>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >>> phone: (617) 496-7981 Cambridge, MA 02138-1516 >>> fax: (617) 496-7577 USA >>> ________________________________________________________ >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >> >> >> -- >> Erik >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > > -- > ************************************ > Chris Beaumont > Graduate Student > Institute for Astronomy > University of Hawaii at Manoa > 2680 Woodlawn Drive > Honolulu, HI 96822 > www.ifa.hawaii.edu/~beaumont > ************************************ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at depagne.org Thu Jun 20 13:47:24 2013 From: eric at depagne.org (=?iso-8859-1?q?=C9ric_Depagne?=) Date: Thu, 20 Jun 2013 19:47:24 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: Message-ID: <201306201947.24279.eric@depagne.org> Just a quick question: Since the discussion is about what pandas, wouldn't it be possible to add in cc some pandas devs, to let them know? ?ric. > On Thu, Jun 20, 2013 at 12:50 PM, Chris Beaumont wrote: > > I thought I'd chime in on the pandas discussion :) > > > > I'm starting to use pandas a bit more in my day-to-day work. The two > > features most useful to me are: > > > > 1) Its file parsers are pretty robust and fast. I always try parsing CSV > > with pandas first > > I've wondered how hard it would be to incorporate some of the pandas CSV > fast reading functions for the easy cases. I'm assuming it is licensed so > that would be an option. > > > 2) For tables tables with lots of categorical data, the grouping > > functionality is very nice. For example, calculations like "the mean age > > of each spectral type of star in my catalog" are usually one liners like > > df.groupby(['spectral_type']).age.mean. I spend a lot of time on the > > "split-apply-combine" page on the pandas docs ( > > http://pandas.pydata.org/pandas-docs/stable/groupby.html). > > Group-by and related functionality is top on my list of priorities for > astropy.table (in fact I see it every day on my google keep app...). Join > and merging are in master now. In my tests the astropy table join is > within a factor of 2 to 3 in speed relative to pandas, so in most use cases > it should be good enough. > > It's probably worth pointing out to the community that it was not a > lightly-taken decision to reject pandas for use as the base data storage > container. For the case of tables there is one show-stopper which is that > pandas DataFrame does not support arbitrary multi-dimensional columns, i.e. > column where each element is itself an N-d array. These occur enough in > astronomy and are supported by FITS and VO standards, so the astropy Table > must be able to represent that. The lack of support for table and column > metadata is a smaller but still important issue. > > Having said that, there is no question pandas has a ton of highly-efficient > and useful machinery and we are working on ways to improve > inter-operability. This includes being able convert between Table and > DataFrame easily. Suggestions and (especially) pull requests welcome. > > > I won't speculate about whether that's enough an asset to warrant a > > dependency in astropy. I do agree that lots of other pandas features > > don't translate as well into astronomy use. > > > > On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud wrote: > >> I'm of mixed minds about traits UI because once you know it you can make > >> great GUIs with it, but I've spent a lot of time troubleshooting > >> people's python installations to get traits to work. That is, in > >> general it can be tricky to get installed because of all the > >> dependencies. Maybe this has improved recently with Enthought's Canopy > >> (or other new python distros), but that's been my past experience. > >> > >> More generally, the view in the astropy core package is that we don't > >> want to put GUIs in the core because GUIs always carry lots of > >> dependencies, which we don't want to be forced to deal with. But part > >> of the whole reason for affiliated packages was to get around this, so > >> we're happy to see GUI-based affiliated packages. > >> > >> > >> As for Pandas, to be totally honest, I don't see a huge amount to be > >> gained from adding a Pandas dependency Astropy. It's honestly not clear > >> what it gives the astronomy community that numpy does not already have. > >> > >> The following quote from the Pandas web site has guided me to that > >> > >> conclusion: "*pandas* helps fill this gap, enabling you to carry out > >> your entire data analysis workflow in Python without having to switch to > >> a more domain specific language like R." > >> > >> I have been carrying out my entire data analysis workflow for some time > >> now in python without using Pandas. It looks to me like Pandas is a > >> tool that was written by and for statisticians who use R. While we can > >> take lessons from this, it's not clear we get much out of it in an > >> astronomy context. For example, how would it make astropy's NDData, > >> Quantity, or Table better to use a Pandas DataFrame vs. a numpy array? > >> Most of what we are doing is building astronomy-convenient interfaces, > >> and I'm not sure what Pandas adds there, at the cost of a pretty > >> heavy-weight dependency. > >> > >> It could just be that I don't know enough about Pandas, though. So if > >> someone who knows Pandas better can speak to this, I'm all ears. > >> > >> > >> > >> > >> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen > >> >> > >> > wrote: > >>> Pandas is a part of the newly-defined SciPy stack, after all, so that > >>> > >>> would be part of any science-oriented distribution worth its salt. In > >>> fact, I think it could be a good idea for astropy in general to use > >>> under the hood, but again, could clash with the philosophy of the > >>> project and possibly also maintainabillity. > >>> > >>> As for offering my code or just my experience, I'll have to square it > >>> with my supervisor first, and I also think it depends on what direction > >>> the project in question will take. I'm positive about the idea (which > >>> is why I wrote in the first place), but supervisor might think it is a > >>> better idea to actually get my paper in the project wrapped up before > >>> sending the code out there. Will get back about that one! > >>> > >>> /Emil > >>> > >>> > >>> > >>> > >>> > >>> On 2013-06-18 20:53, Slavin, Jonathan wrote: > >>> > >>> Hi Emil, > >>> > >>> That looks very nice! I don't see Pandas as a big issue in terms of > >>> > >>> dependencies. I don't know that much about traits, etc. My thought > >>> about the gui was just based on my experience with matplotlib, and the > >>> fact that it is widely used -- though I would agree that too many > >>> dependencies can be a deterrent to people using something. Are you > >>> offering your code as a starting point for the project? It strikes me > >>> that many have gotten some sort of fitting package to a point of > >>> personal usability but no one has the time/interest/motivation to make > >>> a more generally usable package. > >>> > >>> Jon > >>> > >>> On Tue, Jun 18, 2013 at 2:34 PM, wrote: > >>>> Date: Tue, 18 Jun 2013 20:39:55 +0200 > >>>> From: Th?ger Rivera-Thorsen > >>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > >>>> To: astropy at scipy.org > >>>> Message-ID: <51C0A97B.8090703 at gmail.com> > >>>> Content-Type: text/plain; charset="iso-8859-1" > >>>> > >>>> I have been working on a fitting GUI for a while, although it is made > >>>> with a specific task in mind. > >>>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco > >>>> and Pandas. It is made for a specific projhect I'm working and as > >>>> such not yet usable for more general cases, but it could be a > >>>> starting point, if the dependencies don't conflict with astropy > >>>> politics. > >>>> > >>>> Especially, I am happy about the choice of Pandas for managing a quite > >>>> complex data structure (the fitted and/or guessed values of an > >>>> arbitrary number of transitions for an arbitrary number of rows or > >>>> collapsed rows of a spatially resolved spectrum) of a), but also with > >>>> the Traits-based interactive interface to build complex line profiles > >>>> from single gaussians, good for fitting-by-eye and giving good > >>>> initial guesses for fitting of complex line profiles. It hooks > >>>> directly up to a wrapper I've made for lmfit, but given the > >>>> modularity, it should be relatively easy to change to other backends. > >>>> > >>>> It's still a work-in-progress, but there are some screenshots here: > >>>> http://flic.kr/s/aHsjGaEMGg . > >>>> I know the choice and number of dependencies may be prohibitive but it > >>>> saved a lot of work on the GUI, and Pandas means the difference > >>>> between sanity and madness when it comes to keeping track of so many > >>>> parameters. > >>>> > >>>> Cheers, > >>>> Emil > >>>> > >>> ________________________________________________________ > >>> > >>> Jonathan D. Slavin Harvard-Smithsonian CfA > >>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > >>> phone: (617) 496-7981 Cambridge, MA 02138-1516 > >>> fax: (617) 496-7577 USA > >>> ________________________________________________________ > >>> > >>> > >>> > >>> _______________________________________________ > >>> AstroPy mailing > >>> listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy > >>> > >>> > >>> > >>> _______________________________________________ > >>> AstroPy mailing list > >>> AstroPy at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/astropy > >> > >> -- > >> Erik > >> > >> _______________________________________________ > >> AstroPy mailing list > >> AstroPy at scipy.org > >> http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > > ************************************ > > Chris Beaumont > > Graduate Student > > Institute for Astronomy > > University of Hawaii at Manoa > > 2680 Woodlawn Drive > > Honolulu, HI 96822 > > www.ifa.hawaii.edu/~beaumont > > ************************************ > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy Un clavier azerty en vaut deux ---------------------------------------------------------- ?ric Depagne eric at depagne.org From eric at depagne.org Thu Jun 20 13:47:24 2013 From: eric at depagne.org (=?iso-8859-1?q?=C9ric_Depagne?=) Date: Thu, 20 Jun 2013 19:47:24 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: Message-ID: <201306201947.24279.eric@depagne.org> Just a quick question: Since the discussion is about what pandas, wouldn't it be possible to add in cc some pandas devs, to let them know? ?ric. > On Thu, Jun 20, 2013 at 12:50 PM, Chris Beaumont wrote: > > I thought I'd chime in on the pandas discussion :) > > > > I'm starting to use pandas a bit more in my day-to-day work. The two > > features most useful to me are: > > > > 1) Its file parsers are pretty robust and fast. I always try parsing CSV > > with pandas first > > I've wondered how hard it would be to incorporate some of the pandas CSV > fast reading functions for the easy cases. I'm assuming it is licensed so > that would be an option. > > > 2) For tables tables with lots of categorical data, the grouping > > functionality is very nice. For example, calculations like "the mean age > > of each spectral type of star in my catalog" are usually one liners like > > df.groupby(['spectral_type']).age.mean. I spend a lot of time on the > > "split-apply-combine" page on the pandas docs ( > > http://pandas.pydata.org/pandas-docs/stable/groupby.html). > > Group-by and related functionality is top on my list of priorities for > astropy.table (in fact I see it every day on my google keep app...). Join > and merging are in master now. In my tests the astropy table join is > within a factor of 2 to 3 in speed relative to pandas, so in most use cases > it should be good enough. > > It's probably worth pointing out to the community that it was not a > lightly-taken decision to reject pandas for use as the base data storage > container. For the case of tables there is one show-stopper which is that > pandas DataFrame does not support arbitrary multi-dimensional columns, i.e. > column where each element is itself an N-d array. These occur enough in > astronomy and are supported by FITS and VO standards, so the astropy Table > must be able to represent that. The lack of support for table and column > metadata is a smaller but still important issue. > > Having said that, there is no question pandas has a ton of highly-efficient > and useful machinery and we are working on ways to improve > inter-operability. This includes being able convert between Table and > DataFrame easily. Suggestions and (especially) pull requests welcome. > > > I won't speculate about whether that's enough an asset to warrant a > > dependency in astropy. I do agree that lots of other pandas features > > don't translate as well into astronomy use. > > > > On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud wrote: > >> I'm of mixed minds about traits UI because once you know it you can make > >> great GUIs with it, but I've spent a lot of time troubleshooting > >> people's python installations to get traits to work. That is, in > >> general it can be tricky to get installed because of all the > >> dependencies. Maybe this has improved recently with Enthought's Canopy > >> (or other new python distros), but that's been my past experience. > >> > >> More generally, the view in the astropy core package is that we don't > >> want to put GUIs in the core because GUIs always carry lots of > >> dependencies, which we don't want to be forced to deal with. But part > >> of the whole reason for affiliated packages was to get around this, so > >> we're happy to see GUI-based affiliated packages. > >> > >> > >> As for Pandas, to be totally honest, I don't see a huge amount to be > >> gained from adding a Pandas dependency Astropy. It's honestly not clear > >> what it gives the astronomy community that numpy does not already have. > >> > >> The following quote from the Pandas web site has guided me to that > >> > >> conclusion: "*pandas* helps fill this gap, enabling you to carry out > >> your entire data analysis workflow in Python without having to switch to > >> a more domain specific language like R." > >> > >> I have been carrying out my entire data analysis workflow for some time > >> now in python without using Pandas. It looks to me like Pandas is a > >> tool that was written by and for statisticians who use R. While we can > >> take lessons from this, it's not clear we get much out of it in an > >> astronomy context. For example, how would it make astropy's NDData, > >> Quantity, or Table better to use a Pandas DataFrame vs. a numpy array? > >> Most of what we are doing is building astronomy-convenient interfaces, > >> and I'm not sure what Pandas adds there, at the cost of a pretty > >> heavy-weight dependency. > >> > >> It could just be that I don't know enough about Pandas, though. So if > >> someone who knows Pandas better can speak to this, I'm all ears. > >> > >> > >> > >> > >> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen > >> >> > >> > wrote: > >>> Pandas is a part of the newly-defined SciPy stack, after all, so that > >>> > >>> would be part of any science-oriented distribution worth its salt. In > >>> fact, I think it could be a good idea for astropy in general to use > >>> under the hood, but again, could clash with the philosophy of the > >>> project and possibly also maintainabillity. > >>> > >>> As for offering my code or just my experience, I'll have to square it > >>> with my supervisor first, and I also think it depends on what direction > >>> the project in question will take. I'm positive about the idea (which > >>> is why I wrote in the first place), but supervisor might think it is a > >>> better idea to actually get my paper in the project wrapped up before > >>> sending the code out there. Will get back about that one! > >>> > >>> /Emil > >>> > >>> > >>> > >>> > >>> > >>> On 2013-06-18 20:53, Slavin, Jonathan wrote: > >>> > >>> Hi Emil, > >>> > >>> That looks very nice! I don't see Pandas as a big issue in terms of > >>> > >>> dependencies. I don't know that much about traits, etc. My thought > >>> about the gui was just based on my experience with matplotlib, and the > >>> fact that it is widely used -- though I would agree that too many > >>> dependencies can be a deterrent to people using something. Are you > >>> offering your code as a starting point for the project? It strikes me > >>> that many have gotten some sort of fitting package to a point of > >>> personal usability but no one has the time/interest/motivation to make > >>> a more generally usable package. > >>> > >>> Jon > >>> > >>> On Tue, Jun 18, 2013 at 2:34 PM, wrote: > >>>> Date: Tue, 18 Jun 2013 20:39:55 +0200 > >>>> From: Th?ger Rivera-Thorsen > >>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > >>>> To: astropy at scipy.org > >>>> Message-ID: <51C0A97B.8090703 at gmail.com> > >>>> Content-Type: text/plain; charset="iso-8859-1" > >>>> > >>>> I have been working on a fitting GUI for a while, although it is made > >>>> with a specific task in mind. > >>>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco > >>>> and Pandas. It is made for a specific projhect I'm working and as > >>>> such not yet usable for more general cases, but it could be a > >>>> starting point, if the dependencies don't conflict with astropy > >>>> politics. > >>>> > >>>> Especially, I am happy about the choice of Pandas for managing a quite > >>>> complex data structure (the fitted and/or guessed values of an > >>>> arbitrary number of transitions for an arbitrary number of rows or > >>>> collapsed rows of a spatially resolved spectrum) of a), but also with > >>>> the Traits-based interactive interface to build complex line profiles > >>>> from single gaussians, good for fitting-by-eye and giving good > >>>> initial guesses for fitting of complex line profiles. It hooks > >>>> directly up to a wrapper I've made for lmfit, but given the > >>>> modularity, it should be relatively easy to change to other backends. > >>>> > >>>> It's still a work-in-progress, but there are some screenshots here: > >>>> http://flic.kr/s/aHsjGaEMGg . > >>>> I know the choice and number of dependencies may be prohibitive but it > >>>> saved a lot of work on the GUI, and Pandas means the difference > >>>> between sanity and madness when it comes to keeping track of so many > >>>> parameters. > >>>> > >>>> Cheers, > >>>> Emil > >>>> > >>> ________________________________________________________ > >>> > >>> Jonathan D. Slavin Harvard-Smithsonian CfA > >>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > >>> phone: (617) 496-7981 Cambridge, MA 02138-1516 > >>> fax: (617) 496-7577 USA > >>> ________________________________________________________ > >>> > >>> > >>> > >>> _______________________________________________ > >>> AstroPy mailing > >>> listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy > >>> > >>> > >>> > >>> _______________________________________________ > >>> AstroPy mailing list > >>> AstroPy at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/astropy > >> > >> -- > >> Erik > >> > >> _______________________________________________ > >> AstroPy mailing list > >> AstroPy at scipy.org > >> http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > > ************************************ > > Chris Beaumont > > Graduate Student > > Institute for Astronomy > > University of Hawaii at Manoa > > 2680 Woodlawn Drive > > Honolulu, HI 96822 > > www.ifa.hawaii.edu/~beaumont > > ************************************ > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy Un clavier azerty en vaut deux ---------------------------------------------------------- ?ric Depagne eric at depagne.org From wkerzendorf at gmail.com Thu Jun 20 14:09:17 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Thu, 20 Jun 2013 14:09:17 -0400 Subject: [AstroPy] Pandas vs Astropy tables. In-Reply-To: <201306201947.24279.eric@depagne.org> References: <201306201947.24279.eric@depagne.org> Message-ID: I think we're all forgetting one of the messages of astropy: foster interoperability between Python astronomy (and science) packages. I use pandas a lot in my radiative transfer code as I need (among other things) to do group by functionality as fast as possible (and sqlite is too slow). However when I read in the tables they first go through astropy tables because it can handle units and converts them all to cgs. I think we should not try to compete with PANDAS but foster interoperability (similarly to us not trying to compete with numpy or scipy) . PANDAS will never care about writing tables to a deluxe table and that's where astropy shines (for example). So if you need to read in CSV files (and speed is important, which it is often not) I have no problem recommending pandas, however if you need to deal with DAOPhot, CDS, etc. files there's no alternative to astropy.tables. For now I don't see a reason for including PANDAS in astropy core, as we don't have a need for its functionality there. But including this in affiliated packages (if needed) I think is a good idea. My 2 cents, Wolfgang On 2013-06-20, at 1:47 PM, ?ric Depagne wrote: > Just a quick question: > > Since the discussion is about what pandas, wouldn't it be possible to add in > cc some pandas devs, to let them know? > > ?ric. >> On Thu, Jun 20, 2013 at 12:50 PM, Chris Beaumont wrote: >>> I thought I'd chime in on the pandas discussion :) >>> >>> I'm starting to use pandas a bit more in my day-to-day work. The two >>> features most useful to me are: >>> >>> 1) Its file parsers are pretty robust and fast. I always try parsing CSV >>> with pandas first >> >> I've wondered how hard it would be to incorporate some of the pandas CSV >> fast reading functions for the easy cases. I'm assuming it is licensed so >> that would be an option. >> >>> 2) For tables tables with lots of categorical data, the grouping >>> functionality is very nice. For example, calculations like "the mean age >>> of each spectral type of star in my catalog" are usually one liners like >>> df.groupby(['spectral_type']).age.mean. I spend a lot of time on the >>> "split-apply-combine" page on the pandas docs ( >>> http://pandas.pydata.org/pandas-docs/stable/groupby.html). >> >> Group-by and related functionality is top on my list of priorities for >> astropy.table (in fact I see it every day on my google keep app...). Join >> and merging are in master now. In my tests the astropy table join is >> within a factor of 2 to 3 in speed relative to pandas, so in most use cases >> it should be good enough. >> >> It's probably worth pointing out to the community that it was not a >> lightly-taken decision to reject pandas for use as the base data storage >> container. For the case of tables there is one show-stopper which is that >> pandas DataFrame does not support arbitrary multi-dimensional columns, i.e. >> column where each element is itself an N-d array. These occur enough in >> astronomy and are supported by FITS and VO standards, so the astropy Table >> must be able to represent that. The lack of support for table and column >> metadata is a smaller but still important issue. >> >> Having said that, there is no question pandas has a ton of highly-efficient >> and useful machinery and we are working on ways to improve >> inter-operability. This includes being able convert between Table and >> DataFrame easily. Suggestions and (especially) pull requests welcome. >> >>> I won't speculate about whether that's enough an asset to warrant a >>> dependency in astropy. I do agree that lots of other pandas features >>> don't translate as well into astronomy use. >>> >>> On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud > wrote: >>>> I'm of mixed minds about traits UI because once you know it you can make >>>> great GUIs with it, but I've spent a lot of time troubleshooting >>>> people's python installations to get traits to work. That is, in >>>> general it can be tricky to get installed because of all the >>>> dependencies. Maybe this has improved recently with Enthought's Canopy >>>> (or other new python distros), but that's been my past experience. >>>> >>>> More generally, the view in the astropy core package is that we don't >>>> want to put GUIs in the core because GUIs always carry lots of >>>> dependencies, which we don't want to be forced to deal with. But part >>>> of the whole reason for affiliated packages was to get around this, so >>>> we're happy to see GUI-based affiliated packages. >>>> >>>> >>>> As for Pandas, to be totally honest, I don't see a huge amount to be >>>> gained from adding a Pandas dependency Astropy. It's honestly not clear >>>> what it gives the astronomy community that numpy does not already have. >>>> >>>> The following quote from the Pandas web site has guided me to that >>>> >>>> conclusion: "*pandas* helps fill this gap, enabling you to carry out >>>> your entire data analysis workflow in Python without having to switch to >>>> a more domain specific language like R." >>>> >>>> I have been carrying out my entire data analysis workflow for some time >>>> now in python without using Pandas. It looks to me like Pandas is a >>>> tool that was written by and for statisticians who use R. While we can >>>> take lessons from this, it's not clear we get much out of it in an >>>> astronomy context. For example, how would it make astropy's NDData, >>>> Quantity, or Table better to use a Pandas DataFrame vs. a numpy array? >>>> Most of what we are doing is building astronomy-convenient interfaces, >>>> and I'm not sure what Pandas adds there, at the cost of a pretty >>>> heavy-weight dependency. >>>> >>>> It could just be that I don't know enough about Pandas, though. So if >>>> someone who knows Pandas better can speak to this, I'm all ears. >>>> >>>> >>>> >>>> >>>> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen >>>> >>> >>>>> wrote: >>>>> Pandas is a part of the newly-defined SciPy stack, after all, so that >>>>> >>>>> would be part of any science-oriented distribution worth its salt. In >>>>> fact, I think it could be a good idea for astropy in general to use >>>>> under the hood, but again, could clash with the philosophy of the >>>>> project and possibly also maintainabillity. >>>>> >>>>> As for offering my code or just my experience, I'll have to square it >>>>> with my supervisor first, and I also think it depends on what direction >>>>> the project in question will take. I'm positive about the idea (which >>>>> is why I wrote in the first place), but supervisor might think it is a >>>>> better idea to actually get my paper in the project wrapped up before >>>>> sending the code out there. Will get back about that one! >>>>> >>>>> /Emil >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 2013-06-18 20:53, Slavin, Jonathan wrote: >>>>> >>>>> Hi Emil, >>>>> >>>>> That looks very nice! I don't see Pandas as a big issue in terms of >>>>> >>>>> dependencies. I don't know that much about traits, etc. My thought >>>>> about the gui was just based on my experience with matplotlib, and the >>>>> fact that it is widely used -- though I would agree that too many >>>>> dependencies can be a deterrent to people using something. Are you >>>>> offering your code as a starting point for the project? It strikes me >>>>> that many have gotten some sort of fitting package to a point of >>>>> personal usability but no one has the time/interest/motivation to make >>>>> a more generally usable package. >>>>> >>>>> Jon >>>>> >>>>> On Tue, Jun 18, 2013 at 2:34 PM, wrote: >>>>>> Date: Tue, 18 Jun 2013 20:39:55 +0200 >>>>>> From: Th?ger Rivera-Thorsen >>>>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>>>>> To: astropy at scipy.org >>>>>> Message-ID: <51C0A97B.8090703 at gmail.com> >>>>>> Content-Type: text/plain; charset="iso-8859-1" >>>>>> >>>>>> I have been working on a fitting GUI for a while, although it is made >>>>>> with a specific task in mind. >>>>>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco >>>>>> and Pandas. It is made for a specific projhect I'm working and as >>>>>> such not yet usable for more general cases, but it could be a >>>>>> starting point, if the dependencies don't conflict with astropy >>>>>> politics. >>>>>> >>>>>> Especially, I am happy about the choice of Pandas for managing a quite >>>>>> complex data structure (the fitted and/or guessed values of an >>>>>> arbitrary number of transitions for an arbitrary number of rows or >>>>>> collapsed rows of a spatially resolved spectrum) of a), but also with >>>>>> the Traits-based interactive interface to build complex line profiles >>>>>> from single gaussians, good for fitting-by-eye and giving good >>>>>> initial guesses for fitting of complex line profiles. It hooks >>>>>> directly up to a wrapper I've made for lmfit, but given the >>>>>> modularity, it should be relatively easy to change to other backends. >>>>>> >>>>>> It's still a work-in-progress, but there are some screenshots here: >>>>>> http://flic.kr/s/aHsjGaEMGg . >>>>>> I know the choice and number of dependencies may be prohibitive but it >>>>>> saved a lot of work on the GUI, and Pandas means the difference >>>>>> between sanity and madness when it comes to keeping track of so many >>>>>> parameters. >>>>>> >>>>>> Cheers, >>>>>> Emil >>>>>> >>>>> ________________________________________________________ >>>>> >>>>> Jonathan D. Slavin Harvard-Smithsonian CfA >>>>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >>>>> phone: (617) 496-7981 Cambridge, MA 02138-1516 >>>>> fax: (617) 496-7577 USA >>>>> ________________________________________________________ >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> AstroPy mailing >>>>> listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> AstroPy mailing list >>>>> AstroPy at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/astropy >>>> >>>> -- >>>> Erik >>>> >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> -- >>> ************************************ >>> Chris Beaumont >>> Graduate Student >>> Institute for Astronomy >>> University of Hawaii at Manoa >>> 2680 Woodlawn Drive >>> Honolulu, HI 96822 >>> www.ifa.hawaii.edu/~beaumont >>> ************************************ >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy > Un clavier azerty en vaut deux > ---------------------------------------------------------- > ?ric Depagne eric at depagne.org > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From stsci.perry at gmail.com Thu Jun 20 14:57:55 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Thu, 20 Jun 2013 14:57:55 -0400 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: I'd like echo this point as well. But I also recognize that this is a very broad area that one tool isn't going to necessarily cover well. Particularly a GUI-based tool. I'll make some comments on that in a separate message. Even so, if all the tools could share many of the basic elements, that would be a great help. Models is one such thing, units are another, a common data structure for observed spectra (or set of models if needed which probably is the case), which itself ought be be able to handle most of the I/O needs. There should be a lot in common with the different needs. Perry On Jun 20, 2013, at 12:27 PM, Erik Tollerud wrote: > I want to issue a generic plea on this issue: lets do our best to join > forces. There are already a number of tools that do part of spectrum > plotting and line-fitting, and if we'd all work together, with less > effort we'd have a tool that does all of those and more. That's the > main point of Astropy, after all: encouraging collobaration on and > re-use of python tools for astronomy. I'm partly guilty here, as I > have a similar fitting tool in astropysics - > http://pythonhosted.org/Astropysics/gui/spylot.html, but I'd rather > encourage people to use a tool that we can all get behind. I'll call > this the "Astropy spirit" :) > > That said, I see the virtue of designing a new tool (ideally an > astropy affiliated package) to combine the shared wisdom (perhaps with > SOCIS, if it goes through). But the *worst* thing to have happen is to > have the other efforts continue in parallel, with everyone designing > similar but slightly incompatible tools. > > Along similar lines, Emil and Adam (although I think Adam already > knows about it), I'd *strongly* encourage you to take a look at the > astropy modeling framework > (http://docs.astropy.org/en/latest/modeling/index.html) - this will be > in the next major release, and I'd hate to see you re-implementing all > of that in an incompatible way. Or, if you don't like something about > how that works for your needs, please help us make it better! > > > On Thu, Jun 20, 2013 at 8:42 AM, Slavin, Jonathan > wrote: >> Hi Adam, >> >> I have used pyspeckit. There are parts of it that are very useful, but I >> found overall that it didn't do what I wanted. For example, I wanted to >> give it my spectrum, have it display it, allow me to manually (in the gui) >> place my guesses for line centers (would probably need guesses for widths as >> well) and then fit the lines (and continuum) and return the fitted line >> centers, widths, and amplitudes, etc. along with error estimates. It's been >> a while now since I used it, but as I recall, pyspeckit did not do all of >> those things. There may have been other reasons as well, but in the end I >> wrote my own fitting routines to work with the data. Am I wrong about the >> capabilities of pyspeckit? >> >> Jon >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >> phone: (617) 496-7981 Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> On Wed, Jun 19, 2013 at 5:21 PM, wrote: >>> >>> Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) >>> From: Adam Ginsburg >>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>> To: astropy-dev at googlegroups.com >>> Cc: astropy at scipy.org >>> Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5 at googlegroups.com> >>> Content-Type: text/plain; charset="utf-8" >>> >>> >>> Emil and Jon - there is a package that fits exactly those requirements, >>> called pyspeckit: >>> pyspeckit.bitbucket.org >>> >>> One of my major goals over the next year is to incorporate it into astropy >>> bit by bit, i.e. make it work with astropy.models, astropy.units, and >>> specutils. Only a little progress in that direction has been made (in >>> part >>> because my coding efforts have been focused more on astroquery). >>> >>> Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. >>> You can check that it agrees with yours: >>> >>> http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt >>> >>> I'm bringing this up on this particular thread because I had considered >>> submitting pyspeckit as a SOCIS project, but I think focusing on specutils >>> instead will be better for the community. Nonetheless, there are some >>> specific project ideas in place that may help support the case for a >>> student working on specutils via SOCIS: >>> pyspeckit.bitbucket.org/html/sphinx/projects.html >>> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > > -- > Erik > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From stsci.perry at gmail.com Thu Jun 20 15:09:07 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Thu, 20 Jun 2013 15:09:07 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: On Jun 20, 2013, at 12:34 PM, Erik Tollerud wrote: > I'm of mixed minds about traits UI because once you know it you can make great GUIs with it, but I've spent a lot of time troubleshooting people's python installations to get traits to work. That is, in general it can be tricky to get installed because of all the dependencies. Maybe this has improved recently with Enthought's Canopy (or other new python distros), but that's been my past experience. > > More generally, the view in the astropy core package is that we don't want to put GUIs in the core because GUIs always carry lots of dependencies, which we don't want to be forced to deal with. But part of the whole reason for affiliated packages was to get around this, so we're happy to see GUI-based affiliated packages. To expand on the GUI issue a bit. I certainly see the benefit of an interactive GUI for these tools. But they raise a number of issues that the authors should think about. 1) They require one to select a windowing system. It would be nice if we standardized on one. Two obvious (but not the only) candidates are Qt and Tkinter. Qt is much more modern, but as far as I can see still presents significant installation issues for some. Tkinter is long in the tooth, but installation issues are generally much fewer. Yes, there's Wx, Gtk, and others, not to mention the whole browser interface issue, which ought to be considered as well. 2) I've seen the danger of someone starting right off with a GUI framework for their spectral (or any other kind of tool), and making their computational functionality entirely intertwined with it. I think that is a mistake. Eventually, many would like to script a lot of that functionality, or use it in other contexts. It's really important to keep the core computational stuff independent of the GUI. 3) GUI's are expensive to do well, and expensive to make changes to. For that reason I tend to favor the no-GUI approach for the initial functionality as much as possible, and limit the interactions to simpler tools (or simple GUI) until all the pieces are ready for integration into a full GUI interface. It helps accomplish 2) as well, and gives some experience before too much is invested in a GUI design. So even if a GUI is needed, it's quite likely a lot of the underlying machinery should go into the core, or at least into a non-GUI package. Perry From adrianmpw at gmail.com Thu Jun 20 15:20:59 2013 From: adrianmpw at gmail.com (Adrian Price-Whelan) Date: Thu, 20 Jun 2013 15:20:59 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: I'm totally lost on what thread I'm responding to, but +100 to what Perry said about GUIs! IMHO there is plenty to keep us busy working on the modeling and backend, and we should focus our efforts on making that code super slick and useable *first*, then worry about a GUI (which, also IMO, are overrated! why do you want to click on your absorption lines anyways??). On Thu, Jun 20, 2013 at 3:09 PM, Perry Greenfield wrote: > > On Jun 20, 2013, at 12:34 PM, Erik Tollerud wrote: > >> I'm of mixed minds about traits UI because once you know it you can make great GUIs with it, but I've spent a lot of time troubleshooting people's python installations to get traits to work. That is, in general it can be tricky to get installed because of all the dependencies. Maybe this has improved recently with Enthought's Canopy (or other new python distros), but that's been my past experience. >> >> More generally, the view in the astropy core package is that we don't want to put GUIs in the core because GUIs always carry lots of dependencies, which we don't want to be forced to deal with. But part of the whole reason for affiliated packages was to get around this, so we're happy to see GUI-based affiliated packages. > > To expand on the GUI issue a bit. I certainly see the benefit of an interactive GUI for these tools. But they raise a number of issues that the authors should think about. > > 1) They require one to select a windowing system. It would be nice if we standardized on one. Two obvious (but not the only) candidates are Qt and Tkinter. Qt is much more modern, but as far as I can see still presents significant installation issues for some. Tkinter is long in the tooth, but installation issues are generally much fewer. Yes, there's Wx, Gtk, and others, not to mention the whole browser interface issue, which ought to be considered as well. > > 2) I've seen the danger of someone starting right off with a GUI framework for their spectral (or any other kind of tool), and making their computational functionality entirely intertwined with it. I think that is a mistake. Eventually, many would like to script a lot of that functionality, or use it in other contexts. It's really important to keep the core computational stuff independent of the GUI. > > 3) GUI's are expensive to do well, and expensive to make changes to. For that reason I tend to favor the no-GUI approach for the initial functionality as much as possible, and limit the interactions to simpler tools (or simple GUI) until all the pieces are ready for integration into a full GUI interface. It helps accomplish 2) as well, and gives some experience before too much is invested in a GUI design. > > So even if a GUI is needed, it's quite likely a lot of the underlying machinery should go into the core, or at least into a non-GUI package. > > Perry > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw From beaumont at hawaii.edu Thu Jun 20 15:51:41 2013 From: beaumont at hawaii.edu (Chris Beaumont) Date: Thu, 20 Jun 2013 15:51:41 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: I'm more optimistic about GUIs than Adrian (who I bet is reading this email with a GUI!), but agree that computation and presentation logic should have sharp boundaries between them -- it makes developing both much cleaner. Another option to consider is to what extent astropy GUIs can revolve around javascript and the IPython notebook. Installation is much easier, and this is the python workflow of the future. Unfortunately, matplotlib in the notebook is still mostly non-interactive, but I have to think the MPL people are working on this (another reason to delay interface design!) -------------- next part -------------- An HTML attachment was scrubbed... URL: From hodge at stsci.edu Thu Jun 20 15:52:55 2013 From: hodge at stsci.edu (Phil Hodge) Date: Thu, 20 Jun 2013 15:52:55 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <51C35D97.9040203@stsci.edu> On 06/20/2013 03:20 PM, Adrian Price-Whelan wrote: > (which, also IMO, are overrated! why do you want to click on your > absorption lines anyways??). To get information into a Python list or dictionary, so I can pass it to that super slick and useable code without further reliance on the GUI. Phil From adrianmpw at gmail.com Thu Jun 20 15:58:48 2013 From: adrianmpw at gmail.com (Adrian Price-Whelan) Date: Thu, 20 Jun 2013 15:58:48 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: Wrong -- I'm using Pine! Just kidding...you're right, that came off wrong, I indeed love GUI's but feel like the existing options don't add much when doing data analysis with spectra (e.g. line measurements). Exploring and viewing spectra is a separate issue -- I *do* think it's nice to be able to zoom, pan, resample, etc. on the fly. Anyway, I am interested in the prospect of web front-ending, but also still skeptical of javascript as a long-term solution... On Thu, Jun 20, 2013 at 3:51 PM, Chris Beaumont wrote: > I'm more optimistic about GUIs than Adrian (who I bet is reading this email > with a GUI!), but agree that computation and presentation logic should have > sharp boundaries between them -- it makes developing both much cleaner. > > Another option to consider is to what extent astropy GUIs can revolve around > javascript and the IPython notebook. Installation is much easier, and this > is the python workflow of the future. Unfortunately, matplotlib in the > notebook is still mostly non-interactive, but I have to think the MPL people > are working on this (another reason to delay interface design!) -- Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw From wkerzendorf at gmail.com Thu Jun 20 16:01:20 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Thu, 20 Jun 2013 16:01:20 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <69FCA857-09B3-4FC4-B64E-5EB84C5F84E7@gmail.com> I agree with the statement that we should very much separate the computation from the GUI. However GUIs are important tools for analysis and we should have at least a little bit of an idea what to recommend for affiliated packages. One of the projects in this SOCIS application is calling for a GUI and I think if we get a student on this, it would be interesting to get him to do a review of what's out there and maybe help us choose one or at least give us a better feel for the possibilities (I think the browser is the future, but I also don't quite know how to utilize that yet). Cheers, Wolfgang On 2013-06-20, at 3:51 PM, Chris Beaumont wrote: > I'm more optimistic about GUIs than Adrian (who I bet is reading this email with a GUI!), but agree that computation and presentation logic should have sharp boundaries between them -- it makes developing both much cleaner. > > Another option to consider is to what extent astropy GUIs can revolve around javascript and the IPython notebook. Installation is much easier, and this is the python workflow of the future. Unfortunately, matplotlib in the notebook is still mostly non-interactive, but I have to think the MPL people are working on this (another reason to delay interface design!) > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From aldcroft at head.cfa.harvard.edu Thu Jun 20 16:02:33 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 20 Jun 2013 16:02:33 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: On Thu, Jun 20, 2013 at 3:20 PM, Adrian Price-Whelan wrote: > I'm totally lost on what thread I'm responding to, but +100 to what > Perry said about GUIs! IMHO there is plenty to keep us busy working on > the modeling and backend, and we should focus our efforts on making > that code super slick and useable *first*, then worry about a GUI > (which, also IMO, are overrated! why do you want to click on your > absorption lines anyways??). > I understand you are being a bit sarcastic, but I did half my thesis on QSO absorption lines and I'll tell you that being able to click on those lines was a lifesaver. I wrote a Fortran / PGPLOT-driven GUI tool for analyzing absorption lines which people were still using 10 years later. Suffice to say that GUI's are useful, but I fully agree with comments regarding the separation of computation from interface. - Tom > > On Thu, Jun 20, 2013 at 3:09 PM, Perry Greenfield > wrote: > > > > On Jun 20, 2013, at 12:34 PM, Erik Tollerud wrote: > > > >> I'm of mixed minds about traits UI because once you know it you can > make great GUIs with it, but I've spent a lot of time troubleshooting > people's python installations to get traits to work. That is, in general > it can be tricky to get installed because of all the dependencies. Maybe > this has improved recently with Enthought's Canopy (or other new python > distros), but that's been my past experience. > >> > >> More generally, the view in the astropy core package is that we don't > want to put GUIs in the core because GUIs always carry lots of > dependencies, which we don't want to be forced to deal with. But part of > the whole reason for affiliated packages was to get around this, so we're > happy to see GUI-based affiliated packages. > > > > To expand on the GUI issue a bit. I certainly see the benefit of an > interactive GUI for these tools. But they raise a number of issues that the > authors should think about. > > > > 1) They require one to select a windowing system. It would be nice if we > standardized on one. Two obvious (but not the only) candidates are Qt and > Tkinter. Qt is much more modern, but as far as I can see still presents > significant installation issues for some. Tkinter is long in the tooth, but > installation issues are generally much fewer. Yes, there's Wx, Gtk, and > others, not to mention the whole browser interface issue, which ought to be > considered as well. > > > > 2) I've seen the danger of someone starting right off with a GUI > framework for their spectral (or any other kind of tool), and making their > computational functionality entirely intertwined with it. I think that is a > mistake. Eventually, many would like to script a lot of that functionality, > or use it in other contexts. It's really important to keep the core > computational stuff independent of the GUI. > > > > 3) GUI's are expensive to do well, and expensive to make changes to. For > that reason I tend to favor the no-GUI approach for the initial > functionality as much as possible, and limit the interactions to simpler > tools (or simple GUI) until all the pieces are ready for integration into a > full GUI interface. It helps accomplish 2) as well, and gives some > experience before too much is invested in a GUI design. > > > > So even if a GUI is needed, it's quite likely a lot of the underlying > machinery should go into the core, or at least into a non-GUI package. > > > > Perry > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Thu Jun 20 16:53:08 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Thu, 20 Jun 2013 16:53:08 -0400 Subject: [AstroPy] AstroPy Digest, Vol 817676788676, Issue 1263429639647 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <6DDDB090-6052-410F-87FD-BD82F3C6B2B2@gmail.com> On Jun 20, 2013, at 4:02 PM, Aldcroft, Thomas wrote: > > > > On Thu, Jun 20, 2013 at 3:20 PM, Adrian Price-Whelan wrote: > I'm totally lost on what thread I'm responding to, but +100 to what > Perry said about GUIs! IMHO there is plenty to keep us busy working on > the modeling and backend, and we should focus our efforts on making > that code super slick and useable *first*, then worry about a GUI > (which, also IMO, are overrated! why do you want to click on your > absorption lines anyways??). > > I understand you are being a bit sarcastic, but I did half my thesis on QSO absorption lines and I'll tell you that being able to click on those lines was a lifesaver. I wrote a Fortran / PGPLOT-driven GUI tool for analyzing absorption lines which people were still using 10 years later. Suffice to say that GUI's are useful, but I fully agree with comments regarding the separation of computation from interface. Don't get me wrong, interaction is very important for many tasks, but initially sometimes a simpler tool can get most of the important interaction done before investing in a big integrated functionality GUI (and perhaps a few different simple GUI tools would be better than one trying to do it all, at least to start). Doing the interactions in the browser may be closer than we realize also. Perry From thoger.emil at gmail.com Thu Jun 20 17:08:12 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Thu, 20 Jun 2013 23:08:12 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <51C36F3C.7040906@gmail.com> My reason for writing it was especially that it looks like `astropy.tables` does a lot of effort duplication by reinventing functionality that is already in Pandas.It may be that `astropy.tables` already has alevel of maturity and functionality which means it really isn't worth it to start tinkering with a package like pandas, I don't know. What I do know is that Pandas has been immensely helpful for my particular, personal use, which is as an extended record array. The killer feature, in my view, is the ability to store heterogeneous data with integer- or label-based, possibly hierarchical, indexing. In my case, where my model consists of amplitude, fwhm, centroid parameter sets of multiple components of multiple transitions for multiple spatial regions of multiple exposures and /keeping track of which ones belong to the same physical subsystem/, Pandas is indispensable. It allows me to sort, regroup, etc. by a multitude of criteria. "Show me the amplitudes of all the components 'c' of the 10th through 21st spatial row of the Balmer lines" is a one-liner. The slicing-and-dicing capabilities are quite handy, also for someone who's never used R or done any statistical work worth mentioning. It is true that TraitsUI (and Chaco, by the way), have had some stability issues, and unfortunately still have. I've found that the MacPorts packages work fine. But no, they are not as mature and thoroughly tested as e.g. Matplotlib. So if it were only for the plotting capabilities, the choive of matplotlib would be a no-brainer. But as we have both experienced, traitsui saves you a *lot* of coding time when building a simpler GUI. I honestly don't know if it is a good choice of basis for an affiliated package, all things considered, but I completely agree that it should stay well outside of core. On 20-06-2013 18:34, Erik Tollerud wrote: > I'm of mixed minds about traits UI because once you know it you can > make great GUIs with it, but I've spent a lot of time troubleshooting > people's python installations to get traits to work. That is, in > general it can be tricky to get installed because of all the > dependencies. Maybe this has improved recently with Enthought's > Canopy (or other new python distros), but that's been my past experience. > > More generally, the view in the astropy core package is that we don't > want to put GUIs in the core because GUIs always carry lots of > dependencies, which we don't want to be forced to deal with. But part > of the whole reason for affiliated packages was to get around this, so > we're happy to see GUI-based affiliated packages. > > > As for Pandas, to be totally honest, I don't see a huge amount to be > gained from adding a Pandas dependency Astropy. It's honestly not > clear what it gives the astronomy community that numpy does not > already have. The following quote from the Pandas web site has guided > me to that conclusion: "/pandas/ helps fill this gap, enabling you to > carry out your entire data analysis workflow in Python without having > to switch to a more domain specific language like R." > > I have been carrying out my entire data analysis workflow for some > time now in python without using Pandas. It looks to me like Pandas > is a tool that was written by and for statisticians who use R. While > we can take lessons from this, it's not clear we get much out of it in > an astronomy context. For example, how would it make astropy's NDData, > Quantity, or Table better to use a Pandas DataFrame vs. a numpy array? > Most of what we are doing is building astronomy-convenient interfaces, > and I'm not sure what Pandas adds there, at the cost of a pretty > heavy-weight dependency. > > It could just be that I don't know enough about Pandas, though. So if > someone who knows Pandas better can speak to this, I'm all ears. > > > > On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen > > wrote: > > Pandas is a part of the newly-defined SciPy stack, after all, so > that would be part of any science-oriented distribution worth its > salt. In fact, I think it could be a good idea for astropy in > general to use under the hood, but again, could clash with the > philosophy of the project and possibly also maintainabillity. > > As for offering my code or just my experience, I'll have to square > it with my supervisor first, and I also think it depends on what > direction the project in question will take. I'm positive about > the idea (which is why I wrote in the first place), but supervisor > might think it is a better idea to actually get my paper in the > project wrapped up before sending the code out there. Will get > back about that one! > > /Emil > > > > > > On 2013-06-18 20:53, Slavin, Jonathan wrote: >> Hi Emil, >> >> That looks very nice! I don't see Pandas as a big issue in terms >> of dependencies. I don't know that much about traits, etc. My >> thought about the gui was just based on my experience with >> matplotlib, and the fact that it is widely used -- though I would >> agree that too many dependencies can be a deterrent to people >> using something. Are you offering your code as a starting point >> for the project? It strikes me that many have gotten some sort >> of fitting package to a point of personal usability but no one >> has the time/interest/motivation to make a more generally usable >> package. >> >> Jon >> >> On Tue, Jun 18, 2013 at 2:34 PM, > > wrote: >> >> Date: Tue, 18 Jun 2013 20:39:55 +0200 >> From: Th?ger Rivera-Thorsen > > >> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy at scipy.org >> Message-ID: <51C0A97B.8090703 at gmail.com >> > >> Content-Type: text/plain; charset="iso-8859-1" >> >> I have been working on a fitting GUI for a while, although it >> is made >> with a specific task in mind. >> However, it is not based on Matplotlib but on >> Traits/Traitsui/Chaco and >> Pandas. It is made for a specific projhect I'm working and as >> such not >> yet usable for more general cases, but it could be a starting >> point, if >> the dependencies don't conflict with astropy politics. >> >> Especially, I am happy about the choice of Pandas for >> managing a quite >> complex data structure (the fitted and/or guessed values of >> an arbitrary >> number of transitions for an arbitrary number of rows or >> collapsed rows >> of a spatially resolved spectrum) of a), but also with the >> Traits-based >> interactive interface to build complex line profiles from single >> gaussians, good for fitting-by-eye and giving good initial >> guesses for >> fitting of complex line profiles. It hooks directly up to a >> wrapper I've >> made for lmfit, but given the modularity, it should be >> relatively easy >> to change to other backends. >> >> It's still a work-in-progress, but there are some screenshots >> here: >> http://flic.kr/s/aHsjGaEMGg . >> I know the choice and number of dependencies may be >> prohibitive but it >> saved a lot of work on the GUI, and Pandas means the >> difference between >> sanity and madness when it comes to keeping track of so many >> parameters. >> >> Cheers, >> Emil >> >> >> >> >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 >> Garden Street, MS 83 >> phone: (617) 496-7981 Cambridge, >> MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Erik > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoger.emil at gmail.com Thu Jun 20 17:24:14 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Thu, 20 Jun 2013 23:24:14 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: References: Message-ID: <51C372FE.8090501@gmail.com> On 20-06-2013 18:27, Erik Tollerud wrote: > I want to issue a generic plea on this issue: lets do our best to join > forces. I think everyone has agreed that the specutils should be an affiliated package, resting as heavily as possible on astropy core. > There are already a number of tools that do part of spectrum > plotting and line-fitting, and if we'd all work together, with less > effort we'd have a tool that does all of those and more. That's the > main point of Astropy, after all: encouraging collobaration on and > re-use of python tools for astronomy. I'm partly guilty here, as I > have a similar fitting tool in astropysics - > http://pythonhosted.org/Astropysics/gui/spylot.html, but I'd rather > encourage people to use a tool that we can all get behind. I'll call > this the "Astropy spirit" :) I have tried spylot myself and liked it, but it didn't match my needs and I found it easier to code my own thing than modify yours, which I actually considered. But I think that the idea has resonated so well is exactly that many of us have made an effort to create something, but kinda-sorta only made it halfway to something generally usable. That gives us a lot of design ideas floating around and a lot of coding and designing experience to exchange, but I believe the wise thing to do would be to build something simple but modular and extensible that integrates well with and makes strong use of astropy core. So I believe we are actually on the same page here. > That said, I see the virtue of designing a new tool (ideally an > astropy affiliated package) to combine the shared wisdom (perhaps with > SOCIS, if it goes through). But the *worst* thing to have happen is to > have the other efforts continue in parallel, with everyone designing > similar but slightly incompatible tools. We shall not disagree about that point, either! > Along similar lines, Emil and Adam (although I think Adam already > knows about it), I'd *strongly* encourage you to take a look at the > astropy modeling framework > (http://docs.astropy.org/en/latest/modeling/index.html) - this will be > in the next major release, and I'd hate to see you re-implementing all > of that in an incompatible way. Or, if you don't like something about > how that works for your needs, please help us make it better! I am more discussing design and not going to be much of a coding work horse, as my summer and fall is quite swamped - but again, I completely agree and it is also my impression that this was the general consensus between anyone who's shown interest in the project. I am, however, going to continue working on my own personal app, since it is highly specialized and I am tayloring it for a specific science project and I think chances are low that astropy will be usable for that within the time frame I need. But while writing it, there are also some more common functionality that I have written into it that could possibly be useful for an astropy affiliated package - e.g. a Traits GUI for interactively building compound lie profiles. Cheers, Emil > On Thu, Jun 20, 2013 at 8:42 AM, Slavin, Jonathan > wrote: >> Hi Adam, >> >> I have used pyspeckit. There are parts of it that are very useful, but I >> found overall that it didn't do what I wanted. For example, I wanted to >> give it my spectrum, have it display it, allow me to manually (in the gui) >> place my guesses for line centers (would probably need guesses for widths as >> well) and then fit the lines (and continuum) and return the fitted line >> centers, widths, and amplitudes, etc. along with error estimates. It's been >> a while now since I used it, but as I recall, pyspeckit did not do all of >> those things. There may have been other reasons as well, but in the end I >> wrote my own fitting routines to work with the data. Am I wrong about the >> capabilities of pyspeckit? >> >> Jon >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >> phone: (617) 496-7981 Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> On Wed, Jun 19, 2013 at 5:21 PM, wrote: >>> Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) >>> From: Adam Ginsburg >>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>> To: astropy-dev at googlegroups.com >>> Cc: astropy at scipy.org >>> Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5 at googlegroups.com> >>> Content-Type: text/plain; charset="utf-8" >>> >>> >>> Emil and Jon - there is a package that fits exactly those requirements, >>> called pyspeckit: >>> pyspeckit.bitbucket.org >>> >>> One of my major goals over the next year is to incorporate it into astropy >>> bit by bit, i.e. make it work with astropy.models, astropy.units, and >>> specutils. Only a little progress in that direction has been made (in >>> part >>> because my coding efforts have been focused more on astroquery). >>> >>> Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. >>> You can check that it agrees with yours: >>> >>> http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt >>> >>> I'm bringing this up on this particular thread because I had considered >>> submitting pyspeckit as a SOCIS project, but I think focusing on specutils >>> instead will be better for the community. Nonetheless, there are some >>> specific project ideas in place that may help support the case for a >>> student working on specutils via SOCIS: >>> pyspeckit.bitbucket.org/html/sphinx/projects.html >>> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > From keflavich at gmail.com Thu Jun 20 17:26:04 2013 From: keflavich at gmail.com (Adam) Date: Thu, 20 Jun 2013 15:26:04 -0600 Subject: [AstroPy] specutils, pyspeckit, etc. Message-ID: (apologies for the subject and my delay in replying, I have been receiving astropy-users e-mails in digest form and have missed nearly everything. So this e-mail is also a vote to move to astropy-users at google) As Erik said, it's best to merge these parallel efforts and try to develop one common package with a nice interface. My vision for the future of pyspeckit is essentially to have it be a wrapper of specutils + astropy.models + astropy.units, and in fact my current coding effort is mostly going in that direction (see, e.g., spectroscopic equivalencies in astropy). I wanted to raise it as an example of something that works *right now*, but doesn't work perfectly (as Jonathan rightly pointed out). Most of the models implemented in pyspeckit should be very easy to re-write as astropy.models... models. That goal, in particular, is a big component of one of the specutils SOCIS proposed projects. Also, pyspeckit's frontend is matplotlib-only, which is slow but compatible with everything. Because it's matplotlib-only, and matplotlib doesn't have a ton of GUI features (more than you'd think, fewer than you need), the GUI side is very IRAF-splot-like. Re: Jonathan - pyspeckit does all those things. If you want them interactively, have a look at this example: pyspeckit.readthedocs.org/en/latest/interactive.html That said, the GUI interface really is not the best - it's not what I want, but it is what is possible with matplotlib right now. A summer-of-code student could, I think, turn my half-assed GUI into a real one (for specutils). If anything in particular is missing that you *really* want, though, I'm happy to try to support it in pyspeckit now, with the idea of porting it to specutils once things come together. Re: Kelle - I think pyspeckit can do anything splot can at this point, though I'm not so sure about spectral indices. Ratios should be easy... I think there is a lot to be learned from pyspeckit (and Erik's pymodelfit/astropysics) when developing specutils, some of which will just be "lessons learned" (GUI features?) but other components that can really be reused code (especially the models). -- Adam From keflavich at gmail.com Thu Jun 20 17:38:20 2013 From: keflavich at gmail.com (Adam) Date: Thu, 20 Jun 2013 15:38:20 -0600 Subject: [AstroPy] Specutils / Duplication of Effort Message-ID: Regarding Erik and Emil's last e-mails, I've made a wiki page on specutils to track possible duplication of effort so we can get a clearer idea of what tools we can cannibalize / learn from in order to satisfy everyone's long list of feature requests: https://github.com/astropy/specutils/wiki/Specutils---Other-Packages I put up only what I know of right now (spylot, pyspeckit, Emil's composite-model builder). I think all of these packages include some overlap but some unique facilities that can and should be included in specutils. Erik, Emil, and others developing along these lines, would you mind helping fill out this page? Thanks, Adam From thoger.emil at gmail.com Thu Jun 20 17:42:14 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Thu, 20 Jun 2013 23:42:14 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <51C37736.8010705@gmail.com> [*Snip*] > Group-by and related functionality is top on my list of priorities for > astropy.table (in fact I see it every day on my google keep app...). > Join and merging are in master now. In my tests the astropy table > join is within a factor of 2 to 3 in speed relative to pandas, so in > most use cases it should be good enough. Join and merge would follow the pandas behaviour? Because that is one of its major assets, I believe - its elegant handling of missing data and misaligned indices etc. Speed is a minor issue in my world, we're not often working with the data set sizes the quants are. > It's probably worth pointing out to the community that it was not a > lightly-taken decision to reject pandas for use as the base data > storage container. For the case of tables there is one show-stopper > which is that pandas DataFrame does not support arbitrary > multi-dimensional columns, i.e. column where each element is itself an > N-d array. These occur enough in astronomy and are supported by FITS > and VO standards, so the astropy Table must be able to represent that. > The lack of support for table and column metadata is a smaller but > still important issue. > That is very interesting to hear. I had, in fact, been wondering what was the rationale, but it certainly makes sense. I was under the impression, though, that pandas pretty much supported arbitrary objects as entries in its data structures. But I was apparently mistaken? > Having said that, there is no question pandas has a ton of > highly-efficient and useful machinery and we are working on ways to > improve inter-operability. This includes being able convert between > Table and DataFrame easily. Suggestions and (especially) pull > requests welcome. An easy and near-seamless conversion between a DataFrame and an astropy table (as long as data type support allows for it, of course) would definitely be a great thing. > > I won't speculate about whether that's enough an asset to warrant > a dependency in astropy. I do agree that lots of other pandas > features don't translate as well into astronomy use. > > > > On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud > > wrote: > > I'm of mixed minds about traits UI because once you know it > you can make great GUIs with it, but I've spent a lot of time > troubleshooting people's python installations to get traits to > work. That is, in general it can be tricky to get installed > because of all the dependencies. Maybe this has improved > recently with Enthought's Canopy (or other new python > distros), but that's been my past experience. > > More generally, the view in the astropy core package is that > we don't want to put GUIs in the core because GUIs always > carry lots of dependencies, which we don't want to be forced > to deal with. But part of the whole reason for affiliated > packages was to get around this, so we're happy to see > GUI-based affiliated packages. > > > As for Pandas, to be totally honest, I don't see a huge amount > to be gained from adding a Pandas dependency Astropy. It's > honestly not clear what it gives the astronomy community that > numpy does not already have. The following quote from the > Pandas web site has guided me to that conclusion: > "/pandas/ helps fill this gap, enabling you to carry out your > entire data analysis workflow in Python without having to > switch to a more domain specific language like R." > > I have been carrying out my entire data analysis workflow for > some time now in python without using Pandas. It looks to me > like Pandas is a tool that was written by and > for statisticians who use R. While we can take lessons from > this, it's not clear we get much out of it in an astronomy > context. For example, how would it make astropy's NDData, > Quantity, or Table better to use a Pandas DataFrame vs. a > numpy array? Most of what we are doing is > building astronomy-convenient interfaces, and I'm not sure > what Pandas adds there, at the cost of a pretty heavy-weight > dependency. > > It could just be that I don't know enough about Pandas, > though. So if someone who knows Pandas better can speak to > this, I'm all ears. > > > > On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen > > wrote: > > Pandas is a part of the newly-defined SciPy stack, after > all, so that would be part of any science-oriented > distribution worth its salt. In fact, I think it could be > a good idea for astropy in general to use under the hood, > but again, could clash with the philosophy of the project > and possibly also maintainabillity. > > As for offering my code or just my experience, I'll have > to square it with my supervisor first, and I also think it > depends on what direction the project in question will > take. I'm positive about the idea (which is why I wrote in > the first place), but supervisor might think it is a > better idea to actually get my paper in the project > wrapped up before sending the code out there. Will get > back about that one! > > /Emil > > > > > > On 2013-06-18 20:53, Slavin, Jonathan wrote: >> Hi Emil, >> >> That looks very nice! I don't see Pandas as a big issue >> in terms of dependencies. I don't know that much about >> traits, etc. My thought about the gui was just based on >> my experience with matplotlib, and the fact that it is >> widely used -- though I would agree that too many >> dependencies can be a deterrent to people using >> something. Are you offering your code as a starting >> point for the project? It strikes me that many have >> gotten some sort of fitting package to a point of >> personal usability but no one has the >> time/interest/motivation to make a more generally usable >> package. >> >> Jon >> >> On Tue, Jun 18, 2013 at 2:34 PM, >> > > wrote: >> >> Date: Tue, 18 Jun 2013 20:39:55 +0200 >> From: Th?ger Rivera-Thorsen > > >> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >> To: astropy at scipy.org >> Message-ID: <51C0A97B.8090703 at gmail.com >> > >> Content-Type: text/plain; charset="iso-8859-1" >> >> I have been working on a fitting GUI for a while, >> although it is made >> with a specific task in mind. >> However, it is not based on Matplotlib but on >> Traits/Traitsui/Chaco and >> Pandas. It is made for a specific projhect I'm >> working and as such not >> yet usable for more general cases, but it could be a >> starting point, if >> the dependencies don't conflict with astropy politics. >> >> Especially, I am happy about the choice of Pandas for >> managing a quite >> complex data structure (the fitted and/or guessed >> values of an arbitrary >> number of transitions for an arbitrary number of rows >> or collapsed rows >> of a spatially resolved spectrum) of a), but also >> with the Traits-based >> interactive interface to build complex line profiles >> from single >> gaussians, good for fitting-by-eye and giving good >> initial guesses for >> fitting of complex line profiles. It hooks directly >> up to a wrapper I've >> made for lmfit, but given the modularity, it should >> be relatively easy >> to change to other backends. >> >> It's still a work-in-progress, but there are some >> screenshots here: >> http://flic.kr/s/aHsjGaEMGg . >> I know the choice and number of dependencies may be >> prohibitive but it >> saved a lot of work on the GUI, and Pandas means the >> difference between >> sanity and madness when it comes to keeping track of >> so many parameters. >> >> Cheers, >> Emil >> >> >> >> >> ________________________________________________________ >> Jonathan D. Slavin Harvard-Smithsonian CfA >> jslavin at cfa.harvard.edu >> 60 Garden Street, MS 83 >> phone: (617) 496-7981 >> Cambridge, MA 02138-1516 >> fax: (617) 496-7577 USA >> ________________________________________________________ >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Erik > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > ************************************ > Chris Beaumont > Graduate Student > Institute for Astronomy > University of Hawaii at Manoa > 2680 Woodlawn Drive > Honolulu, HI 96822 > www.ifa.hawaii.edu/~beaumont > ************************************ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From stoeckel.matt at gmail.com Thu Jun 20 17:51:22 2013 From: stoeckel.matt at gmail.com (Matt Stoeckel) Date: Thu, 20 Jun 2013 16:51:22 -0500 Subject: [AstroPy] unsubscribe In-Reply-To: References: Message-ID: <51C3795A.1070107@gmail.com> On 6/20/13 4:37 PM, astropy-request at scipy.org wrote: > Send AstroPy mailing list submissions to > astropy at scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/astropy > or, via email, send a message with subject or body 'help' to > astropy-request at scipy.org > > You can reach the person managing the list at > astropy-owner at scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of AstroPy digest..." > > > Today's Topics: > > 1. Re: ESA Summer of Code in Space 2013 (Th?ger Emil Rivera-Thorsen) > 2. Re: specutils, pyspeckit, etc. (Adam) > 3. Specutils / Duplication of Effort (Adam) > 4. Re: AstroPy Digest, Vol 81, Issue 12 (Th?ger Emil Rivera-Thorsen) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Jun 2013 23:24:14 +0200 > From: Th?ger Emil Rivera-Thorsen > Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 > To: astropy at scipy.org > Message-ID: <51C372FE.8090501 at gmail.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 20-06-2013 18:27, Erik Tollerud wrote: >> I want to issue a generic plea on this issue: lets do our best to join >> forces. > I think everyone has agreed that the specutils should be an affiliated > package, resting as heavily as possible on astropy core. > >> There are already a number of tools that do part of spectrum >> plotting and line-fitting, and if we'd all work together, with less >> effort we'd have a tool that does all of those and more. That's the >> main point of Astropy, after all: encouraging collobaration on and >> re-use of python tools for astronomy. I'm partly guilty here, as I >> have a similar fitting tool in astropysics - >> http://pythonhosted.org/Astropysics/gui/spylot.html, but I'd rather >> encourage people to use a tool that we can all get behind. I'll call >> this the "Astropy spirit" :) > I have tried spylot myself and liked it, but it didn't match my needs > and I found it easier to code my own thing than modify yours, which I > actually considered. But I think that the idea has resonated so well is > exactly that many of us have made an effort to create something, but > kinda-sorta only made it halfway to something generally usable. That > gives us a lot of design ideas floating around and a lot of coding and > designing experience to exchange, but I believe the wise thing to do > would be to build something simple but modular and extensible that > integrates well with and makes strong use of astropy core. So I believe > we are actually on the same page here. > > >> That said, I see the virtue of designing a new tool (ideally an >> astropy affiliated package) to combine the shared wisdom (perhaps with >> SOCIS, if it goes through). But the *worst* thing to have happen is to >> have the other efforts continue in parallel, with everyone designing >> similar but slightly incompatible tools. > We shall not disagree about that point, either! > >> Along similar lines, Emil and Adam (although I think Adam already >> knows about it), I'd *strongly* encourage you to take a look at the >> astropy modeling framework >> (http://docs.astropy.org/en/latest/modeling/index.html) - this will be >> in the next major release, and I'd hate to see you re-implementing all >> of that in an incompatible way. Or, if you don't like something about >> how that works for your needs, please help us make it better! > I am more discussing design and not going to be much of a coding work > horse, as my summer and fall is quite swamped - but again, I completely > agree and it is also my impression that this was the general consensus > between anyone who's shown interest in the project. > > I am, however, going to continue working on my own personal app, since > it is highly specialized and I am tayloring it for a specific science > project and I think chances are low that astropy will be usable for that > within the time frame I need. But while writing it, there are also some > more common functionality that I have written into it that could > possibly be useful for an astropy affiliated package - e.g. a Traits GUI > for interactively building compound lie profiles. > > Cheers, > Emil > >> On Thu, Jun 20, 2013 at 8:42 AM, Slavin, Jonathan >> wrote: >>> Hi Adam, >>> >>> I have used pyspeckit. There are parts of it that are very useful, but I >>> found overall that it didn't do what I wanted. For example, I wanted to >>> give it my spectrum, have it display it, allow me to manually (in the gui) >>> place my guesses for line centers (would probably need guesses for widths as >>> well) and then fit the lines (and continuum) and return the fitted line >>> centers, widths, and amplitudes, etc. along with error estimates. It's been >>> a while now since I used it, but as I recall, pyspeckit did not do all of >>> those things. There may have been other reasons as well, but in the end I >>> wrote my own fitting routines to work with the data. Am I wrong about the >>> capabilities of pyspeckit? >>> >>> Jon >>> ________________________________________________________ >>> Jonathan D. Slavin Harvard-Smithsonian CfA >>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >>> phone: (617) 496-7981 Cambridge, MA 02138-1516 >>> fax: (617) 496-7577 USA >>> ________________________________________________________ >>> >>> >>> >>> On Wed, Jun 19, 2013 at 5:21 PM, wrote: >>>> Date: Wed, 19 Jun 2013 13:32:31 -0700 (PDT) >>>> From: Adam Ginsburg >>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>>> To: astropy-dev at googlegroups.com >>>> Cc: astropy at scipy.org >>>> Message-ID: <44536fa7-b34e-4f05-941b-2cc05535bcc5 at googlegroups.com> >>>> Content-Type: text/plain; charset="utf-8" >>>> >>>> >>>> Emil and Jon - there is a package that fits exactly those requirements, >>>> called pyspeckit: >>>> pyspeckit.bitbucket.org >>>> >>>> One of my major goals over the next year is to incorporate it into astropy >>>> bit by bit, i.e. make it work with astropy.models, astropy.units, and >>>> specutils. Only a little progress in that direction has been made (in >>>> part >>>> because my coding efforts have been focused more on astroquery). >>>> >>>> Emil - there is a scipy-dependent voigt profile implemented in pyspeckit. >>>> You can check that it agrees with yours: >>>> >>>> http://pyspeckit.bitbucket.org/html/sphinx/models.html#pyspeckit.spectrum.models.inherited_voigtfitter.voigt >>>> >>>> I'm bringing this up on this particular thread because I had considered >>>> submitting pyspeckit as a SOCIS project, but I think focusing on specutils >>>> instead will be better for the community. Nonetheless, there are some >>>> specific project ideas in place that may help support the case for a >>>> student working on specutils via SOCIS: >>>> pyspeckit.bitbucket.org/html/sphinx/projects.html >>>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >> > > > ------------------------------ > > Message: 2 > Date: Thu, 20 Jun 2013 15:26:04 -0600 > From: Adam > Subject: Re: [AstroPy] specutils, pyspeckit, etc. > To: astropy at scipy.org > Message-ID: > > Content-Type: text/plain; charset=UTF-8 > > (apologies for the subject and my delay in replying, I have been > receiving astropy-users e-mails in digest form and have missed nearly > everything. So this e-mail is also a vote to move to > astropy-users at google) > > As Erik said, it's best to merge these parallel efforts and try to > develop one common package with a nice interface. My vision for the > future of pyspeckit is essentially to have it be a wrapper of > specutils + astropy.models + astropy.units, and in fact my current > coding effort is mostly going in that direction (see, e.g., > spectroscopic equivalencies in astropy). I wanted to raise it as an > example of something that works *right now*, but doesn't work > perfectly (as Jonathan rightly pointed out). Most of the models > implemented in pyspeckit should be very easy to re-write as > astropy.models... models. That goal, in particular, is a big > component of one of the specutils SOCIS proposed projects. > > Also, pyspeckit's frontend is matplotlib-only, which is slow but > compatible with everything. Because it's matplotlib-only, and > matplotlib doesn't have a ton of GUI features (more than you'd think, > fewer than you need), the GUI side is very IRAF-splot-like. > > Re: Jonathan - pyspeckit does all those things. If you want them > interactively, have a look at this example: > pyspeckit.readthedocs.org/en/latest/interactive.html > That said, the GUI interface really is not the best - it's not what I > want, but it is what is possible with matplotlib right now. A > summer-of-code student could, I think, turn my half-assed GUI into a > real one (for specutils). If anything in particular is missing that > you *really* want, though, I'm happy to try to support it in pyspeckit > now, with the idea of porting it to specutils once things come > together. > > Re: Kelle - I think pyspeckit can do anything splot can at this point, > though I'm not so sure about spectral indices. Ratios should be > easy... > > I think there is a lot to be learned from pyspeckit (and Erik's > pymodelfit/astropysics) when developing specutils, some of which will > just be "lessons learned" (GUI features?) but other components that > can really be reused code (especially the models). > From chanley at gmail.com Thu Jun 20 17:58:00 2013 From: chanley at gmail.com (Christopher Hanley) Date: Thu, 20 Jun 2013 17:58:00 -0400 Subject: [AstroPy] Specutils / Duplication of Effort In-Reply-To: References: Message-ID: Hi, I wasn't quite ready to announce this but given the current discussion I wanted to mention that we at STScI are planning to generalize and adapt pysynphot for use as part of the astropy universe. I've added a link to the project on the wiki page Adam has created. When I am slightly further along on the project that I am now I will be sending out a message to the astropy community describing our development roadmap and soliciting input on additional requirements. Cheers, Chris On Thu, Jun 20, 2013 at 5:38 PM, Adam wrote: > Regarding Erik and Emil's last e-mails, I've made a wiki page on > specutils to track possible duplication of effort so we can get a > clearer idea of what tools we can cannibalize / learn from in order to > satisfy everyone's long list of feature requests: > > https://github.com/astropy/specutils/wiki/Specutils---Other-Packages > > I put up only what I know of right now (spylot, pyspeckit, Emil's > composite-model builder). I think all of these packages include some > overlap but some unique facilities that can and should be included in > specutils. > > Erik, Emil, and others developing along these lines, would you mind > helping fill out this page? > > Thanks, > Adam > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoger.emil at gmail.com Thu Jun 20 17:59:57 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Thu, 20 Jun 2013 23:59:57 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <51C37B5D.8050103@gmail.com> On 20-06-2013 21:20, Adrian Price-Whelan wrote: > I'm totally lost on what thread I'm responding to, but +100 to what > Perry said about GUIs! IMHO there is plenty to keep us busy working on > the modeling and backend, and we should focus our efforts on making > that code super slick and useable *first*, then worry about a GUI > (which, also IMO, are overrated! why do you want to click on your > absorption lines anyways??). Most of our work doesn't require a GUI, but a few tasks can be helped immensely by a good one. One of these, but not the only one, is the case of having to give a good initial guess to a fitting package, if you have many lines and many components and many tied, frozen or thawed parameters. Something that is vitally important for writing a good GUI, IMO, is reproducibility, which means that pretty much all settings that one creates should be possible to write into a script or into the model or something, so one doesn't have to rely on one's shaky and coffee-trembling hands to find the same settings multiple times. /Emil > > On Thu, Jun 20, 2013 at 3:09 PM, Perry Greenfield wrote: >> On Jun 20, 2013, at 12:34 PM, Erik Tollerud wrote: >> >>> I'm of mixed minds about traits UI because once you know it you can make great GUIs with it, but I've spent a lot of time troubleshooting people's python installations to get traits to work. That is, in general it can be tricky to get installed because of all the dependencies. Maybe this has improved recently with Enthought's Canopy (or other new python distros), but that's been my past experience. >>> >>> More generally, the view in the astropy core package is that we don't want to put GUIs in the core because GUIs always carry lots of dependencies, which we don't want to be forced to deal with. But part of the whole reason for affiliated packages was to get around this, so we're happy to see GUI-based affiliated packages. >> To expand on the GUI issue a bit. I certainly see the benefit of an interactive GUI for these tools. But they raise a number of issues that the authors should think about. >> >> 1) They require one to select a windowing system. It would be nice if we standardized on one. Two obvious (but not the only) candidates are Qt and Tkinter. Qt is much more modern, but as far as I can see still presents significant installation issues for some. Tkinter is long in the tooth, but installation issues are generally much fewer. Yes, there's Wx, Gtk, and others, not to mention the whole browser interface issue, which ought to be considered as well. >> >> 2) I've seen the danger of someone starting right off with a GUI framework for their spectral (or any other kind of tool), and making their computational functionality entirely intertwined with it. I think that is a mistake. Eventually, many would like to script a lot of that functionality, or use it in other contexts. It's really important to keep the core computational stuff independent of the GUI. >> >> 3) GUI's are expensive to do well, and expensive to make changes to. For that reason I tend to favor the no-GUI approach for the initial functionality as much as possible, and limit the interactions to simpler tools (or simple GUI) until all the pieces are ready for integration into a full GUI interface. It helps accomplish 2) as well, and gives some experience before too much is invested in a GUI design. >> >> So even if a GUI is needed, it's quite likely a lot of the underlying machinery should go into the core, or at least into a non-GUI package. >> >> Perry >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > From thoger.emil at gmail.com Thu Jun 20 18:08:27 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Emil_Rivera-Thorsen?=) Date: Fri, 21 Jun 2013 00:08:27 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <51C37D5B.6050703@gmail.com> On 20-06-2013 22:02, Aldcroft, Thomas wrote: > > > > On Thu, Jun 20, 2013 at 3:20 PM, Adrian Price-Whelan > > wrote: > > I'm totally lost on what thread I'm responding to, but +100 to what > Perry said about GUIs! IMHO there is plenty to keep us busy working on > the modeling and backend, and we should focus our efforts on making > that code super slick and useable *first*, then worry about a GUI > (which, also IMO, are overrated! why do you want to click on your > absorption lines anyways??). > > > I understand you are being a bit sarcastic, but I did half my thesis > on QSO absorption lines and I'll tell you that being able to click on > those lines was a lifesaver. I wrote a Fortran / PGPLOT-driven GUI > tool for analyzing absorption lines which people were still using 10 > years later. Suffice to say that GUI's are useful, but I fully agree > with comments regarding the separation of computation from interface. > > - Tom Interesting - I did my Master's Thesis on QSO absorption lines, and used a fortran/pgplot based GUI for my modeling. VPfit, to be precise. That is also when I learned what a difference a good GUI can make: Joe Liske's VPguess interface to vpfit was a life saver for me. Would your work have happened to be on any of those two tools? It bears repeating again, of course, that any good GUI is characterized by the fact that any important setting an be saved and preferably manually edited in a configuration file and loaded at a later point. > > > On Thu, Jun 20, 2013 at 3:09 PM, Perry Greenfield > > wrote: > > > > On Jun 20, 2013, at 12:34 PM, Erik Tollerud wrote: > > > >> I'm of mixed minds about traits UI because once you know it you > can make great GUIs with it, but I've spent a lot of time > troubleshooting people's python installations to get traits to > work. That is, in general it can be tricky to get installed > because of all the dependencies. Maybe this has improved recently > with Enthought's Canopy (or other new python distros), but that's > been my past experience. > >> > >> More generally, the view in the astropy core package is that we > don't want to put GUIs in the core because GUIs always carry lots > of dependencies, which we don't want to be forced to deal with. > But part of the whole reason for affiliated packages was to get > around this, so we're happy to see GUI-based affiliated packages. > > > > To expand on the GUI issue a bit. I certainly see the benefit of > an interactive GUI for these tools. But they raise a number of > issues that the authors should think about. > > > > 1) They require one to select a windowing system. It would be > nice if we standardized on one. Two obvious (but not the only) > candidates are Qt and Tkinter. Qt is much more modern, but as far > as I can see still presents significant installation issues for > some. Tkinter is long in the tooth, but installation issues are > generally much fewer. Yes, there's Wx, Gtk, and others, not to > mention the whole browser interface issue, which ought to be > considered as well. > > > > 2) I've seen the danger of someone starting right off with a GUI > framework for their spectral (or any other kind of tool), and > making their computational functionality entirely intertwined with > it. I think that is a mistake. Eventually, many would like to > script a lot of that functionality, or use it in other contexts. > It's really important to keep the core computational stuff > independent of the GUI. > > > > 3) GUI's are expensive to do well, and expensive to make changes > to. For that reason I tend to favor the no-GUI approach for the > initial functionality as much as possible, and limit the > interactions to simpler tools (or simple GUI) until all the pieces > are ready for integration into a full GUI interface. It helps > accomplish 2) as well, and gives some experience before too much > is invested in a GUI design. > > > > So even if a GUI is needed, it's quite likely a lot of the > underlying machinery should go into the core, or at least into a > non-GUI package. > > > > Perry > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at physics.ucf.edu Thu Jun 20 19:49:36 2013 From: jh at physics.ucf.edu (Joe Harrington) Date: Fri, 21 Jun 2013 01:49:36 +0200 Subject: [AstroPy] ESA Summer of Code in Space 2013 In-Reply-To: (astropy-request@scipy.org) Message-ID: Th?ger Rivera-Thorsen writes: > Did you mean the specutils project here, or me personally? I meant the project as a whole, but if individuals want code, please ask. --jh-- From aldcroft at head.cfa.harvard.edu Thu Jun 20 23:22:05 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 20 Jun 2013 23:22:05 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <51C37736.8010705@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37736.8010705@gmail.com> Message-ID: On Thu, Jun 20, 2013 at 5:42 PM, Th?ger Emil Rivera-Thorsen < thoger.emil at gmail.com> wrote: > [*Snip*] > > > > > Group-by and related functionality is top on my list of priorities for > astropy.table (in fact I see it every day on my google keep app...). Join > and merging are in master now. In my tests the astropy table join is > within a factor of 2 to 3 in speed relative to pandas, so in most use cases > it should be good enough. > > Join and merge would follow the pandas behaviour? Because that is one of > its major assets, I believe - its elegant handling of missing data and > misaligned indices etc. > Speed is a minor issue in my world, we're not often working with the data > set sizes the quants are. > I was looking at the pandas docs when writing join, so the interface is inspired by, but not the same as, pandas. I believe that the left, right, inner, and outer joins are all correct and handle missing data correctly, but independent testing and feedback would be welcome! astropy.Table does not have the analogous concept of a pandas label, but any column or columns can be designated as the join key. See: http://astropy.readthedocs.org/en/latest/table/operations.html > > > It's probably worth pointing out to the community that it was not a > lightly-taken decision to reject pandas for use as the base data storage > container. For the case of tables there is one show-stopper which is that > pandas DataFrame does not support arbitrary multi-dimensional columns, i.e. > column where each element is itself an N-d array. These occur enough in > astronomy and are supported by FITS and VO standards, so the astropy Table > must be able to represent that. The lack of support for table and column > metadata is a smaller but still important issue. > > That is very interesting to hear. I had, in fact, been wondering what > was the rationale, but it certainly makes sense. > I was under the impression, though, that pandas pretty much supported > arbitrary objects as entries in its data structures. But I was apparently > mistaken? > Yes, pandas supports an object dtype, but that is not really useful for representing a multi-dimensional array since you basically lose all the numpy-ness of the array. A 1000000 x 2 column would be 1000000 numpy array objects = bad news. > > > Having said that, there is no question pandas has a ton of > highly-efficient and useful machinery and we are working on ways to improve > inter-operability. This includes being able convert between Table and > DataFrame easily. Suggestions and (especially) pull requests welcome. > > > An easy and near-seamless conversion between a DataFrame and an astropy > table (as long as data type support allows for it, of course) would > definitely be a great thing. > > > > >> >> I won't speculate about whether that's enough an asset to warrant a >> dependency in astropy. I do agree that lots of other pandas features don't >> translate as well into astronomy use. >> >> >> >> On Thu, Jun 20, 2013 at 12:34 PM, Erik Tollerud wrote: >> >>> I'm of mixed minds about traits UI because once you know it you can >>> make great GUIs with it, but I've spent a lot of time troubleshooting >>> people's python installations to get traits to work. That is, in general >>> it can be tricky to get installed because of all the dependencies. Maybe >>> this has improved recently with Enthought's Canopy (or other new python >>> distros), but that's been my past experience. >>> >>> More generally, the view in the astropy core package is that we don't >>> want to put GUIs in the core because GUIs always carry lots of >>> dependencies, which we don't want to be forced to deal with. But part of >>> the whole reason for affiliated packages was to get around this, so we're >>> happy to see GUI-based affiliated packages. >>> >>> >>> As for Pandas, to be totally honest, I don't see a huge amount to be >>> gained from adding a Pandas dependency Astropy. It's honestly not clear >>> what it gives the astronomy community that numpy does not already have. >>> The following quote from the Pandas web site has guided me to that >>> conclusion: "*pandas* helps fill this gap, enabling you to carry out >>> your entire data analysis workflow in Python without having to switch to a >>> more domain specific language like R." >>> >>> I have been carrying out my entire data analysis workflow for some >>> time now in python without using Pandas. It looks to me like Pandas is a >>> tool that was written by and for statisticians who use R. While we can >>> take lessons from this, it's not clear we get much out of it in an >>> astronomy context. For example, how would it make astropy's NDData, >>> Quantity, or Table better to use a Pandas DataFrame vs. a numpy array? Most >>> of what we are doing is building astronomy-convenient interfaces, and I'm >>> not sure what Pandas adds there, at the cost of a pretty heavy-weight >>> dependency. >>> >>> It could just be that I don't know enough about Pandas, though. So if >>> someone who knows Pandas better can speak to this, I'm all ears. >>> >>> >>> >>> >>> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen < >>> trive at astro.su.se> wrote: >>> >>>> Pandas is a part of the newly-defined SciPy stack, after all, so that >>>> would be part of any science-oriented distribution worth its salt. In fact, >>>> I think it could be a good idea for astropy in general to use under the >>>> hood, but again, could clash with the philosophy of the project and >>>> possibly also maintainabillity. >>>> >>>> As for offering my code or just my experience, I'll have to square it >>>> with my supervisor first, and I also think it depends on what direction the >>>> project in question will take. I'm positive about the idea (which is why I >>>> wrote in the first place), but supervisor might think it is a better idea >>>> to actually get my paper in the project wrapped up before sending the code >>>> out there. Will get back about that one! >>>> >>>> /Emil >>>> >>>> >>>> >>>> >>>> >>>> On 2013-06-18 20:53, Slavin, Jonathan wrote: >>>> >>>> Hi Emil, >>>> >>>> That looks very nice! I don't see Pandas as a big issue in terms of >>>> dependencies. I don't know that much about traits, etc. My thought about >>>> the gui was just based on my experience with matplotlib, and the fact that >>>> it is widely used -- though I would agree that too many dependencies can be >>>> a deterrent to people using something. Are you offering your code as a >>>> starting point for the project? It strikes me that many have gotten some >>>> sort of fitting package to a point of personal usability but no one has the >>>> time/interest/motivation to make a more generally usable package. >>>> >>>> Jon >>>> >>>> On Tue, Jun 18, 2013 at 2:34 PM, wrote: >>>> >>>>> Date: Tue, 18 Jun 2013 20:39:55 +0200 >>>>> From: Th?ger Rivera-Thorsen >>>>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>>>> To: astropy at scipy.org >>>>> Message-ID: <51C0A97B.8090703 at gmail.com> >>>>> Content-Type: text/plain; charset="iso-8859-1" >>>>> >>>>> I have been working on a fitting GUI for a while, although it is made >>>>> with a specific task in mind. >>>>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and >>>>> Pandas. It is made for a specific projhect I'm working and as such not >>>>> yet usable for more general cases, but it could be a starting point, if >>>>> the dependencies don't conflict with astropy politics. >>>>> >>>>> Especially, I am happy about the choice of Pandas for managing a quite >>>>> complex data structure (the fitted and/or guessed values of an >>>>> arbitrary >>>>> number of transitions for an arbitrary number of rows or collapsed rows >>>>> of a spatially resolved spectrum) of a), but also with the Traits-based >>>>> interactive interface to build complex line profiles from single >>>>> gaussians, good for fitting-by-eye and giving good initial guesses for >>>>> fitting of complex line profiles. It hooks directly up to a wrapper >>>>> I've >>>>> made for lmfit, but given the modularity, it should be relatively easy >>>>> to change to other backends. >>>>> >>>>> It's still a work-in-progress, but there are some screenshots here: >>>>> http://flic.kr/s/aHsjGaEMGg . >>>>> I know the choice and number of dependencies may be prohibitive but it >>>>> saved a lot of work on the GUI, and Pandas means the difference between >>>>> sanity and madness when it comes to keeping track of so many >>>>> parameters. >>>>> >>>>> Cheers, >>>>> Emil >>>>> >>>> >>>> >>>> >>>> ________________________________________________________ >>>> Jonathan D. Slavin Harvard-Smithsonian CfA >>>> jslavin at cfa.harvard.edu 60 Garden Street, MS 83 >>>> phone: (617) 496-7981 <%28617%29%20496-7981> Cambridge, MA >>>> 02138-1516 >>>> fax: (617) 496-7577 <%28617%29%20496-7577> USA >>>> ________________________________________________________ >>>> >>>> >>>> >>>> _______________________________________________ >>>> AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy >>>> >>>> >>>> >>>> _______________________________________________ >>>> AstroPy mailing list >>>> AstroPy at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/astropy >>>> >>>> >>> >>> >>> -- >>> Erik >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >> >> >> -- >> ************************************ >> Chris Beaumont >> Graduate Student >> Institute for Astronomy >> University of Hawaii at Manoa >> 2680 Woodlawn Drive >> Honolulu, HI 96822 >> www.ifa.hawaii.edu/~beaumont >> ************************************ >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > > _______________________________________________ > AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Fri Jun 21 07:58:12 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Fri, 21 Jun 2013 07:58:12 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <51C37B5D.8050103@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> Message-ID: <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> On Jun 20, 2013, at 5:59 PM, Th?ger Emil Rivera-Thorsen wrote: > On 20-06-2013 21:20, Adrian Price-Whelan wrote: >> I'm totally lost on what thread I'm responding to, but +100 to what >> Perry said about GUIs! IMHO there is plenty to keep us busy working on >> the modeling and backend, and we should focus our efforts on making >> that code super slick and useable *first*, then worry about a GUI >> (which, also IMO, are overrated! why do you want to click on your >> absorption lines anyways??). > Most of our work doesn't require a GUI, but a few tasks can be helped > immensely by a good one. > One of these, but not the only one, is the case of having to give a good > initial guess to a fitting package, if you have many lines and many > components and many tied, frozen or thawed parameters. > > Something that is vitally important for writing a good GUI, IMO, is > reproducibility, which means that pretty much all settings that one > creates should be possible to write into a script or into the model or > something, so one doesn't have to rely on one's shaky and > coffee-trembling hands to find the same settings multiple times. > > /Emil Yes, I've long felt that was a critical element to a good GUI. They are great to play with the data, and how to view the data. But at some point, one wants to be able to repeat that easily, either to do batch processing, replicate the results (perhaps to tweak a bit more), or be consistent in the processing of other similar data. It would be even better if the GUI could save a script that effectively recreates what was done without requiring any interaction (rather than a lot of hidden machinery to recreate the state). Perry From stsci.perry at gmail.com Fri Jun 21 08:15:20 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Fri, 21 Jun 2013 08:15:20 -0400 Subject: [AstroPy] specutils, pyspeckit, etc. In-Reply-To: References: Message-ID: <76CADD6C-A55E-42E1-9500-45BD3AF13C58@gmail.com> On Jun 20, 2013, at 5:26 PM, Adam wrote: > (apologies for the subject and my delay in replying, I have been > receiving astropy-users e-mails in digest form and have missed nearly > everything. So this e-mail is also a vote to move to > astropy-users at google) > > As Erik said, it's best to merge these parallel efforts and try to > develop one common package with a nice interface. My vision for the > future of pyspeckit is essentially to have it be a wrapper of > specutils + astropy.models + astropy.units, and in fact my current > coding effort is mostly going in that direction (see, e.g., > spectroscopic equivalencies in astropy). I wanted to raise it as an > example of something that works *right now*, but doesn't work > perfectly (as Jonathan rightly pointed out). Most of the models > implemented in pyspeckit should be very easy to re-write as > astropy.models... models. That goal, in particular, is a big > component of one of the specutils SOCIS proposed projects. > > Also, pyspeckit's frontend is matplotlib-only, which is slow but > compatible with everything. Because it's matplotlib-only, and > matplotlib doesn't have a ton of GUI features (more than you'd think, > fewer than you need), the GUI side is very IRAF-splot-like. > I like this approach very much as a starter. Something that gets interactive functionality going quickly, and it is very portable. It's not the equivalent of a good GUI I know, but it's achievable more easily and avoids a lot of installation issues. Which got me thinking a bit. Someone (I forgot who) mentioned that the emails were likely being done in a GUI (and this one certainly is). But it's GUI used by millions of people, and has very substantial developer effort behind it. Arguably it's much simpler than what is needed for spectral analysis and visualization. So what widely used GUI data visualization/analysis applications (let's say non-web for the moment) in astronomy have stood the test of time? I.e., those used only by astronomers. Certainly DS9. (Perhaps imexamine and splot, though those are more like the simpler approach above rather than what we consider GUI's). Aladin? TOPCAT. There are a number of VO-related tools that are GUI's but I'm not sure how heavily they are used. What is people's views of these? I ask since I wonder if there is a consensus of what are successful astronomy GUI's and what we can learn from them. But it also sure seems that there is a very strong movement towards using browsers as the GUI platform and that is where most tools will be headed. With IPython providing a Python/browser connection, I wonder if soon we won't be able to use it for GUIs as well. Perry From te.pickering at gmail.com Fri Jun 21 08:35:40 2013 From: te.pickering at gmail.com (Timothy Pickering) Date: Fri, 21 Jun 2013 14:35:40 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> Message-ID: > Yes, I've long felt that was a critical element to a good GUI. They are great to play with the data, and how to view the data. But at some point, one wants to be able to repeat that easily, either to do batch processing, replicate the results (perhaps to tweak a bit more), or be consistent in the processing of other similar data. It would be even better if the GUI could save a script that effectively recreates what was done without requiring any interaction (rather than a lot of hidden machinery to recreate the state). you mean like a .ipynb file? :) ipython notebooks aren't that far from being a viable platform for building and distributing GUIs. what's needed, though, are tools for better interactivity using javascript, D3, and the like. even so, their current incarnation provides a much greater level of interactivity than a normal script while retaining portability and flexibility. tim -- +-------------------------------------------------------------------+ | T. E. Pickering, Ph.D. | Southern African Large Telescope | | SALT Astronomer | SAAO | | tim at saao.ac.za (520) 305-9823 | Observatory Road | | tim at salt.ac.za +27-71-551-8281 | 7925 Observatory, South Africa | +-------------------------------------------------------------------+ overflow error in /dev/null From stsci.perry at gmail.com Fri Jun 21 08:39:43 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Fri, 21 Jun 2013 08:39:43 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> Message-ID: <2BD96EFD-088A-4B10-9833-44230BE3C06A@gmail.com> On Jun 21, 2013, at 8:35 AM, Timothy Pickering wrote: >> Yes, I've long felt that was a critical element to a good GUI. They are great to play with the data, and how to view the data. But at some point, one wants to be able to repeat that easily, either to do batch processing, replicate the results (perhaps to tweak a bit more), or be consistent in the processing of other similar data. It would be even better if the GUI could save a script that effectively recreates what was done without requiring any interaction (rather than a lot of hidden machinery to recreate the state). > > you mean like a .ipynb file? :) ipython notebooks aren't that far from being a viable platform for building and distributing GUIs. what's needed, though, are tools for better interactivity using javascript, D3, and the like. even so, their current incarnation provides a much greater level of interactivity than a normal script while retaining portability and flexibility. > > tim Well, partly. What IPython doesn't provide is interactive selections on graphics and such, yet (at least entirely within the browser). For example, if I want to plot a spectrum and then select features to fit, or mark initial guesses for centers, etc, IPython doesn't do that. I do think that may be within reach in a year or two. Perry From beaumont at hawaii.edu Fri Jun 21 08:47:05 2013 From: beaumont at hawaii.edu (Chris Beaumont) Date: Fri, 21 Jun 2013 08:47:05 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> Message-ID: For those interested and unaware, many of the patterns in the Design Patterns book ( http://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612, also easily found on google) deal with enabling this kind of functionality. Not coming from a formal CS background, that book taught me a lot about good application design. In particular, the "Command" pattern deals with encapsulating state, and making it more reproducible. The basic idea is to map GUI interactions into explicit request/command objects -- the GUI code creates and fires off these commands, which the backend responds to. This allows you to keep a log of commands, which you can use to recreate state. You can also create a second, scripting interface for building these commands, and even write a translation layer to convert a command log to scripting layer calls. The ChIPS people used this approach for Chandra ( http://cxc.harvard.edu/chips4.4/index.html), and that code has lots of good examples. chris On Fri, Jun 21, 2013 at 7:58 AM, Perry Greenfield wrote: > > On Jun 20, 2013, at 5:59 PM, Th?ger Emil Rivera-Thorsen wrote: > > > On 20-06-2013 21:20, Adrian Price-Whelan wrote: > >> I'm totally lost on what thread I'm responding to, but +100 to what > >> Perry said about GUIs! IMHO there is plenty to keep us busy working on > >> the modeling and backend, and we should focus our efforts on making > >> that code super slick and useable *first*, then worry about a GUI > >> (which, also IMO, are overrated! why do you want to click on your > >> absorption lines anyways??). > > Most of our work doesn't require a GUI, but a few tasks can be helped > > immensely by a good one. > > One of these, but not the only one, is the case of having to give a good > > initial guess to a fitting package, if you have many lines and many > > components and many tied, frozen or thawed parameters. > > > > Something that is vitally important for writing a good GUI, IMO, is > > reproducibility, which means that pretty much all settings that one > > creates should be possible to write into a script or into the model or > > something, so one doesn't have to rely on one's shaky and > > coffee-trembling hands to find the same settings multiple times. > > > > /Emil > > Yes, I've long felt that was a critical element to a good GUI. They are > great to play with the data, and how to view the data. But at some point, > one wants to be able to repeat that easily, either to do batch processing, > replicate the results (perhaps to tweak a bit more), or be consistent in > the processing of other similar data. It would be even better if the GUI > could save a script that effectively recreates what was done without > requiring any interaction (rather than a lot of hidden machinery to > recreate the state). > > Perry > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- ************************************ Chris Beaumont Graduate Student Institute for Astronomy University of Hawaii at Manoa 2680 Woodlawn Drive Honolulu, HI 96822 www.ifa.hawaii.edu/~beaumont ************************************ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewturk at gmail.com Fri Jun 21 08:51:59 2013 From: matthewturk at gmail.com (Matthew Turk) Date: Fri, 21 Jun 2013 08:51:59 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> Message-ID: On Fri, Jun 21, 2013 at 8:47 AM, Chris Beaumont wrote: > For those interested and unaware, many of the patterns in the Design > Patterns book > (http://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612, > also easily found on google) deal with enabling this kind of functionality. > Not coming from a formal CS background, that book taught me a lot about good > application design. > > In particular, the "Command" pattern deals with encapsulating state, and > making it more reproducible. The basic idea is to map GUI interactions into > explicit request/command objects -- the GUI code creates and fires off these > commands, which the backend responds to. This allows you to keep a log of > commands, which you can use to recreate state. You can also create a second, > scripting interface for building these commands, and even write a > translation layer to convert a command log to scripting layer calls. > Since Traits came up earlier, it's worth mentioning that Mayavi is a good example of using Traits to generate scripts that reproduce an interactive workflow. It functions very similarly to how DP lays things out. -Matt > > The ChIPS people used this approach for Chandra > (http://cxc.harvard.edu/chips4.4/index.html), and that code has lots of good > examples. > > chris > > > On Fri, Jun 21, 2013 at 7:58 AM, Perry Greenfield > wrote: >> >> >> On Jun 20, 2013, at 5:59 PM, Th?ger Emil Rivera-Thorsen wrote: >> >> > On 20-06-2013 21:20, Adrian Price-Whelan wrote: >> >> I'm totally lost on what thread I'm responding to, but +100 to what >> >> Perry said about GUIs! IMHO there is plenty to keep us busy working on >> >> the modeling and backend, and we should focus our efforts on making >> >> that code super slick and useable *first*, then worry about a GUI >> >> (which, also IMO, are overrated! why do you want to click on your >> >> absorption lines anyways??). >> > Most of our work doesn't require a GUI, but a few tasks can be helped >> > immensely by a good one. >> > One of these, but not the only one, is the case of having to give a good >> > initial guess to a fitting package, if you have many lines and many >> > components and many tied, frozen or thawed parameters. >> > >> > Something that is vitally important for writing a good GUI, IMO, is >> > reproducibility, which means that pretty much all settings that one >> > creates should be possible to write into a script or into the model or >> > something, so one doesn't have to rely on one's shaky and >> > coffee-trembling hands to find the same settings multiple times. >> > >> > /Emil >> >> Yes, I've long felt that was a critical element to a good GUI. They are >> great to play with the data, and how to view the data. But at some point, >> one wants to be able to repeat that easily, either to do batch processing, >> replicate the results (perhaps to tweak a bit more), or be consistent in the >> processing of other similar data. It would be even better if the GUI could >> save a script that effectively recreates what was done without requiring any >> interaction (rather than a lot of hidden machinery to recreate the state). >> >> Perry >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > ************************************ > Chris Beaumont > Graduate Student > Institute for Astronomy > University of Hawaii at Manoa > 2680 Woodlawn Drive > Honolulu, HI 96822 > www.ifa.hawaii.edu/~beaumont > ************************************ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From matthewturk at gmail.com Fri Jun 21 08:55:45 2013 From: matthewturk at gmail.com (Matthew Turk) Date: Fri, 21 Jun 2013 08:55:45 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <2BD96EFD-088A-4B10-9833-44230BE3C06A@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> <2BD96EFD-088A-4B10-9833-44230BE3C06A@gmail.com> Message-ID: On Fri, Jun 21, 2013 at 8:39 AM, Perry Greenfield wrote: > > On Jun 21, 2013, at 8:35 AM, Timothy Pickering wrote: > >>> Yes, I've long felt that was a critical element to a good GUI. They are great to play with the data, and how to view the data. But at some point, one wants to be able to repeat that easily, either to do batch processing, replicate the results (perhaps to tweak a bit more), or be consistent in the processing of other similar data. It would be even better if the GUI could save a script that effectively recreates what was done without requiring any interaction (rather than a lot of hidden machinery to recreate the state). >> >> you mean like a .ipynb file? :) ipython notebooks aren't that far from being a viable platform for building and distributing GUIs. what's needed, though, are tools for better interactivity using javascript, D3, and the like. even so, their current incarnation provides a much greater level of interactivity than a normal script while retaining portability and flexibility. >> >> tim > > Well, partly. What IPython doesn't provide is interactive selections on graphics and such, yet (at least entirely within the browser). For example, if I want to plot a spectrum and then select features to fit, or mark initial guesses for centers, etc, IPython doesn't do that. I do think that may be within reach in a year or two. Brian Granger recently posted a lengthy summary of the plans for IPython post-1.0, which addresses interactivity directly: http://mail.scipy.org/pipermail/ipython-dev/2013-June/011764.html >From my discussions with Fernando Perez where we discussed swapping the yt web GUI to live on top of IPython instead of a homegrown kernel/server approach, he made it abundantly clear that reproducibility of GUI/widget/interactive elements in the notebook was a top design priority. (Our current approach records all scripting commands and is mediated by python code, but overall leaves a great deal to be desired.) Reproducibility of notebook widgets would be a good topic for discussion at the reproducibility BOF at SciPy this year, which Fernando will be a panelist at. -Matt > > Perry > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From te.pickering at gmail.com Fri Jun 21 09:00:39 2013 From: te.pickering at gmail.com (Timothy Pickering) Date: Fri, 21 Jun 2013 15:00:39 +0200 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <2BD96EFD-088A-4B10-9833-44230BE3C06A@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> <2BD96EFD-088A-4B10-9833-44230BE3C06A@gmail.com> Message-ID: <1362106E-EF2E-4B7C-B3D5-788530044825@gmail.com> > Well, partly. What IPython doesn't provide is interactive selections on graphics and such, yet (at least entirely within the browser). For example, if I want to plot a spectrum and then select features to fit, or mark initial guesses for centers, etc, IPython doesn't do that. I do think that may be within reach in a year or two. exactly. which is where js comes in to handle the interactivity. of course, it gets hairy in detail: http://jakevdp.github.io/blog/2013/06/01/ipython-notebook-javascript-python-communication/ however, as he notes ipython 2.0 will change this up a bit while improving js support overall. tim -- +-------------------------------------------------------------------+ | T. E. Pickering, Ph.D. | Southern African Large Telescope | | SALT Astronomer | SAAO | | tim at saao.ac.za (520) 305-9823 | Observatory Road | | tim at salt.ac.za +27-71-551-8281 | 7925 Observatory, South Africa | +-------------------------------------------------------------------+ overflow error in /dev/null From mdroe at stsci.edu Fri Jun 21 09:42:30 2013 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 21 Jun 2013 09:42:30 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: References: <51C0B69A.6020402@astro.su.se> Message-ID: <51C45846.9090204@stsci.edu> On 06/20/2013 03:51 PM, Chris Beaumont wrote: > I'm more optimistic about GUIs than Adrian (who I bet is reading this > email with a GUI!), but agree that computation and presentation logic > should have sharp boundaries between them -- it makes developing both > much cleaner. > > Another option to consider is to what extent astropy GUIs can revolve > around javascript and the IPython notebook. Installation is much > easier, and this is the python workflow of the future. Unfortunately, > matplotlib in the notebook is still mostly non-interactive, but I have > to think the MPL people are working on this (another reason to delay > interface design!) https://github.com/matplotlib/matplotlib/pull/2054 :) Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From keflavich at gmail.com Fri Jun 21 15:22:53 2013 From: keflavich at gmail.com (Adam) Date: Fri, 21 Jun 2013 13:22:53 -0600 Subject: [AstroPy] specutils, pyspeckit, etc. In-Reply-To: <76CADD6C-A55E-42E1-9500-45BD3AF13C58@gmail.com> References: <76CADD6C-A55E-42E1-9500-45BD3AF13C58@gmail.com> Message-ID: >> Also, pyspeckit's frontend is matplotlib-only, which is slow but >> compatible with everything. Because it's matplotlib-only, and >> matplotlib doesn't have a ton of GUI features (more than you'd think, >> fewer than you need), the GUI side is very IRAF-splot-like. >> > I like this approach very much as a starter. Something that gets interactive functionality going quickly, and it is very portable. It's not the equivalent of a good GUI I know, but it's achievable more easily and avoids a lot of installation issues. Matplotlib also lends itself to easy tests of interactivity, e.g.: https://github.com/pyspeckit/pyspeckit-tests/blob/master/simple_fit_interactive.py Basically, you can mimic any mouse event and let the "Event Manager" parse it. > Which got me thinking a bit. Someone (I forgot who) mentioned that the emails were likely being done in a GUI (and this one certainly is). But it's GUI used by millions of people, and has very substantial developer effort behind it. Arguably it's much simpler than what is needed for spectral analysis and visualization. So what widely used GUI data visualization/analysis applications (let's say non-web for the moment) in astronomy have stood the test of time? I.e., those used only by astronomers. Certainly DS9. (Perhaps imexamine and splot, though those are more like the simpler approach above rather than what we consider GUI's). Aladin? TOPCAT. There are a number of VO-related tools that are GUI's but I'm not sure how heavily they are used. What is people's views of these? ds9 is the only astronomy GUI I used 8 years ago that I *willingly* use now. I'd definitely like to hear others' takes on this. If we want general community feedback, perhaps an astrobetter post asking for community feedback about what they like/dislike about astronomy GUIs would be appropriate? > I ask since I wonder if there is a consensus of what are successful astronomy GUI's and what we can learn from them. > > But it also sure seems that there is a very strong movement towards using browsers as the GUI platform and that is where most tools will be headed. With IPython providing a Python/browser connection, I wonder if soon we won't be able to use it for GUIs as well. I really hope the push @mdboom has been making to incorporate mpl & the browser allow us to use a js frontend for all of our needs. https://github.com/matplotlib/matplotlib/pull/2054 http://jakevdp.github.io/blog/2013/06/01/ipython-notebook-javascript-python-communication/ (both those links were already posted but they deserve even more attention) -- Adam From wkerzendorf at gmail.com Fri Jun 21 16:13:24 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Fri, 21 Jun 2013 16:13:24 -0400 Subject: [AstroPy] specutils, pyspeckit, etc. In-Reply-To: References: <76CADD6C-A55E-42E1-9500-45BD3AF13C58@gmail.com> Message-ID: On 2013-06-21, at 3:22 PM, Adam wrote: >>> Also, pyspeckit's frontend is matplotlib-only, which is slow but >>> compatible with everything. Because it's matplotlib-only, and >>> matplotlib doesn't have a ton of GUI features (more than you'd think, >>> fewer than you need), the GUI side is very IRAF-splot-like. >>> >> I like this approach very much as a starter. Something that gets interactive functionality going quickly, and it is very portable. It's not the equivalent of a good GUI I know, but it's achievable more easily and avoids a lot of installation issues. > > Matplotlib also lends itself to easy tests of interactivity, e.g.: > https://github.com/pyspeckit/pyspeckit-tests/blob/master/simple_fit_interactive.py > > Basically, you can mimic any mouse event and let the "Event Manager" parse it. Matthew Turk this morning pointed to an email about the future of the IPython web-gui and that it's gonna be another half a year or so. However, I think after everything what I've seen from Perez et al. it's always worth the wait. And maybe in the meantime we could just build really quick and simple GUIs using the (somewhat limited) matplotlib gui elements if needed (http://mail.scipy.org/pipermail/ipython-dev/2013-June/011764.html). > >> Which got me thinking a bit. Someone (I forgot who) mentioned that the emails were likely being done in a GUI (and this one certainly is). But it's GUI used by millions of people, and has very substantial developer effort behind it. Arguably it's much simpler than what is needed for spectral analysis and visualization. So what widely used GUI data visualization/analysis applications (let's say non-web for the moment) in astronomy have stood the test of time? I.e., those used only by astronomers. Certainly DS9. (Perhaps imexamine and splot, though those are more like the simpler approach above rather than what we consider GUI's). Aladin? TOPCAT. There are a number of VO-related tools that are GUI's but I'm not sure how heavily they are used. What is people's views of these? > > ds9 is the only astronomy GUI I used 8 years ago that I *willingly* > use now. I'd definitely like to hear others' takes on this. If we > want general community feedback, perhaps an astrobetter post asking > for community feedback about what they like/dislike about astronomy > GUIs would be appropriate? > I love ds9 as it allows me to quickly look at any array I want. It's fast and basically runs on anything. With pyds9 it's trivial to shoot across any ndarray and I have it used often to look at matrices or other mathematical structures and not only images. It doesn't look the greatest, but I agree fully with Adam that it's a really nice tool. The region and wcs features are great. I'm not saying it couldn't use a little bit of a UI overhaul, but I think most of it is great. I think we should reach out to them a little bit and try to get them up to date with the astropy developments. Cheers Wolfgang P.S.: I think this discussion should migrate to astropy-dev. >> I ask since I wonder if there is a consensus of what are successful astronomy GUI's and what we can learn from them. >> >> But it also sure seems that there is a very strong movement towards using browsers as the GUI platform and that is where most tools will be headed. With IPython providing a Python/browser connection, I wonder if soon we won't be able to use it for GUIs as well. > > I really hope the push @mdboom has been making to incorporate mpl & > the browser allow us to use a js frontend for all of our needs. > https://github.com/matplotlib/matplotlib/pull/2054 > http://jakevdp.github.io/blog/2013/06/01/ipython-notebook-javascript-python-communication/ > (both those links were already posted but they deserve even more attention) > > > -- > Adam > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From sd1759 at nyu.edu Mon Jun 24 16:37:06 2013 From: sd1759 at nyu.edu (Sumit Dahal) Date: Mon, 24 Jun 2013 16:37:06 -0400 Subject: [AstroPy] Python equivalent of WVT Binning? Message-ID: <64811230-BBBA-474B-8CA0-20D992DCBFCE@nyu.edu> Hi there, I am trying to plot a hardness ratio map of 2 two-dimensional images. There is a WVT Binning algorithm for this online but it is written in IDL. I was wondering if there is any python equivalent of wvt_xraycolor from WVT binning. Thanks, Sumit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jslavin at cfa.harvard.edu Tue Jun 25 13:19:39 2013 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Tue, 25 Jun 2013 13:19:39 -0400 Subject: [AstroPy] Python equivalent of WVT Binning? Message-ID: On Tue, Jun 25, 2013 at 1:00 PM, wrote: > Date: Mon, 24 Jun 2013 16:37:06 -0400 > From: Sumit Dahal > Subject: [AstroPy] Python equivalent of WVT Binning? > To: astropy at scipy.org > Message-ID: <64811230-BBBA-474B-8CA0-20D992DCBFCE at nyu.edu> > Content-Type: text/plain; charset="us-ascii" > > Hi there, > > I am trying to plot a hardness ratio map of 2 two-dimensional images. > There is a WVT Binning algorithm for this online but it is written in IDL. > I was wondering if there is any python equivalent of wvt_xraycolor from WVT > binning. > > > > Thanks, > Sumit > Sumit, I don't know of a python equivalent to that IDL routine, but if you do have IDL available and just prefer to use python (as I do), then you could use pIDLy to run IDL routines within python. As I say, you need to have IDL installed to use it. Jon ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkloppenborg at gmail.com Wed Jun 26 08:05:55 2013 From: bkloppenborg at gmail.com (Brian Kloppenborg) Date: Wed, 26 Jun 2013 14:05:55 +0200 Subject: [AstroPy] astropy.table remove/delete row? In-Reply-To: <51C45846.9090204@stsci.edu> References: <51C0B69A.6020402@astro.su.se> <51C45846.9090204@stsci.edu> Message-ID: <51CAD923.4030708@gmail.com> Greetings, Today I noticed the astropy.table class ( http://docs.astropy.org/en/latest/_generated/astropy.table.table.Table.html#astropy.table.table.Table) implements many functions for manipulating tables: add_column add_columns add_row remove_column remove_columns but it lacks similar functionality for row manipulation: add_rows - although this is easily accomplished by iteration remove_row remove_rows I could try manipulating the table._data structure directly, but because _data can be a numpy ndarray, dict, list or Table, it is not clear which method I should use without inspecting the data structure. I thought about slicing the table, but this would involve splitting then reassembling the table just to remove one row. I considered creating a mask, but this seems tedious (i.e. something that should be implemented via. a remove_row(s) function) Is a lack of a remove/delete_row(s) missing functionality or am I missing something obvious? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From aldcroft at head.cfa.harvard.edu Wed Jun 26 08:46:17 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Wed, 26 Jun 2013 08:46:17 -0400 Subject: [AstroPy] astropy.table remove/delete row? In-Reply-To: <51CAD923.4030708@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C45846.9090204@stsci.edu> <51CAD923.4030708@gmail.com> Message-ID: On Wed, Jun 26, 2013 at 8:05 AM, Brian Kloppenborg wrote: > Greetings, > > Today I noticed the astropy.table class ( > http://docs.astropy.org/en/latest/_generated/astropy.table.table.Table.html#astropy.table.table.Table) > implements many functions for manipulating tables: > add_column > add_columns > add_row > remove_column > remove_columns > > but it lacks similar functionality for row manipulation: > > add_rows - although this is easily accomplished by iteration > remove_row > remove_rows > > I could try manipulating the table._data structure directly, but because > _data can be a numpy ndarray, dict, list or Table, it is not clear which > method I should use without inspecting the data structure. > > I thought about slicing the table, but this would involve splitting then > reassembling the table just to remove one row. I considered creating a > mask, but this seems tedious (i.e. something that should be implemented > via. a remove_row(s) function) > > Is a lack of a remove/delete_row(s) missing functionality or am I missing > something obvious? > I think this is missing functionality and just opened a feature request issue: https://github.com/astropy/astropy/issues/1205 In the meantime there are some slightly nicer solutions for doing these operations. The current dev version of astropy the table package has a vstack function that lets you stack tables together. So something like the following will work for add_rows, assuming you have a table named `dat` and some `rows`: >>> from astropy.table import vstack, Table >>> dat = vstack(dat, Table(rows)) In this case the Table() initialization is taking care of converting any valid inputs for `rows` into a nice table, and `vstack` is doing all the logic to ensure that the columns match up, etc. Note that at this time you cannot directly initialize a Table from a list of row tuples. This will require some extra keyword arg in the initialization to distinguish a list of rows from a list of columns, and we haven't settled on the best way yet. The way around this is: >>> rows = [[1, 2.0, 'string'], # list of rows [2, 3.0, 'values']] >>> col_arr = zip(*rows) # transpose to a list of columns >>> t = Table(col_arr) For removing rows you just need a couple of extra lines of code: >>> mask = np.ones(len(dat), dtype=bool) >>> mask[4] = False # To remove one row >>> mask[4:25] = False # To remove a slice >>> dat = dat[mask] Cheers, Tom > Thanks, > Brian > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Wed Jun 26 11:11:44 2013 From: embray at stsci.edu (Erik Bray) Date: Wed, 26 Jun 2013 11:11:44 -0400 Subject: [AstroPy] PyFITS Too many open files In-Reply-To: References: Message-ID: <51CB04B0.9030501@stsci.edu> On 06/15/2013 06:42 PM, Nico Nell wrote: > You can try running the python garbage collector after each file close with > gc.collect(). > > For the memmap issue consider submitting a bug report if you have time: > https://github.com/spacetelescope/PyFITS/issues?state=open Please don't :) This isn't a bug. I've seen this sort of thing before when people are opening lots of FITS files in a loop. When you create references to the data with time = fits[1].data["TIME"] energy = fits[1].data["ENERGY"] each one of those is opening file handles to read the data in. I don't know what your code is doing with the data later in the loop, but my suggestion would be at the end of the loop to manually call del time del energy Otherwise what ends up happening is that all your allowed file handles get used up before the Python garbage collector even gets a chance to run and close off unused file handles. Erik > On Sat, Jun 15, 2013 at 4:21 PM, Nicholas Earl > wrote: > > Thanks for the ideas! It seems that memmap=False solves the problem, > although I can see that it does begin using a large percentage of the memory > -- I can only imagine how much if I let it run on the entire set of > directories... Is there not any way to flush it all after I close the file? > > I am using python 2.7.5, and PyFITs 3.1.2. > > Thanks again, everyone, you're awesome. > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > > On Sat, Jun 15, 2013 at 11:01 AM, Nico Nell > wrote: > > Sorry -- I should have read the blip of code better. > > So, what version of python and pyfits are you using? How many files are > you opening? > > What happens if you add 'memmap=False' in each of your pyfits open() > lines. The pyfits mmap stuff is relatively new so maybe there is an issue. > > ~Nick > > > On Sat, Jun 15, 2013 at 11:29 AM, Nicholas Earl > wrote: > > Sorry, but it seems I sent this back to only Nice Nell, and not the > entire list! > > ----- > > Thanks for the reply. I was under the impression that the > with command automatically closed the file? Regardless, I went ahead > and wrote the script up like this > > import pyfits > > for filename in os.listdir(directory): > kep_id = filename[4:13] > fits = pyfits.open(directory + "/" + filename, ignore_missing_end=True) > time = fits[1].data["TIME"] > energy = fits[1].data["ENERGY"] > fits.close() > > Unfortunately, I get the same error. > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > > On Fri, Jun 14, 2013 at 10:08 PM, Nico Nell > wrote: > > You need to close the files once you have retrieved the data... > > See here: > http://pythonhosted.org/pyfits/api_docs/api_hdulists.html#pyfits.HDUList.close > > ~Nick > > > On Fri, Jun 14, 2013 at 10:51 PM, Nicholas Earl > > wrote: > > Hello everyone, > > I'm having an issue with opening and retrieving information > for a large amount of fits files. Mainly, I keep running > into a mmap.error: [Errno 24] Too many open files error. The > code I'm using is: > > import pyfits > > for filename in os.listdir(directory): > kep_id = filename[4:13] > > with pyfits.open(directory + "/" + filename, > ignore_missing_end=True) as fits: > time = fits[1].data["TIME"] > energy = fits[1].data["ENERGY"] > > Any ideas would be greatly appreciated! > > Nicholas Earl > Graduate Research Assistant > San Diego State University > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From embray at stsci.edu Wed Jun 26 11:32:23 2013 From: embray at stsci.edu (Erik Bray) Date: Wed, 26 Jun 2013 11:32:23 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C37B5D.8050103@gmail.com> <62667486-7D81-422F-A1B5-A64D8418EFB4@gmail.com> Message-ID: <51CB0987.3070800@stsci.edu> On 06/21/2013 07:58 AM, Perry Greenfield wrote: > > On Jun 20, 2013, at 5:59 PM, Th?ger Emil Rivera-Thorsen wrote: > >> On 20-06-2013 21:20, Adrian Price-Whelan wrote: >>> I'm totally lost on what thread I'm responding to, but +100 to what >>> Perry said about GUIs! IMHO there is plenty to keep us busy working on >>> the modeling and backend, and we should focus our efforts on making >>> that code super slick and useable *first*, then worry about a GUI >>> (which, also IMO, are overrated! why do you want to click on your >>> absorption lines anyways??). >> Most of our work doesn't require a GUI, but a few tasks can be helped >> immensely by a good one. >> One of these, but not the only one, is the case of having to give a good >> initial guess to a fitting package, if you have many lines and many >> components and many tied, frozen or thawed parameters. >> >> Something that is vitally important for writing a good GUI, IMO, is >> reproducibility, which means that pretty much all settings that one >> creates should be possible to write into a script or into the model or >> something, so one doesn't have to rely on one's shaky and >> coffee-trembling hands to find the same settings multiple times. > > Yes, I've long felt that was a critical element to a good GUI. They are great to play with the data, and how to view the data. But at some point, one wants to be able to repeat that easily, either to do batch processing, replicate the results (perhaps to tweak a bit more), or be consistent in the processing of other similar data. It would be even better if the GUI could save a script that effectively recreates what was done without requiring any interaction (rather than a lot of hidden machinery to recreate the state). Yes, yes, yes. I've spent hours trying to help people debug problems that involved, at some point or another, interacting with a GUI. And I've seen people swear up and down that they were doing the exact same thing they did before and are now getting different results. Only to realize, eventually, that they were actually doing something subtly different than their original process. Any GUI that involves processes like selecting absorption lines or setting initial guesses for fits should be able to record the process as a script (either in a simple DSL or even as executable code) that can reproduce the exact steps taken without invoking the GUI at all. Erik From keflavich at gmail.com Wed Jun 26 11:47:08 2013 From: keflavich at gmail.com (Adam) Date: Wed, 26 Jun 2013 09:47:08 -0600 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 32 In-Reply-To: References: Message-ID: > Any GUI that involves processes like selecting absorption lines or setting > initial guesses for fits should be able to record the process as a script > (either in a simple DSL or even as executable code) that can reproduce the exact > steps taken without invoking the GUI at all. Great point. It's actually fairly easy to do in matplotlib: you can just make your "event manager" routine record a history of events, e.g.: self.event_history.append(event) and as long as you have one overarching event manager for the GUI (which is probably an important design decision...), it should be able to track exactly what has been done and repeat it. That said, events can be tied to individual axes, so if a window is closed, it would be essential to re-create that window in the same state it was originally initialized. Anyway, 2 cents on GUI design... -- Adam From embray at stsci.edu Wed Jun 26 11:51:52 2013 From: embray at stsci.edu (Erik Bray) Date: Wed, 26 Jun 2013 11:51:52 -0400 Subject: [AstroPy] AstroPy Digest, Vol 81, Issue 12 In-Reply-To: <51C36F3C.7040906@gmail.com> References: <51C0B69A.6020402@astro.su.se> <51C36F3C.7040906@gmail.com> Message-ID: <51CB0E18.9030906@stsci.edu> On 06/20/2013 05:08 PM, Th?ger Emil Rivera-Thorsen wrote: > My reason for writing it was especially that it looks like `astropy.tables` does > a lot of effort duplication by reinventing functionality that is already in > Pandas.It may be that `astropy.tables` already has alevel of maturity and > functionality which means it really isn't worth it to start tinkering with a > package like pandas, I don't know. > > What I do know is that Pandas has been immensely helpful for my particular, > personal use, which is as an extended record array. The killer feature, in my > view, is the ability to store heterogeneous data with integer- or label-based, > possibly hierarchical, indexing. In my case, where my model consists of > amplitude, fwhm, centroid parameter sets of multiple components of multiple > transitions for multiple spatial regions of multiple exposures and /keeping > track of which ones belong to the same physical subsystem/, Pandas is > indispensable. It allows me to sort, regroup, etc. by a multitude of criteria. > "Show me the amplitudes of all the components 'c' of the 10th through 21st > spatial row of the Balmer lines" is a one-liner. The slicing-and-dicing > capabilities are quite handy, also for someone who's never used R or done any > statistical work worth mentioning. I could be wrong, but I don't think anyone is arguing that Pandas isn't incredibly powerful and useful. Pandas DataFrames simply aren't suitable as the built-in "table" implementation in Astropy--on some level it's overkill, and on other levels it's insufficient for some required use cases. Erik > It is true that TraitsUI (and Chaco, by the way), have had some stability > issues, and unfortunately still have. I've found that the MacPorts packages work > fine. But no, they are not as mature and thoroughly tested as e.g. Matplotlib. > So if it were only for the plotting capabilities, the choive of matplotlib would > be a no-brainer. But as we have both experienced, traitsui saves you a *lot* of > coding time when building a simpler GUI. I honestly don't know if it is a good > choice of basis for an affiliated package, all things considered, but I > completely agree that it should stay well outside of core. > > > > On 20-06-2013 18:34, Erik Tollerud wrote: > >> I'm of mixed minds about traits UI because once you know it you can make great >> GUIs with it, but I've spent a lot of time troubleshooting people's python >> installations to get traits to work. That is, in general it can be tricky to >> get installed because of all the dependencies. Maybe this has improved >> recently with Enthought's Canopy (or other new python distros), but that's >> been my past experience. >> >> More generally, the view in the astropy core package is that we don't want to >> put GUIs in the core because GUIs always carry lots of dependencies, which we >> don't want to be forced to deal with. But part of the whole reason for >> affiliated packages was to get around this, so we're happy to see GUI-based >> affiliated packages. >> >> >> As for Pandas, to be totally honest, I don't see a huge amount to be gained >> from adding a Pandas dependency Astropy. It's honestly not clear what it >> gives the astronomy community that numpy does not already have. The following >> quote from the Pandas web site has guided me to that conclusion: >> "/pandas/ helps fill this gap, enabling you to carry out your entire data >> analysis workflow in Python without having to switch to a more domain specific >> language like R." >> >> I have been carrying out my entire data analysis workflow for some time now in >> python without using Pandas. It looks to me like Pandas is a tool that was >> written by and for statisticians who use R. While we can take lessons from >> this, it's not clear we get much out of it in an astronomy context. For >> example, how would it make astropy's NDData, Quantity, or Table better to use >> a Pandas DataFrame vs. a numpy array? Most of what we are doing is >> building astronomy-convenient interfaces, and I'm not sure what Pandas adds >> there, at the cost of a pretty heavy-weight dependency. >> >> It could just be that I don't know enough about Pandas, though. So if someone >> who knows Pandas better can speak to this, I'm all ears. >> >> >> >> On Tue, Jun 18, 2013 at 3:35 PM, Th?ger Rivera-Thorsen > > wrote: >> >> Pandas is a part of the newly-defined SciPy stack, after all, so that >> would be part of any science-oriented distribution worth its salt. In >> fact, I think it could be a good idea for astropy in general to use under >> the hood, but again, could clash with the philosophy of the project and >> possibly also maintainabillity. >> >> As for offering my code or just my experience, I'll have to square it with >> my supervisor first, and I also think it depends on what direction the >> project in question will take. I'm positive about the idea (which is why I >> wrote in the first place), but supervisor might think it is a better idea >> to actually get my paper in the project wrapped up before sending the code >> out there. Will get back about that one! >> >> /Emil >> >> >> >> >> >> On 2013-06-18 20:53, Slavin, Jonathan wrote: >>> Hi Emil, >>> >>> That looks very nice! I don't see Pandas as a big issue in terms of >>> dependencies. I don't know that much about traits, etc. My thought >>> about the gui was just based on my experience with matplotlib, and the >>> fact that it is widely used -- though I would agree that too many >>> dependencies can be a deterrent to people using something. Are you >>> offering your code as a starting point for the project? It strikes me >>> that many have gotten some sort of fitting package to a point of personal >>> usability but no one has the time/interest/motivation to make a more >>> generally usable package. >>> >>> Jon >>> >>> On Tue, Jun 18, 2013 at 2:34 PM, >> > wrote: >>> >>> Date: Tue, 18 Jun 2013 20:39:55 +0200 >>> From: Th?ger Rivera-Thorsen >> > >>> Subject: Re: [AstroPy] ESA Summer of Code in Space 2013 >>> To: astropy at scipy.org >>> Message-ID: <51C0A97B.8090703 at gmail.com >>> > >>> Content-Type: text/plain; charset="iso-8859-1" >>> >>> I have been working on a fitting GUI for a while, although it is made >>> with a specific task in mind. >>> However, it is not based on Matplotlib but on Traits/Traitsui/Chaco and >>> Pandas. It is made for a specific projhect I'm working and as such not >>> yet usable for more general cases, but it could be a starting point, if >>> the dependencies don't conflict with astropy politics. >>> >>> Especially, I am happy about the choice of Pandas for managing a quite >>> complex data structure (the fitted and/or guessed values of an arbitrary >>> number of transitions for an arbitrary number of rows or collapsed rows >>> of a spatially resolved spectrum) of a), but also with the Traits-based >>> interactive interface to build complex line profiles from single >>> gaussians, good for fitting-by-eye and giving good initial guesses for >>> fitting of complex line profiles. It hooks directly up to a wrapper I've >>> made for lmfit, but given the modularity, it should be relatively easy >>> to change to other backends. >>> >>> It's still a work-in-progress, but there are some screenshots here: >>> http://flic.kr/s/aHsjGaEMGg . >>> I know the choice and number of dependencies may be prohibitive but it >>> saved a lot of work on the GUI, and Pandas means the difference between >>> sanity and madness when it comes to keeping track of so many parameters. >>> >>> Cheers, >>> Emil >>> >>> >>> >>> >>> ________________________________________________________ >>> Jonathan D. Slavin Harvard-Smithsonian CfA >>> jslavin at cfa.harvard.edu 60 Garden >>> Street, MS 83 >>> phone: (617) 496-7981 Cambridge, MA >>> 02138-1516 >>> fax: (617) 496-7577 USA >>> ________________________________________________________ >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> >> -- >> Erik >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From eric.emsellem at eso.org Wed Jun 26 15:52:52 2013 From: eric.emsellem at eso.org (Eric Emsellem) Date: Wed, 26 Jun 2013 21:52:52 +0200 Subject: [AstroPy] Python equivalent of WVT Binning? Message-ID: <51CB4694.1050008@eso.org> Hi I have in fact a python equivalent of the WVT binning you may be thinking of. It is part of a bigger package I am going to release this Summer to do modelling, Jeans Eq solving, N body simulations using the MGE formalism In case you're interested I can certainly send it to you (just the WVT) Eric -- ================================================================= Eric Emsellem European Southern Observatory Head of the Office for Science Karl-Schwarzschild Str. 2 e-mail: eric.emsellem at eso.org 85748 Garching bei Muenchen tel: +49 (0)89 3200 6914 Germany fax: +49 (0)89 3200 6480 ================================================================= From masato.onodera at gmail.com Thu Jun 27 12:51:50 2013 From: masato.onodera at gmail.com (Masato Onodera) Date: Thu, 27 Jun 2013 18:51:50 +0200 Subject: [AstroPy] question on fits header entry on WCS Message-ID: Hello, I have a fits image with 1225x1326 pixels and header section seen in the following URL: https://gist.github.com/monodera/e3d95d2bcdc8afcf860a The wcs information of the image has been put with ccmap in IRAF. Then I wrote a script to derive RA and Dec from pixel coordinates. import numpy as np import astropy.io.fits as fits import astropy.wcs as wcs hdu = fits.open('testimage.fits') w = wcs.WCS(hdu[0].header) pixcrd = np.array([[128, 128], [hdu[0].header['NAXIS1']/2, hdu[0].header['NAXIS2']/2], [1024,1024]], np.float_) world = w.wcs_pix2world(pixcrd, 1) print(world) With this code, I got the following output. WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / Equatorial coordinate system RADECSYS is non-standard, use RADESYSa. [astropy.wcs.wcs] WARNING: FITSFixedWarning: PROJP1 = 0.0 / Projection Type of the first axis Defunct PROJPn keyword rejected. [astropy.wcs.wcs] WARNING: FITSFixedWarning: PROJP2 = 0.0 / Projection Type of the second axis Defunct PROJPn keyword rejected. [astropy.wcs.wcs] WARNING: FITSFixedWarning: PC001001= -0.939693 / Coordinate translation matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] WARNING: FITSFixedWarning: PC001002= 0.342020 / Coordinate translation matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] WARNING: FITSFixedWarning: PC002001= 0.342020 / Coordinate translation matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] WARNING: FITSFixedWarning: PC002002= 0.939693 / Coordinate translation matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] WARNING: FITSFixedWarning: 'datfix' made the change 'Changed '2013-04-29 ' to '2013-04-29T00:00:00.0''. This FITS header contains non-standard content. [astropy.wcs.wcs] WARNING: FITSFixedWarning: 'unitfix' made the change 'Changed units: 'degree ' -> 'deg', 'degree ' -> 'deg''. This FITS header contains non-standard content. [astropy.wcs.wcs] [[ 286.2970047 -60.82647447] [ 268.12706319 -0.58471502] [ 127.71449436 66.63600366]] However, other programs which can convert coorinates such as ds9 and xy2sky from wcstools give the following output and this is actually what I expect. $ xy2sky -d testimage.fits 128 128 612 663 1024 1024 222.31848 8.92023 J2000 128.000 128.000 222.31446 8.93005 J2000 612.000 663.000 222.31055 8.93710 J2000 1024.000 1024.000 Then I found that "w = wcs.WCS(hdu[0].header, relax=False)" gives the expected result. Since I want to make a figure with APLpy and it apparently does not accept "relax=False", I would like to know which header entries cause the wrong result when I put "relax=True" and how to fix it. I guess they would be the entries mentioned in the warning messages, but I'm not sure exactly... Thank you in advance. Masato Onodera -------------- next part -------------- An HTML attachment was scrubbed... URL: From pweilbacher at aip.de Fri Jun 28 07:11:46 2013 From: pweilbacher at aip.de (Peter Weilbacher) Date: Fri, 28 Jun 2013 13:11:46 +0200 (CEST) Subject: [AstroPy] question on fits header entry on WCS In-Reply-To: References: Message-ID: On Thu, 27 Jun 2013, Masato Onodera wrote: > I have a fits image with 1225x1326 pixels and header section seen in the > following URL: https://gist.github.com/monodera/e3d95d2bcdc8afcf860a [... snip ...] > WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / Equatorial coordinate system > RADECSYS is non-standard, use RADESYSa. [astropy.wcs.wcs] > WARNING: FITSFixedWarning: PROJP1 = 0.0 / Projection Type of the first axis > Defunct PROJPn keyword rejected. [astropy.wcs.wcs] > WARNING: FITSFixedWarning: PROJP2 = 0.0 / Projection Type of the second > axis Defunct PROJPn keyword rejected. [astropy.wcs.wcs] > WARNING: FITSFixedWarning: PC001001= -0.939693 / Coordinate translation > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > WARNING: FITSFixedWarning: PC001002= 0.342020 / Coordinate translation > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > WARNING: FITSFixedWarning: PC002001= 0.342020 / Coordinate translation > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > WARNING: FITSFixedWarning: PC002002= 0.939693 / Coordinate translation > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] [... snip ...] > Then I found that "w = wcs.WCS(hdu[0].header, relax=False)" gives the > expected result. Since I want to make a figure with APLpy and it > apparently does not accept "relax=False", I would like to know which header > entries cause the wrong result when I put "relax=True" and how to fix it. > I guess they would be the entries mentioned in the warning messages, but > I'm not sure exactly... That's my guess, too. Look at for all those obsolete keywords... In your case the PC00i00j keywords seem to contain a totally different WCS than CDi_j which would explain the unlikely output your script gives you. Cheers, Peter. -- Dr. Peter Weilbacher http://www.aip.de/People/PWeilbacher Phone +49 331 74 99-667 encryption key ID 7D6B4AA0 ------------------------------------------------------------------------ Leibniz-Institut f?r Astrophysik Potsdam (AIP) An der Sternwarte 16, D-14482 Potsdam Vorstand: Prof. Dr. Matthias Steinmetz, Dr. Ulrich M?ller Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 From ejensen1 at swarthmore.edu Fri Jun 28 10:04:14 2013 From: ejensen1 at swarthmore.edu (Eric L. N. Jensen) Date: Fri, 28 Jun 2013 10:04:14 -0400 Subject: [AstroPy] question on fits header entry on WCS In-Reply-To: References: Message-ID: <5DE61B98-E806-4264-8770-4EE9E6A8F9DB@swarthmore.edu> Hi Masato, I agree that the problem is the one Peter points out, which is that the header seems to specify the WCS in two different ways. I"ll just add that the FITS standard specifically says "Note that the CDi_j formalism is an exclusive alternative to PCi_j, and the CDi_j and PCi_j keywords must not appear together within an HDU." (http://www.aanda.org/articles/aa/full_html/2010/16/aa15362-10/aa15362-10.html#S196, Section 8.2). But the header you link to has both of those kinds of keywords (PC and CD) in it. The PCi_j keywords would appear to define a rotation of the coordinate axes, but it doesn't have any CDELTi keywords to go along with it. So as a first cut I'd try deleting all of the PCij and PROJPi keywords, and changing the RADECSYS to RADESYS, and then see if that gives you the result you expect. Eric On Jun 28, 2013, at 7:11 AM, Peter Weilbacher wrote: > On Thu, 27 Jun 2013, Masato Onodera wrote: > >> I have a fits image with 1225x1326 pixels and header section seen in the >> following URL: https://gist.github.com/monodera/e3d95d2bcdc8afcf860a > [... snip ...] >> WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / Equatorial coordinate system >> RADECSYS is non-standard, use RADESYSa. [astropy.wcs.wcs] >> WARNING: FITSFixedWarning: PROJP1 = 0.0 / Projection Type of the first axis >> Defunct PROJPn keyword rejected. [astropy.wcs.wcs] >> WARNING: FITSFixedWarning: PROJP2 = 0.0 / Projection Type of the second >> axis Defunct PROJPn keyword rejected. [astropy.wcs.wcs] >> WARNING: FITSFixedWarning: PC001001= -0.939693 / Coordinate translation >> matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] >> WARNING: FITSFixedWarning: PC001002= 0.342020 / Coordinate translation >> matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] >> WARNING: FITSFixedWarning: PC002001= 0.342020 / Coordinate translation >> matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] >> WARNING: FITSFixedWarning: PC002002= 0.939693 / Coordinate translation >> matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > [... snip ...] >> Then I found that "w = wcs.WCS(hdu[0].header, relax=False)" gives the >> expected result. Since I want to make a figure with APLpy and it >> apparently does not accept "relax=False", I would like to know which header >> entries cause the wrong result when I put "relax=True" and how to fix it. >> I guess they would be the entries mentioned in the warning messages, but >> I'm not sure exactly... > > That's my guess, too. Look at > for all those obsolete > keywords... > In your case the PC00i00j keywords seem to contain a totally different > WCS than CDi_j which would explain the unlikely output your script gives > you. > > Cheers, > Peter. > - -------------- next part -------------- An HTML attachment was scrubbed... URL: From slianou at uwo.ca Fri Jun 28 10:09:09 2013 From: slianou at uwo.ca (Sofia Lianou) Date: Fri, 28 Jun 2013 10:09:09 -0400 Subject: [AstroPy] IRAF tasks to Astropy Message-ID: <7680ff4471c32.51cd60c5@uwo.ca> Hi, I was wondering if there are Python/Astropy equivalents to (or plans to do so in the near future for) the following IRAF tasks: noao.artdata.mkpattern images.immatch.wregister images.imcoords.ccsetwcs Thank you, Sophia -------------- next part -------------- An HTML attachment was scrubbed... URL: From masato.onodera at gmail.com Sat Jun 29 19:40:37 2013 From: masato.onodera at gmail.com (Masato Onodera) Date: Sun, 30 Jun 2013 01:40:37 +0200 Subject: [AstroPy] question on fits header entry on WCS In-Reply-To: <5DE61B98-E806-4264-8770-4EE9E6A8F9DB@swarthmore.edu> References: <5DE61B98-E806-4264-8770-4EE9E6A8F9DB@swarthmore.edu> Message-ID: Hi Peter and Eric, after following the suggestion from you, it worked perfectly! Thank you very much. Also, thanks for pointing a link on WCS-related header entries. Cheers, Masato On Fri, Jun 28, 2013 at 4:04 PM, Eric L. N. Jensen wrote: > Hi Masato, > > I agree that the problem is the one Peter points out, which is that the > header seems to specify the WCS in two different ways. I"ll just add that > the FITS standard specifically says > > "Note that the CD*i_j *formalism is an exclusive alternative to PC*i_j*, > and the CD*i_j *and PC*i_j *keywords *must not *appear together within > an HDU." ( > http://www.aanda.org/articles/aa/full_html/2010/16/aa15362-10/aa15362-10.html#S196, > Section 8.2). > > But the header you link to has both of those kinds of keywords (PC and CD) > in it. The PCi_j keywords would appear to define a rotation of the > coordinate axes, but it doesn't have any CDELTi keywords to go along with > it. So as a first cut I'd try deleting all of the PCij and PROJPi > keywords, and changing the RADECSYS to RADESYS, and then see if that gives > you the result you expect. > > Eric > > > On Jun 28, 2013, at 7:11 AM, Peter Weilbacher wrote: > > On Thu, 27 Jun 2013, Masato Onodera wrote: > > I have a fits image with 1225x1326 pixels and header section seen in the > > following URL: https://gist.github.com/monodera/e3d95d2bcdc8afcf860a > > [... snip ...] > > WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / Equatorial coordinate system > > RADECSYS is non-standard, use RADESYSa. [astropy.wcs.wcs] > > WARNING: FITSFixedWarning: PROJP1 = 0.0 / Projection Type of the first axis > > Defunct PROJPn keyword rejected. [astropy.wcs.wcs] > > WARNING: FITSFixedWarning: PROJP2 = 0.0 / Projection Type of the second > > axis Defunct PROJPn keyword rejected. [astropy.wcs.wcs] > > WARNING: FITSFixedWarning: PC001001= -0.939693 / Coordinate translation > > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > > WARNING: FITSFixedWarning: PC001002= 0.342020 / Coordinate translation > > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > > WARNING: FITSFixedWarning: PC002001= 0.342020 / Coordinate translation > > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > > WARNING: FITSFixedWarning: PC002002= 0.939693 / Coordinate translation > > matrix Defunct form of PCi_ja keyword. [astropy.wcs.wcs] > > [... snip ...] > > Then I found that "w = wcs.WCS(hdu[0].header, relax=False)" gives the > > expected result. Since I want to make a figure with APLpy and it > > apparently does not accept "relax=False", I would like to know which header > > entries cause the wrong result when I put "relax=True" and how to fix it. > > I guess they would be the entries mentioned in the warning messages, but > > I'm not sure exactly... > > > That's my guess, too. Look at > for all those obsolete > keywords... > In your case the PC00i00j keywords seem to contain a totally different > WCS than CDi_j which would explain the unlikely output your script gives > you. > > Cheers, > Peter. > - > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rahul.poruri at gmail.com Sun Jun 30 11:46:48 2013 From: rahul.poruri at gmail.com (rahul .poruri) Date: Sun, 30 Jun 2013 21:16:48 +0530 Subject: [AstroPy] Questions & Clarifications Message-ID: Sorry for the vague subject but i have a couple of questions - Clarification 1 - for a mollweide projection, having done subplot(111, projection="mollweide") scatter (ra,dec) what should the RA range be i.e from 0:360 or -180:180 ?! There was a discussion in the mailing list on mollweide projection plotting earlier where -180:180 was used as the range. Just wanted to get it clarified. Also, are mollweide projections the standard in astronomy?! Clarification 2 - I was unable to open spectra FITS files (from the SDSS DR9) on DS9. Am i missing any feature or does DS9 not have the capability to open spectra FITS files?! If so, is there any other program i can use to open and *blink* through the spectra?! (other than fv by NASA's HEASARC) Question 3 - I want to plot the spectra on the same window, to compare - in case there isn't a GUI to do the same. I created a list of all of the spectra in my directory and it's a file with ',\n' as the line separators. I then changed the attributes of * * *class** *astropy.io.ascii.basic.Basic * * to rdr = ascii.get_reader(Reader=ascii.Basic)#rdr.header.splitter.delimiter = ' 'rdr.data.splitter.delimiter = ' ,\n '#rdr.header.start_line = 0rdr.data.start_line = 0rdr.data.end_line = None#rdr.header.comment = r'\s*#'rdr.data.comment = r'\s*#' Now, i'm just using list = astropy.io.ascii.read('fits_list.dat') to read the file and use list[i][0] to access the file name, which i feed to my python code to extract and plot the spectra. Is there a different/more straight forward way to do this?! Thanks in advance for the replies :)... -------------- next part -------------- An HTML attachment was scrubbed... URL: From adrianmpw at gmail.com Sun Jun 30 11:52:44 2013 From: adrianmpw at gmail.com (Adrian Price-Whelan) Date: Sun, 30 Jun 2013 11:52:44 -0400 Subject: [AstroPy] Questions & Clarifications In-Reply-To: References: Message-ID: Hi Rahul -- Answers in-line below.. On Sun, Jun 30, 2013 at 11:46 AM, rahul .poruri wrote: > Sorry for the vague subject but i have a couple of questions - > > Clarification 1 - > for a mollweide projection, having done > > subplot(111, projection="mollweide") > scatter (ra,dec) > > what should the RA range be i.e from 0:360 or -180:180 ?! > There was a discussion in the mailing list on mollweide projection plotting > earlier where -180:180 was used as the range. Just wanted to get it > clarified. > Also, are mollweide projections the standard in astronomy?! When plotting to a spherical projection with matplotlib, you have to provide angles in *radians*. The longitudinal angle should be bound (-180,180) and the latitudinal angle should be (-90,90). > Clarification 2 - > I was unable to open spectra FITS files (from the SDSS DR9) on DS9. > Am i missing any feature or does DS9 not have the capability to open spectra > FITS files?! Can you tell me a bit more about where you downloaded the spectrum file from, and also send the error you got from astropy.io.fits? > If so, is there any other program i can use to open and *blink* through the > spectra?! > (other than fv by NASA's HEASARC) > > Question 3 - > I want to plot the spectra on the same window, to compare - in case there > isn't a GUI to do the same. I created a list of all of the spectra in my > directory and it's a file with ',\n' as the line separators. I then changed > the attributes of > > class astropy.io.ascii.basic.Basic > > to > > > rdr = ascii.get_reader(Reader=ascii.Basic) > #rdr.header.splitter.delimiter = ' ' > rdr.data.splitter.delimiter = ' ,\n ' > #rdr.header.start_line = 0 > rdr.data.start_line = 0 > rdr.data.end_line = None > #rdr.header.comment = r'\s*#' > rdr.data.comment = r'\s*#' > > Now, i'm just using > list = astropy.io.ascii.read('fits_list.dat') to read the file and use > list[i][0] to access the file name, which i feed to my python code to > extract and plot the spectra. > > Is there a different/more straight forward way to do this?! > > Thanks in advance for the replies :)... Do you want to plot all of the spectra on the same plot, or on different plots on the same figure? -- Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw From rahul.poruri at gmail.com Sun Jun 30 12:04:06 2013 From: rahul.poruri at gmail.com (rahul .poruri) Date: Sun, 30 Jun 2013 21:34:06 +0530 Subject: [AstroPy] Questions & Clarifications In-Reply-To: References: Message-ID: > Clarification 2 - > > I was unable to open spectra FITS files (from the SDSS DR9) on DS9. > > Am i missing any feature or does DS9 not have the capability to open > spectra > > FITS files?! > > Can you tell me a bit more about where you downloaded the spectrum > file from, and also send the error you got from astropy.io.fits? like i mentioned, my spectrum files are from the Data Release 9 of the SDSS catalogue and they're spectra of quasars. And i was trying to open then in DS9 - not using astropy.io.fits. I've been able to open the fits files in python using astropy and then plot then but i wanted to use the blink functionality in DS9 to be able to run through the spectra and look for variations. The error is just Unable to load fits file.fits > > Question 3 - > > I want to plot the spectra on the same window, to compare - in case there > > isn't a GUI to do the same. I created a list of all of the spectra in my > > directory and it's a file with ',\n' as the line separators. I then > changed > > the attributes of > > > > class astropy.io.ascii.basic.Basic > > > > to > > > > > > rdr = ascii.get_reader(Reader=ascii.Basic) > > #rdr.header.splitter.delimiter = ' ' > > rdr.data.splitter.delimiter = ' ,\n ' > > #rdr.header.start_line = 0 > > rdr.data.start_line = 0 > > rdr.data.end_line = None > > #rdr.header.comment = r'\s*#' > > rdr.data.comment = r'\s*#' > > > > Now, i'm just using > > list = astropy.io.ascii.read('fits_list.dat') to read the file and use > > list[i][0] to access the file name, which i feed to my python code to > > extract and plot the spectra. > > > > Is there a different/more straight forward way to do this?! > > > > Thanks in advance for the replies :)... > > Do you want to plot all of the spectra on the same plot, or on > different plots on the same figure? > I want to plot all of the spectra on the same plot, as i mentioned rather vaguely(sorry for that). It's ~20 spectra so i don't know if subplots will make it any easier to compare them - although plotting them on the same plot will make it crowded, i feel it's a better option to having ~20 subplots in the same figure. Or is there anyother way to go about plotting and comparing spectra?! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.jenness at gmail.com Sun Jun 30 12:07:08 2013 From: tim.jenness at gmail.com (Tim Jenness) Date: Sun, 30 Jun 2013 09:07:08 -0700 Subject: [AstroPy] Questions & Clarifications In-Reply-To: References: Message-ID: On Sun, Jun 30, 2013 at 8:46 AM, rahul .poruri wrote: > > > Clarification 2 - > I was unable to open spectra FITS files (from the SDSS DR9) on DS9. > Am i missing any feature or does DS9 not have the capability to open > spectra FITS files?! > If so, is there any other program i can use to open and *blink* through > the spectra?! > (other than fv by NASA's HEASARC) > DS9 is for images and cubes isn't it? Starlink SPLAT should be great for a collection of spectra. -- Tim Jenness -------------- next part -------------- An HTML attachment was scrubbed... URL: