From andrew.hearin at yale.edu Thu May 7 09:47:14 2015 From: andrew.hearin at yale.edu (Andrew Hearin) Date: Thu, 7 May 2015 09:47:14 -0400 Subject: [AstroPy] Using test classes with astropy testing framework Message-ID: <2513F6AA-AAFD-4BF7-99F5-C04F1B3F7187@yale.edu> Hi everyone, I?m using the Astropy package template to develop a package, and have thus far had an entirely painless experience building a test suite using the py.test settings that come pre-configured with the template. However, I am now trying to use python classes to construct a series of tests, and for some (almost undoubtedly simple) reason, py.test is not detecting my tests that appear within a class. In the following example, I have done my best to match the pattern used in the test_representation module that appears in the coordinates sub-package of astropy: def trivial_test(): assert 42 == 42 class DummyTestClass(object): def dummy_test(self): assert 42 == 43 Ok, so that?s my setup. Now here?s the problem. When I run "python setup.py test? from my sub-package root directory, the testing module in which the above code appears does indeed execute, but the result only registers as ?1 passed?, rather than ?1 failed, 1 passed?, meaning that dummy_test is not executing. How should I modify my test suite to run tests that are embedded in classes? In the examples I have looked at online, all of them seem to use a __main__ execution, but Astropy testing does not seem to proceed this way, and I would rather do what y?all do. Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From deil.christoph at googlemail.com Thu May 7 09:52:03 2015 From: deil.christoph at googlemail.com (Christoph Deil) Date: Thu, 7 May 2015 15:52:03 +0200 Subject: [AstroPy] Using test classes with astropy testing framework In-Reply-To: <2513F6AA-AAFD-4BF7-99F5-C04F1B3F7187@yale.edu> References: <2513F6AA-AAFD-4BF7-99F5-C04F1B3F7187@yale.edu> Message-ID: <3D5BC1E6-7988-4C21-92F3-8B96D7A38B3A@gmail.com> > On 07 May 2015, at 15:47, Andrew Hearin wrote: > > Hi everyone, > > I?m using the Astropy package template to develop a package, and have thus far had an entirely painless experience building a test suite using the py.test settings that come pre-configured with the template. However, I am now trying to use python classes to construct a series of tests, and for some (almost undoubtedly simple) reason, py.test is not detecting my tests that appear within a class. > > In the following example, I have done my best to match the pattern used in the test_representation module that appears in the coordinates sub-package of astropy: > > > def trivial_test(): > assert 42 == 42 > > class DummyTestClass(object): > > def dummy_test(self): > assert 42 == 43 > > > Ok, so that?s my setup. Now here?s the problem. When I run "python setup.py test? from my sub-package root directory, the testing module in which the above code appears does indeed execute, but the result only registers as ?1 passed?, rather than ?1 failed, 1 passed?, meaning that dummy_test is not executing. > > How should I modify my test suite to run tests that are embedded in classes? In the examples I have looked at online, all of them seem to use a __main__ execution, but Astropy testing does not seem to proceed this way, and I would rather do what y?all do. > > Cheers, > Andrew > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy You have to follow the pytest conventions for test discovery: http://pytest.org/latest/goodpractises.html#conventions-for-python-test-discovery I.e. start functions with `test_` and classes with `Test`. This works: def test_trivial(): assert 42 == 42 class TestDummyClass(object): def test_dummy(self): assert 42 == 43 Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.hearin at yale.edu Thu May 7 10:03:53 2015 From: andrew.hearin at yale.edu (Andrew Hearin) Date: Thu, 7 May 2015 10:03:53 -0400 Subject: [AstroPy] Using test classes with astropy testing framework In-Reply-To: <3D5BC1E6-7988-4C21-92F3-8B96D7A38B3A@gmail.com> References: <2513F6AA-AAFD-4BF7-99F5-C04F1B3F7187@yale.edu> <3D5BC1E6-7988-4C21-92F3-8B96D7A38B3A@gmail.com> Message-ID: <6B8004B9-3FE1-406D-8503-1E8D1233FEAA@yale.edu> Thanks, Christoph, that did the trick! On May 7, 2015, at 9:52 AM, Christoph Deil wrote: > >> On 07 May 2015, at 15:47, Andrew Hearin wrote: >> >> Hi everyone, >> >> I?m using the Astropy package template to develop a package, and have thus far had an entirely painless experience building a test suite using the py.test settings that come pre-configured with the template. However, I am now trying to use python classes to construct a series of tests, and for some (almost undoubtedly simple) reason, py.test is not detecting my tests that appear within a class. >> >> In the following example, I have done my best to match the pattern used in the test_representation module that appears in the coordinates sub-package of astropy: >> >> >> def trivial_test(): >> assert 42 == 42 >> >> class DummyTestClass(object): >> >> def dummy_test(self): >> assert 42 == 43 >> >> >> Ok, so that?s my setup. Now here?s the problem. When I run "python setup.py test? from my sub-package root directory, the testing module in which the above code appears does indeed execute, but the result only registers as ?1 passed?, rather than ?1 failed, 1 passed?, meaning that dummy_test is not executing. >> >> How should I modify my test suite to run tests that are embedded in classes? In the examples I have looked at online, all of them seem to use a __main__ execution, but Astropy testing does not seem to proceed this way, and I would rather do what y?all do. >> >> Cheers, >> Andrew >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > You have to follow the pytest conventions for test discovery: > http://pytest.org/latest/goodpractises.html#conventions-for-python-test-discovery > > I.e. start functions with `test_` and classes with `Test`. > > This works: > > > def test_trivial(): > assert 42 == 42 > > class TestDummyClass(object): > > def test_dummy(self): > assert 42 == 43 > > > Christoph > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=RLWB8rl3RSYWx6Wm8DLb-_R8alZVMzm0vZd1GeWw2EA&s=kxBB-3eJOCFP1EoxxGKLsCMpT_LHmardon2ymwxgaPI&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Fri May 8 03:51:11 2015 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Fri, 8 May 2015 09:51:11 +0200 Subject: [AstroPy] ANN: reproject v0.1 released! Message-ID: Hi everyone, I have just released the first stable version (v0.1) of the reproject package, which is an Astropy-affiliated package that provides a simple API to reproject data from one arbitrary world coordinate system (WCS) to another. Docs: http://reproject.readthedocs.org To install: pip install reproject The main features are: * Support for various algorithms: at the moment, the package supports reprojection by interpolation (nearest-neighbor, bilinear, etc.), as well as the Montage algorithm (which basically finds the overlap between pixels on the celestial sphere). * Support for easy reprojection to/from HEALPIX projections, using healpy as a backend * For the interpolation, support for reprojecting n-dimensional datasets (including spectral cubes) as long as they have two celestial axes, and as long as these are independent from the other axes. * A uniform and simple API for all the functionality * All reprojection functions return a footprint array that can be used to show the coverage of the input image in the WCS projection of the output image. Note that at the moment, the reprojection functions all assume that the images are in surface brightness units (so e.g. not counts). Future features will include: * Support for non-celestial world coordinate systems * Support for the drizzle algorithm * A function to find the area of all pixels in an image, which will facilitate the conversion from flux (e.g. counts) to surface brightness. * Mosaicking capabilities Please note that this is only the first release, so I recommend that you check the results carefully. If you run into any issues, or have ideas of features that should be implemented, please open an issue on GitHub: https://github.com/astrofrog/reproject/issues Thanks to everyone who has contributed to the package so far (https://github.com/astrofrog/reproject/graphs/contributors) and in particular thanks to Christoph Deil who has been very helpful in the last couple of weeks with reviewing and testing the package before release! If you are interested in getting involved with future development of the package, please let me know! Cheers, Tom From jmwalawender at gmail.com Sun May 10 05:24:49 2015 From: jmwalawender at gmail.com (Josh Walawender) Date: Sat, 9 May 2015 23:24:49 -1000 Subject: [AstroPy] Proper Motions in astropy.coordinates Message-ID: Hi all, Is there a method do calculations of proper motions in astropy.coordinates? ?It looks to me as if the SkyCoord has obstime and equinox properties but how would I transform from a coordinate defined for a particular obstime and equinox to another obstime (e.g. for the same equinox)? I just did this calculation in a brute force way (multiplying the proper motion in mas/yr by number of years and adding that angle to the ra and dec independently), but it felt like there should be a better way. thanks! Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.hearin at yale.edu Sun May 10 10:29:21 2015 From: andrew.hearin at yale.edu (Andrew Hearin) Date: Sun, 10 May 2015 10:29:21 -0400 Subject: [AstroPy] package template setup file Message-ID: <0968BA70-B746-4CA6-A1CE-99F533729CAB@yale.edu> Hi everyone, I have a question about the setup.py configurations in the astropy package template. In the arguments to ?setup?, the astropy package template defaults has a keyword argument for ?requires? as well as ?install_requires?. If I correctly understand the description on pythonhosted.org, the ?install_requires? argument will make sure that all previously uninstalled dependencies will automatically be pip installed upon call to "python setup.py install.? However, what is the independent purpose of the ?requires? argument? Under what circumstances would these arguments have different values? Cheers, Andrew From ghang.naoc at gmail.com Mon May 11 23:49:04 2015 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Tue, 12 May 2015 11:49:04 +0800 Subject: [AstroPy] Proper Motions in astropy.coordinates In-Reply-To: References: Message-ID: I have a similar question. Is there any other package about PM calculation? Thank you. Hang On Sun, May 10, 2015 at 5:24 PM, Josh Walawender wrote: > Hi all, > > Is there a method do calculations of proper motions in > astropy.coordinates? It looks to me as if the SkyCoord has obstime and > equinox properties but how would I transform from a coordinate defined for > a particular obstime and equinox to another obstime (e.g. for the same > equinox)? > > I just did this calculation in a brute force way (multiplying the proper > motion in mas/yr by number of years and adding that angle to the ra and dec > independently), but it felt like there should be a better way. > > thanks! > Josh > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Tue May 12 11:49:54 2015 From: embray at stsci.edu (Erik Bray) Date: Tue, 12 May 2015 11:49:54 -0400 Subject: [AstroPy] package template setup file In-Reply-To: <0968BA70-B746-4CA6-A1CE-99F533729CAB@yale.edu> References: <0968BA70-B746-4CA6-A1CE-99F533729CAB@yale.edu> Message-ID: <55522122.1010108@stsci.edu> On 05/10/2015 10:29 AM, Andrew Hearin wrote: > Hi everyone, > > I have a question about the setup.py configurations in the astropy package > template. In the arguments to ?setup?, the astropy package template defaults > has a keyword argument for ?requires? as well as ?install_requires?. If I > correctly understand the description on pythonhosted.org, the > ?install_requires? argument will make sure that all previously uninstalled > dependencies will automatically be pip installed upon call to "python > setup.py install.? However, what is the independent purpose of the ?requires? > argument? Under what circumstances would these arguments have different > values? Good question. I've made a little noise about this in the past but not enough to do anything about. To skip to the punchline: the requires argument to setup() should not be used *at all* and the example of its usage should be removed from the package-template IMO. Now for the longer version, the reasons for the dual existence of requires and install_requires is historical. The *requires* keyword is provided by the original distutils implementation (along with provides and obsoletes) to specify information about what Python *modules* (i.e. the name you actually import) a distribution requires to work: https://docs.python.org/2/distutils/setupscript.html#relationships-between-distributions-and-packages These were included in the original distutils metadata specification in anticipation of automated package management tools that might make use of them (but did not yet exist; at the time there was no setuptools, easy_install, pip, or even PyPI). At the end of the day, no tools that used these metadata keywords were ever implemented. Part of the reason for that is that it was not considered a robust way to specify requirements for a distribution. For example, say you had: setup( ... requires=['tools'], ... ) Well, what 'tools' module does that refer to? There is no top-level namespace of globally unique Python module names. This requirement could be satisfied if you ran $ touch tools.py before running the above setup.py. However, with the creation of PyPI (n?e The Cheeseshop) a sort of "global namespace" was created. However, the keys in that namespace are not module names, but *distribution* names. In many cases the name of a Python distribution coincide with the name of a (single) module included in that distribution. But that doesn't have to be the case--a single distribution can include multiple modules. For example the distribution named "setuptools" contains both the `setuptools` and `pkg_resources` modules. So when setuptools was created around the same time as PyPI (I can't remember which came first), it made more sense to add a setup() keyword that specifies unique distributions that are requirements for a distribution. Since requires=[...] was already taken, and had different semantics, setuptools added install_requires. In practice install_requires ended up being far more useful, and is the only field used to specify (Python) requirements of a Python distribution. Very few distributions still actually use requires=[...], and many that do use it incorrectly. In fact, the old requires field is deprecated in PEP 345: https://www.python.org/dev/peps/pep-0345/ However, most tools (i.e. setuptools) are not currently generating PEP 345-compatible metadata as a practical measure for backwards compatibility (this is changing, but very slowly), so it's not very obvious (there are no warnings) that these metadata fields will be deprecated, yet. But the community consensus (albeit confusing to someone who doesn't follow these issues, since the distutils docs haven't even been updated yet), is that these keywords (`requires`, `provides`, etc.) should not be used. The last time I could find that this was discussed extensively online was in this thread: https://mail.python.org/pipermail/distutils-sig/2013-October/022859.html However, after all that, it seems no immediate action ended up being taken (I think it was just harmless enough that it fell to the backburner). Nevertheless, it's clear that confusion about these keywords still exists. Erik From rowen at uw.edu Tue May 12 12:47:48 2015 From: rowen at uw.edu (Russell Owen) Date: Tue, 12 May 2015 09:47:48 -0700 Subject: [AstroPy] Proper Motions in astropy.coordinates In-Reply-To: References: Message-ID: It is not a hard computation, but I suggest working in spherical coordinates or using spherical trig for best results. If you want a ready made solution, I wrote two that might be of interest: - RO.Astro performs astronomical coordinate conversions, and handles proper motion, parallax and radial velocity. It is part of the RO package, which is pure python . pip install works. - CoordConv is much faster (C++ with Python wrappers), but has many dependencies, including slalib, so it takes more work to install. . I wrote this package for the Apache Point Observatory telescope control system. Both packages also include a spherical trig solver and convenient conversion between spherical and cartesian coordinates. -- Russell On May 11, 2015, at 8:49 PM, gonghang.naoc wrote: > I have a similar question. Is there any other package about PM calculation? > Thank you. > Hang > > On Sun, May 10, 2015 at 5:24 PM, Josh Walawender wrote: > Hi all, > > Is there a method do calculations of proper motions in astropy.coordinates? It looks to me as if the SkyCoord has obstime and equinox properties but how would I transform from a coordinate defined for a particular obstime and equinox to another obstime (e.g. for the same equinox)? > > I just did this calculation in a brute force way (multiplying the proper motion in mas/yr by number of years and adding that angle to the ra and dec independently), but it felt like there should be a better way. > > thanks! > Josh > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From adrianmpw at gmail.com Tue May 12 12:57:24 2015 From: adrianmpw at gmail.com (Adrian Price-Whelan) Date: Tue, 12 May 2015 12:57:24 -0400 Subject: [AstroPy] Proper Motions in astropy.coordinates In-Reply-To: References: Message-ID: Right now, there is no proper motion or velocity support in Astropy, but this is going to be the next major feature push for the coordinates subpackage. Thanks, Adrian On Mon, May 11, 2015 at 11:49 PM, gonghang.naoc wrote: > I have a similar question. Is there any other package about PM calculation? > Thank you. > Hang > > On Sun, May 10, 2015 at 5:24 PM, Josh Walawender > wrote: >> >> Hi all, >> >> Is there a method do calculations of proper motions in >> astropy.coordinates? It looks to me as if the SkyCoord has obstime and >> equinox properties but how would I transform from a coordinate defined for a >> particular obstime and equinox to another obstime (e.g. for the same >> equinox)? >> >> I just did this calculation in a brute force way (multiplying the proper >> motion in mas/yr by number of years and adding that angle to the ra and dec >> independently), but it felt like there should be a better way. >> >> thanks! >> Josh >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw From jmwalawender at gmail.com Wed May 13 17:02:36 2015 From: jmwalawender at gmail.com (Josh Walawender) Date: Wed, 13 May 2015 11:02:36 -1000 Subject: [AstroPy] Proper Motions in astropy.coordinates In-Reply-To: References: Message-ID: Glad to hear it. ?I would encourage support for handling uncertainties in both the initial position, the proper motion values, and the uncertainty in the calculated position. ?This would have been valuable to me in planning a recent observing run. ?I did some calculations for that run using the small angle approximation. ?I need to upgrade those to use proper spherical geometry. ?I may play around with doing my own subclassing of SkyCoord to add this. ?If I come up with something useful I?ll post it. cheers, Josh On May 12, 2015 at 6:57:29 AM, Adrian Price-Whelan (adrianmpw at gmail.com) wrote: Right now, there is no proper motion or velocity support in Astropy, but this is going to be the next major feature push for the coordinates subpackage. Thanks, Adrian On Mon, May 11, 2015 at 11:49 PM, gonghang.naoc wrote: > I have a similar question. Is there any other package about PM calculation? > Thank you. > Hang > > On Sun, May 10, 2015 at 5:24 PM, Josh Walawender > wrote: >> >> Hi all, >> >> Is there a method do calculations of proper motions in >> astropy.coordinates? It looks to me as if the SkyCoord has obstime and >> equinox properties but how would I transform from a coordinate defined for a >> particular obstime and equinox to another obstime (e.g. for the same >> equinox)? >> >> I just did this calculation in a brute force way (multiplying the proper >> motion in mas/yr by number of years and adding that angle to the ra and dec >> independently), but it felt like there should be a better way. >> >> thanks! >> Josh >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Adrian M. Price-Whelan ~ Columbia University ~ http://adrian.pw _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghang.naoc at gmail.com Thu May 14 05:52:00 2015 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Thu, 14 May 2015 17:52:00 +0800 Subject: [AstroPy] non-linear least square fitting in python Message-ID: Hi all, We can use scipy.optimize.leastsq to do least square fitting in python. However we need to make an initial guess. ( http://python4mpia.github.io/fitting_data/least-squares-fitting.html). So I am not sure we can get a global optimum. There is other simple way we can fit a *y=a*e^-(b*x)+c* curve? My data are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we can not make a log on both sides and use linear fitting any more. Thank you. best Hang -------------- next part -------------- An HTML attachment was scrubbed... URL: From cwp.paxson at gmail.com Thu May 14 06:08:46 2015 From: cwp.paxson at gmail.com (Charles Paxson) Date: Thu, 14 May 2015 06:08:46 -0400 Subject: [AstroPy] non-linear least square fitting in python In-Reply-To: References: Message-ID: Hi Hang, What if you formulated the equation y-c = a * e^-(b*x)? You solve for a, b using logs, then in another round of fitting you would solve for c only. Sincerely, Charles On Thu, May 14, 2015 at 5:52 AM, gonghang.naoc wrote: > Hi all, > We can use scipy.optimize.leastsq to do least square fitting in python. > However we need to make an initial guess. ( > http://python4mpia.github.io/fitting_data/least-squares-fitting.html). So > I am not sure we can get a global optimum. > > There is other simple way we can fit a *y=a*e^-(b*x)+c* curve? My data > are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we > can not make a log on both sides and use linear fitting any more. > > Thank you. > best > Hang > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yanbin.yang at obspm.fr Thu May 14 06:35:40 2015 From: yanbin.yang at obspm.fr (Yanbin YANG) Date: Thu, 14 May 2015 12:35:40 +0200 Subject: [AstroPy] non-linear least square fitting in python In-Reply-To: References: Message-ID: <31AF593D-605C-4333-BC44-9926559284BE@obspm.fr> Hi, Hang, perhaps you can try "MPFIT" when you have more complex fonction. I know its IDL version (or MPFITEXPR.pro) which works perfectly. Here is a python version, which I never used, but you can try. https://code.google.com/p/agpy/source/browse/trunk/mpfit/mpfit.py?r=399 regards, Yanbin. Le 14 mai 2015 ? 12:08, Charles Paxson a ?crit : > Hi Hang, > > What if you formulated the equation y-c = a * e^-(b*x)? You solve for a, b using logs, then in another round of fitting you would solve for c only. > > Sincerely, > > Charles > > On Thu, May 14, 2015 at 5:52 AM, gonghang.naoc wrote: > Hi all, > We can use scipy.optimize.leastsq to do least square fitting in python. However we need to make an initial guess. (http://python4mpia.github.io/fitting_data/least-squares-fitting.html). So I am not sure we can get a global optimum. > > There is other simple way we can fit a y=a*e^-(b*x)+c curve? My data are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we can not make a log on both sides and use linear fitting any more. > > Thank you. > best > Hang > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From aldcroft at head.cfa.harvard.edu Thu May 14 06:47:28 2015 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 14 May 2015 06:47:28 -0400 Subject: [AstroPy] non-linear least square fitting in python In-Reply-To: References: Message-ID: On Thu, May 14, 2015 at 5:52 AM, gonghang.naoc wrote: > Hi all, > We can use scipy.optimize.leastsq to do least square fitting in python. > However we need to make an initial guess. ( > http://python4mpia.github.io/fitting_data/least-squares-fitting.html). So > I am not sure we can get a global optimum. > Ensuring that you have a global optimum for a non-linear is generally a difficult problem. In your case though you have a reasonably simple fit function so that if you provide a decent initial guess then most optimizers should converge to something near the global optimum. You can make an initial guess in your case by taking 3 data points, at the beginning, middle, and end of your data. With those three data points you can analytically solve for a, b, and c. Then use those as the starting point. If your data are very noisy you might average a few. Cheers, Tom > > There is other simple way we can fit a *y=a*e^-(b*x)+c* curve? My data > are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we > can not make a log on both sides and use linear fitting any more. > > Thank you. > best > Hang > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matteo at matteobachetti.it Thu May 14 08:45:22 2015 From: matteo at matteobachetti.it (Matteo) Date: Thu, 14 May 2015 12:45:22 +0000 Subject: [AstroPy] non-linear least square fitting in python In-Reply-To: References: Message-ID: In scipy, there is a fitting algorithm that is called basinhopping. It's *far* slower than normal least-square fitting routines, but it's very helpful when you think you risk having local minima. http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.optimize.basinhopping.html If execution time is not an issue (datasets are sufficiently small), it's worth trying. Cheers, Matteo On Thu, May 14, 2015 at 12:47 PM Aldcroft, Thomas < aldcroft at head.cfa.harvard.edu> wrote: > On Thu, May 14, 2015 at 5:52 AM, gonghang.naoc > wrote: > >> Hi all, >> We can use scipy.optimize.leastsq to do least square fitting in python. >> However we need to make an initial guess. ( >> http://python4mpia.github.io/fitting_data/least-squares-fitting.html). >> So I am not sure we can get a global optimum. >> > > Ensuring that you have a global optimum for a non-linear is generally a > difficult problem. In your case though you have a reasonably simple fit > function so that if you provide a decent initial guess then most optimizers > should converge to something near the global optimum. > > You can make an initial guess in your case by taking 3 data points, at the > beginning, middle, and end of your data. With those three data points you > can analytically solve for a, b, and c. Then use those as the starting > point. If your data are very noisy you might average a few. > > Cheers, > Tom > > >> >> There is other simple way we can fit a *y=a*e^-(b*x)+c* curve? My data >> are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we >> can not make a log on both sides and use linear fitting any more. >> >> Thank you. >> best >> Hang >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghang.naoc at gmail.com Thu May 14 09:04:32 2015 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Thu, 14 May 2015 21:04:32 +0800 Subject: [AstroPy] non-linear least square fitting in python In-Reply-To: References: Message-ID: *Hi Tom,For y=a*e^(-b*x) , we can get an analytic solution. So it is not necessary to guess. For y=a*e^(-b*x)+c, we can not get an analytic solution, right?* *Hang* On Thu, May 14, 2015 at 6:47 PM, Aldcroft, Thomas < aldcroft at head.cfa.harvard.edu> wrote: > > > On Thu, May 14, 2015 at 5:52 AM, gonghang.naoc > wrote: > >> Hi all, >> We can use scipy.optimize.leastsq to do least square fitting in python. >> However we need to make an initial guess. ( >> http://python4mpia.github.io/fitting_data/least-squares-fitting.html). >> So I am not sure we can get a global optimum. >> > > Ensuring that you have a global optimum for a non-linear is generally a > difficult problem. In your case though you have a reasonably simple fit > function so that if you provide a decent initial guess then most optimizers > should converge to something near the global optimum. > > You can make an initial guess in your case by taking 3 data points, at the > beginning, middle, and end of your data. With those three data points you > can analytically solve for a, b, and c. Then use those as the starting > point. If your data are very noisy you might average a few. > > Cheers, > Tom > > >> >> There is other simple way we can fit a *y=a*e^-(b*x)+c* curve? My data >> are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we >> can not make a log on both sides and use linear fitting any more. >> >> Thank you. >> best >> Hang >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.gullikson at gmail.com Thu May 14 10:05:56 2015 From: kevin.gullikson at gmail.com (Kevin Gullikson) Date: Thu, 14 May 2015 09:05:56 -0500 Subject: [AstroPy] non-linear least square fitting in python In-Reply-To: References: Message-ID: A few thoughts: - As Tom mentioned, the function is simple enough that I would bet that scipy.optimize.leastsq is already finding the global minimum. - You could do a full grid search in parameters, and visualize the parameter space in terms of parameter vs chi^2. That way you can see where the global minimum is. - "chi by eye": A local minimum probably won't "look" right if you plot the solution curve on top of the data. If it looks right, it probably is. Kevin Gullikson On Thu, May 14, 2015 at 8:04 AM, gonghang.naoc wrote: > > > *Hi Tom,For y=a*e^(-b*x) , we can get an analytic solution. So it is not > necessary to guess. For y=a*e^(-b*x)+c, we can not get an analytic > solution, right?* > > *Hang* > > On Thu, May 14, 2015 at 6:47 PM, Aldcroft, Thomas < > aldcroft at head.cfa.harvard.edu> wrote: > >> >> >> On Thu, May 14, 2015 at 5:52 AM, gonghang.naoc >> wrote: >> >>> Hi all, >>> We can use scipy.optimize.leastsq to do least square fitting in python. >>> However we need to make an initial guess. ( >>> http://python4mpia.github.io/fitting_data/least-squares-fitting.html). >>> So I am not sure we can get a global optimum. >>> >> >> Ensuring that you have a global optimum for a non-linear is generally a >> difficult problem. In your case though you have a reasonably simple fit >> function so that if you provide a decent initial guess then most optimizers >> should converge to something near the global optimum. >> >> You can make an initial guess in your case by taking 3 data points, at >> the beginning, middle, and end of your data. With those three data points >> you can analytically solve for a, b, and c. Then use those as the starting >> point. If your data are very noisy you might average a few. >> >> Cheers, >> Tom >> >> >>> >>> There is other simple way we can fit a *y=a*e^-(b*x)+c* curve? My data >>> are Xn, Yn and ERRORn. A constant c makes the fitting difficult since we >>> can not make a log on both sides and use linear fitting any more. >>> >>> Thank you. >>> best >>> Hang >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Philip.Koreski at pcli.com Thu May 14 15:49:49 2015 From: Philip.Koreski at pcli.com (Philip Koreski) Date: Thu, 14 May 2015 19:49:49 +0000 Subject: [AstroPy] Astropy for imaging Message-ID: I'm writing an app for astrophotography. I need it to do image stacking and I'm looking for an algorithm that can not only blend the images but align each image so all stars are over each other. I'm also wandering about the 'fits' format, is there a way around it? My images need to be jpg's in and jpg's out so I'm wanting just to convert the image to a numpy array, process then convert back to jpg and save. I'm not sure if the overhead of converting to 'fits' is worthwhile. Also I was wanting to know if there is good way to run these algorithms on the GPU. Sorry for asking so many questions but I would appreciate any help that can be offered in this. Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Thu May 14 16:03:37 2015 From: stsci.perry at gmail.com (Perry Greenfield) Date: Thu, 14 May 2015 16:03:37 -0400 Subject: [AstroPy] Astropy for imaging In-Reply-To: References: Message-ID: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> Have you ever done this sort of thing before? I ask that since if the camera or telescope has significant field distortion, then aligning overlapping issues can be fairly involved in that you will need to characterize the distortion fairly well. Perhaps not if distortion is small. This is a common operation in astronomy, but usually not that trivial unless the shifts are fairly small. A good deal of effort often goes into getting the stacking right if distortions are significant and shifts are also significant (e.g., you are creating a mosaic of a larger field). That the images are in jpg format isn?t problem. PIL or similar can be used to convert to numpy arrays. If simple shifts work you can just shift images by integer numbers of pixels. The FITS file is an advantage if it has good distortion information (the WCS aspect of the file), but since you apparently don?t have that, it won?t help you. There are various programs to do this sort of stacking, though they don?t always worry about cosmetic effects at the boundaries of images (you may see noticeable edges depending on noise, flat fielding errors, etc). Perry On May 14, 2015, at 3:49 PM, Philip Koreski wrote: > I?m writing an app for astrophotography. I need it to do image stacking and I?m looking for an algorithm that can not only blend the images but align each image so all stars are over each other. I?m also wandering about the ?fits? format, is there a way around it? My images need to be jpg?s in and jpg?s out so I?m wanting just to convert the image to a numpy array, process then convert back to jpg and save. I?m not sure if the overhead of converting to ?fits? is worthwhile. Also I was wanting to know if there is good way to run these algorithms on the GPU. > Sorry for asking so many questions but I would appreciate any help that can be offered in this. > > Thank you! > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From te.pickering at gmail.com Thu May 14 16:28:07 2015 From: te.pickering at gmail.com (Timothy Pickering) Date: Thu, 14 May 2015 16:28:07 -0400 Subject: [AstroPy] Astropy for imaging In-Reply-To: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> References: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> Message-ID: <83F5021E-03EB-403F-8828-B8DCA5246604@gmail.com> > That the images are in jpg format isn?t problem. PIL or similar can be used to convert to numpy arrays. If simple shifts work you can just shift images by integer numbers of pixels. The FITS file is an advantage if it has good distortion information (the WCS aspect of the file), but since you apparently don?t have that, it won?t help you. > > There are various programs to do this sort of stacking, though they don?t always worry about cosmetic effects at the boundaries of images (you may see noticeable edges depending on noise, flat fielding errors, etc). if you want to calculate distortion solutions in WCS format for wide-field (> 5 arcmin) astrophotography, check out http://astrometry.net/. for single images or small samples, you can use their web service. for batch processing, you can install the software and index files locally. once you have the WCS solutions, you can use astropy and affiliated utilities to manipulate and combine the images. for smaller fields of view, something like http://deepskystacker.free.fr/english/index.html might be a simpler solution for you. tim ? +-------------------------------------------------------------------+ | T. E. Pickering, Ph.D. | Space Telescope Science Institute | | Systems Software Engineer | 3700 San Martin Dr. | | +1-520-305-9823 | Baltimore, MD, 21218, USA | | te.pickering at gmail.com | pickering at stsci.edu | +-------------------------------------------------------------------+ From Philip.Koreski at pcli.com Thu May 14 17:53:41 2015 From: Philip.Koreski at pcli.com (Philip Koreski) Date: Thu, 14 May 2015 21:53:41 +0000 Subject: [AstroPy] Astropy for imaging In-Reply-To: <83F5021E-03EB-403F-8828-B8DCA5246604@gmail.com> References: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> <83F5021E-03EB-403F-8828-B8DCA5246604@gmail.com> Message-ID: <84068e5ecd6848d3a6563dcd88993bfd@Exchange02.pcli.com> Yes the requirement for this app is for there to be minor adjustments made (1 to 2 second exposures for a max of like 5 minutes). I've done some tests with pretty good success using an algorithm for panoramic image stitching using the scikit-image package as detailed in the last example from this link: https://peerj.com/articles/453/. It works pretty well for this purpose but the problem I've been having with it is its speed of execution. I've been thinking of different ways to optimize the process and one was to make only adjustments on the X and Y axis as I assumed that unless I was tracking an object for a considerable amount of time skewing and rotating wouldn't be needed. But if this is an issue maybe I would have to go through those processes anyway and should stick with scikit-image. I was wanting to know about the GPU and astropy , is it possible? With scikit-image it doesn't look like it is. I hate to ask too much but I haven't been able to find good material for coming up with image stacking algorithms of my own, would anybody be able to point me to some material that covers that in detail, preferably with Python examples? > There are various programs to do this sort of stacking, though they don't always worry about cosmetic effects at the boundaries of images (you may see noticeable edges depending on noise, flat fielding errors, etc). Thanks for the suggestions but I actually need to write my own image stacking software. -----Original Message----- From: astropy-bounces at scipy.org [mailto:astropy-bounces at scipy.org] On Behalf Of Timothy Pickering Sent: Thursday, May 14, 2015 1:28 PM To: Astronomical Python mailing list Subject: Re: [AstroPy] Astropy for imaging > That the images are in jpg format isn't problem. PIL or similar can be used to convert to numpy arrays. If simple shifts work you can just shift images by integer numbers of pixels. The FITS file is an advantage if it has good distortion information (the WCS aspect of the file), but since you apparently don't have that, it won't help you. > > There are various programs to do this sort of stacking, though they don't always worry about cosmetic effects at the boundaries of images (you may see noticeable edges depending on noise, flat fielding errors, etc). if you want to calculate distortion solutions in WCS format for wide-field (> 5 arcmin) astrophotography, check out http://astrometry.net/. for single images or small samples, you can use their web service. for batch processing, you can install the software and index files locally. once you have the WCS solutions, you can use astropy and affiliated utilities to manipulate and combine the images. for smaller fields of view, something like http://deepskystacker.free.fr/english/index.html might be a simpler solution for you. tim - +-------------------------------------------------------------------+ | T. E. Pickering, Ph.D. | Space Telescope Science Institute | | Systems Software Engineer | 3700 San Martin Dr. | | +1-520-305-9823 | Baltimore, MD, 21218, USA | | te.pickering at gmail.com | pickering at stsci.edu | +-------------------------------------------------------------------+ _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy From rays at blue-cove.com Fri May 15 13:58:38 2015 From: rays at blue-cove.com (R Schumacher) Date: Fri, 15 May 2015 10:58:38 -0700 Subject: [AstroPy] Astropy for imaging In-Reply-To: <84068e5ecd6848d3a6563dcd88993bfd@Exchange02.pcli.com> References: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> <83F5021E-03EB-403F-8828-B8DCA5246604@gmail.com> <84068e5ecd6848d3a6563dcd88993bfd@Exchange02.pcli.com> Message-ID: <201505151758.t4FHwvDv016684@blue-cove.com> At 02:53 PM 5/14/2015, you wrote: > I hate to ask too much but I haven't been able to find good material for c > oming up with image stacking algorithms of my own, would anybody > be able to point me to some material that covers that in detail, > preferably with Python examples? Take a look at Xie's Matlab work, which I translated to Python http://rjs.org/astro/1004x/Python/register/ and another paper for log-polar registration (ie scale/shift) https://etd.ohiolink.edu/rws_etd/document/get/osu1236610454/inline Jan Kybic http://cmp.felk.cvut.cz/~kybic/ has also published advanced registration methods, some very recently. His older "image registration by warping" code is archived at https://web.archive.org/web/20041209143140/http://cmp.felk.cvut.cz/~kybic/thesis/jkpyreg-0.2.tgz https://github.com/KitwareMedical/pyLAR http://www0.cs.ucl.ac.uk/opensource_mia_ws_2012/links.html#registration Ray Schumacher Programmer/Consultant -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- """ ; Authors: ; Hongjie Xie, Nigel Hicks, G. Randy Keller, Haitao Huang, Vladik Kreinovich ; ; Title of the paper: ; An IDL/ENVI implementation of the FFT Based Algorithm for Automatic Image Registration ; IDL source code for shift only ; ---------------------------------------------------------------------------- """ import numarray import numarray.fft as fft #import numarray.nd_image as nd_image #import math import string import sys def SizeImage(FileName): hdr_lun1 = open(FileName+'.hdr', 'r') info = hdr_lun1.readlines() hdr_lun1.close() samples = int(string.split(info[3])[2]) lines = int(string.split(info[4])[2]) return (samples, lines) def LoadImage(FileName, samples, lines): ## cols=samples, rows=lines data_lun1 = open(FileName+'.img', 'rb') IArray = numarray.fromstring(data_lun1.read(), numarray.UInt8, shape=(lines, samples)).astype(numarray.Float) data_lun1.close() return numarray.transpose(IArray) ##===================================================================== ## ## MAIN ## ##===================================================================== print 'Loading images...' FileName = 't350380ori' #FileName = '6cx7r' cols, rows = SizeImage(FileName) FFTnorm = complex(cols*rows, 0) print 'cols=', cols ,'rows=',rows Im1 = LoadImage(FileName, cols, rows) FileName = 't350380shf' #FileName = '6cx7r' cols, rows = SizeImage(FileName) Im2 = LoadImage(FileName, cols, rows) print Im1[0,0], Im1[1,0], Im1[0,1], Im1[cols/2,rows/2] print '' print 'Doing FFT on two images...' Ffft_im1=fft.fft2d(Im1)/FFTnorm Ffft_im2=fft.fft2d(Im2)/FFTnorm print Ffft_im1[0,0], Ffft_im1[cols/2,rows/2] print 'doing the ratio' R1= (Ffft_im1 * Ffft_im2.conjugate()) print R1[0,0], R1[1,0], R1[cols/2,rows/2] R2= (abs(Ffft_im1) * abs(Ffft_im2)) print R2[0,0], R2[cols/2,rows/2] R = R1 / R2 print R[0,0], R[cols/2,rows/2] print 'doing inverse_fft2d' IR=fft.inverse_fft2d(R) * FFTnorm print IR[0,0], IR[cols/2,rows/2] flat_IR = numarray.ravel(numarray.transpose(IR)) print flat_IR.type() I= numarray.argmax(flat_IR) maxn=flat_IR[I] IX= I % cols IY= I / cols X=IX Y=IY print 'I=', I, 'X=', X, 'Y=', Y if((X > 0.5*cols) and ( Y > 0.5*rows)): X=X-cols Y=Y-rows IX=X IY=Y elif(X > 0.5*cols): X=X-cols IX=X IY=IY elif(Y > 0.5*rows): Y=Y-rows IX=IX IY=Y else: IX=IX IY=IY print 'this is max:', maxn print 'position: x=', IX, ', y=', IY From Philip.Koreski at pcli.com Fri May 15 14:29:07 2015 From: Philip.Koreski at pcli.com (Philip Koreski) Date: Fri, 15 May 2015 18:29:07 +0000 Subject: [AstroPy] Astropy for imaging In-Reply-To: <201505151758.t4FHwvDv016684@blue-cove.com> References: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> <83F5021E-03EB-403F-8828-B8DCA5246604@gmail.com> <84068e5ecd6848d3a6563dcd88993bfd@Exchange02.pcli.com> <201505151758.t4FHwvDv016684@blue-cove.com> Message-ID: Man, I can't thank you enough for all those links and the sample code. If I replace the file extensions '.hdr' and '.img' in the sample code with '.jpg' or '.png' will it still work? Thanks again! From: astropy-bounces at scipy.org [mailto:astropy-bounces at scipy.org] On Behalf Of R Schumacher Sent: Friday, May 15, 2015 10:59 AM To: Astronomical Python mailing list Subject: Re: [AstroPy] Astropy for imaging At 02:53 PM 5/14/2015, you wrote: I hate to ask too much but I haven't been able to find good material for c oming up with image stacking algorithms of my own, would anybody be able to point me to some material that covers that in detail, preferably with Python examples? Take a look at Xie's Matlab work, which I translated to Python http://rjs.org/astro/1004x/Python/register/ and another paper for log-polar registration (ie scale/shift) https://etd.ohiolink.edu/rws_etd/document/get/osu1236610454/inline Jan Kybic http://cmp.felk.cvut.cz/~kybic/ has also published advanced registration methods, some very recently. His older "image registration by warping" code is archived at https://web.archive.org/web/20041209143140/http://cmp.felk.cvut.cz/~kybic/thesis/jkpyreg-0.2.tgz https://github.com/KitwareMedical/pyLAR http://www0.cs.ucl.ac.uk/opensource_mia_ws_2012/links.html#registration Ray Schumacher Programmer/Consultant -------------- next part -------------- An HTML attachment was scrubbed... URL: From rays at blue-cove.com Fri May 15 15:13:12 2015 From: rays at blue-cove.com (R Schumacher) Date: Fri, 15 May 2015 12:13:12 -0700 Subject: [AstroPy] Astropy for imaging In-Reply-To: References: <4F401000-CC26-42BB-BEB8-537C9225999D@gmail.com> <83F5021E-03EB-403F-8828-B8DCA5246604@gmail.com> <84068e5ecd6848d3a6563dcd88993bfd@Exchange02.pcli.com> <201505151758.t4FHwvDv016684@blue-cove.com> Message-ID: <201505151913.t4FJDGto020654@blue-cove.com> The samples' .hdr is a text header/description, and .img are raw byte files; ie, 1.img is samples = 100 lines = 100 bands = 1 a 100x100 pixel, one byte per pixel http://www.exelisvis.com/docs/ENVIImageFiles.html Xie's paper: http://www.utsa.edu/lrsg/Paper/Xie-registration.pdf - Ray At 11:29 AM 5/15/2015, you wrote: >Content-Language: en-US >Content-Type: multipart/alternative; > boundary="_000_e54ae0496c524f46abf54c9b8dfa7920Exchange02pclicom_" > >Man, I can't thank you enough for all those links and the sample >code. If I replace the file extensions '.hdr' and '.img' in the >sample code with '.jpg' or '.png' will it still work? > >Thanks again! > >From: astropy-bounces at scipy.org [mailto:astropy-bounces at scipy.org] >On Behalf Of R Schumacher >Sent: Friday, May 15, 2015 10:59 AM >To: Astronomical Python mailing list >Subject: Re: [AstroPy] Astropy for imaging > >At 02:53 PM 5/14/2015, you wrote: > > I hate to ask too much but I haven't been able to find good material for c > oming up with image stacking algorithms of my own, would anybody > be able to point me to some material that covers that in detail, > preferably with Python examples? > > >Take a look at Xie's Matlab work, which I translated to Python >http://rjs.org/astro/1004x/Python/register/ >and another paper for log-polar registration (ie scale/shift) >https://etd.ohiolink.edu/rws_etd/document/get/osu1236610454/inline > >Jan Kybic >http://cmp.felk.cvut.cz/~kybic/ >has also published advanced registration methods, some very >recently. His older "image registration by warping" code is archived at >https://web.archive.org/web/20041209143140/http://cmp.felk.cvut.cz/~kybic/thesis/jkpyreg-0.2.tgz > > >https://github.com/KitwareMedical/pyLAR > >http://www0.cs.ucl.ac.uk/opensource_mia_ws_2012/links.html#registration > > > > >Ray Schumacher >Programmer/Consultant >_______________________________________________ >AstroPy mailing list >AstroPy at scipy.org >http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From leo.p.singer at nasa.gov Sat May 16 16:06:01 2015 From: leo.p.singer at nasa.gov (Singer, Leo P. (GSFC-661.0)[OAK RIDGE ASSOCIATED UNIVERSITIES (ORAU)]) Date: Sat, 16 May 2015 20:06:01 +0000 Subject: [AstroPy] Querying HST archive Message-ID: Hi, Is there any Python tool (maybe in Astroquery?) for searching and retrieving images from the HST archive? Thanks, Leo P. Singer NASA Postdoctoral Program Fellow Goddard Space Flight Center From erwin at mpe.mpg.de Sat May 16 16:41:30 2015 From: erwin at mpe.mpg.de (Peter Erwin) Date: Sat, 16 May 2015 22:41:30 +0200 Subject: [AstroPy] Querying HST archive In-Reply-To: References: Message-ID: My archive-searching tool telarchive will search the HST archive, among others, for images and spectra within a specified radius of a position (or object), but it doesn?t retrieve anything. http://www.mpe.mpg.de/~erwin/code/ cheers, Peter > On May 16, 2015, at 10:06 PM, Singer, Leo P. (GSFC-661.0)[OAK RIDGE ASSOCIATED UNIVERSITIES (ORAU)] wrote: > > Hi, > > Is there any Python tool (maybe in Astroquery?) for searching and retrieving images from the HST archive? > > Thanks, > Leo P. Singer > NASA Postdoctoral Program Fellow > Goddard Space Flight Center > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)176 2481 7713 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin From adam.g.ginsburg at gmail.com Sun May 17 09:10:47 2015 From: adam.g.ginsburg at gmail.com (Adam Ginsburg) Date: Sun, 17 May 2015 15:10:47 +0200 Subject: [AstroPy] Querying HST archive In-Reply-To: References: Message-ID: Hi Leo, Astroquery can't do this yet. It would be a great feature to add if anyone wanted to try. We have an open wishlist: https://github.com/astropy/astroquery/issues/226 that includes MAST, and the CADC archive: https://github.com/astropy/astroquery/issues/516 would also give access to HST data if implemented. On Sat, May 16, 2015 at 10:06 PM, Singer, Leo P. (GSFC-661.0)[OAK RIDGE ASSOCIATED UNIVERSITIES (ORAU)] wrote: > Hi, > > Is there any Python tool (maybe in Astroquery?) for searching and retrieving images from the HST archive? > > Thanks, > Leo P. Singer > NASA Postdoctoral Program Fellow > Goddard Space Flight Center > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Adam Ginsburg Fellow, European Southern Observatory http://www.adamgginsburg.com/ From jebaldwin at gmail.com Sun May 17 10:11:12 2015 From: jebaldwin at gmail.com (Jennifer Baldwin) Date: Mon, 18 May 2015 00:11:12 +1000 Subject: [AstroPy] Astropy and large VOTable files Message-ID: Hi all, I was trying to find an answer to this but could not. I am wondering if parse_single_table will attempt to read an entire VOTable file? Or if it will operate the same way as for FITS files so that when you slice the returned data array, it only loads the part it needs into memory? I'm concerned with how it will perform with extremely large xml files, but could not find a direct answer anywhere in the documentation. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Mon May 18 14:00:59 2015 From: mdroe at stsci.edu (Michael Droettboom) Date: Mon, 18 May 2015 14:00:59 -0400 Subject: [AstroPy] Astropy and large VOTable files In-Reply-To: References: Message-ID: <555A28DB.5000207@stsci.edu> Thanks for the question. Unfortunately, it will read the entire file into memory each time. It does read it in as a Numpy array, so the memory used should generally be less than the space on disk, however, depending on the content. XML doesn't really support the kind of slicing that FITS (or another binary format) can, because you can't know how big something is (or even what it is!) without parsing the whole file. That said, given the constraint of the file format, minimal memory usage is one of the main design features of astropy.io.votable, so I'd recommend trying it on large files and seeing how it goes. It shouldn't ever take significantly more memory than a binary array of data, i.e. the same as the equivalent FITS file loaded entirely into memory. Cheers, Mike On 05/17/2015 10:11 AM, Jennifer Baldwin wrote: > Hi all, > > I was trying to find an answer to this but could not. I am wondering > if parse_single_table will attempt to read an entire VOTable file? Or > if it will operate the same way as for FITS files so that when you > slice the returned data array, it only loads the part it needs into > memory? I'm concerned with how it will perform with extremely large > xml files, but could not find a direct answer anywhere in the > documentation. > > Thanks! > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.hearin at yale.edu Mon May 18 14:09:44 2015 From: andrew.hearin at yale.edu (Andrew Hearin) Date: Mon, 18 May 2015 14:09:44 -0400 Subject: [AstroPy] Astropy and large VOTable files In-Reply-To: <555A28DB.5000207@stsci.edu> References: <555A28DB.5000207@stsci.edu> Message-ID: <40C5B651-04C8-4E78-ABD3-F68E8A3195C3@yale.edu> Being able to read large data in chunks, make cuts on the chunks, and return a table of rows that pass the cuts is a pretty common data mining task that I think would be good to include in Astropy. I?m happy to (re-)raise a GitHub issue for this purpose, and contribute some code, but first: Jennifer, this is the functionality you are describing, right? If so: Mike, do you see any fundamental obstacles with this? On May 18, 2015, at 2:00 PM, Michael Droettboom wrote: > Thanks for the question. > > Unfortunately, it will read the entire file into memory each time. It does read it in as a Numpy array, so the memory used should generally be less than the space on disk, however, depending on the content. > > XML doesn't really support the kind of slicing that FITS (or another binary format) can, because you can't know how big something is (or even what it is!) without parsing the whole file. That said, given the constraint of the file format, minimal memory usage is one of the main design features of astropy.io.votable, so I'd recommend trying it on large files and seeing how it goes. It shouldn't ever take significantly more memory than a binary array of data, i.e. the same as the equivalent FITS file loaded entirely into memory. > > Cheers, > Mike > > On 05/17/2015 10:11 AM, Jennifer Baldwin wrote: >> Hi all, >> >> I was trying to find an answer to this but could not. I am wondering if parse_single_table will attempt to read an entire VOTable file? Or if it will operate the same way as for FITS files so that when you slice the returned data array, it only loads the part it needs into memory? I'm concerned with how it will perform with extremely large xml files, but could not find a direct answer anywhere in the documentation. >> >> Thanks! >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=fqrZPrNFrzwqmHSxKJ-shiCsIXJN8_SWmuwg5yOr9sA&s=m6R7fy7bDIllNOJ0BaVKj5GdN1j87_QtxcNSxOty56I&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Mon May 18 16:29:57 2015 From: mdroe at stsci.edu (Michael Droettboom) Date: Mon, 18 May 2015 16:29:57 -0400 Subject: [AstroPy] Astropy and large VOTable files In-Reply-To: <40C5B651-04C8-4E78-ABD3-F68E8A3195C3@yale.edu> References: <555A28DB.5000207@stsci.edu> <40C5B651-04C8-4E78-ABD3-F68E8A3195C3@yale.edu> Message-ID: <555A4BC5.5080004@stsci.edu> I understood Jennifer's question to be specific to the VOTable XML file format and the problems are really specific to parsing XML. In iterator/streaming interface could probably be built on top of it, however. Arbitrary (random access) slicing isn't really possible with XML, though. Mike On 05/18/2015 02:09 PM, Andrew Hearin wrote: > Being able to read large data in chunks, make cuts on the chunks, and > return a table of rows that pass the cuts is a pretty common data > mining task that I think would be good to include in Astropy. I?m > happy to (re-)raise a GitHub issue for this purpose, and contribute > some code, but first: Jennifer, this is the functionality you are > describing, right? If so: Mike, do you see any fundamental obstacles > with this? > > > > On May 18, 2015, at 2:00 PM, Michael Droettboom > wrote: > >> Thanks for the question. >> >> Unfortunately, it will read the entire file into memory each time. >> It does read it in as a Numpy array, so the memory used should >> generally be less than the space on disk, however, depending on the >> content. >> >> XML doesn't really support the kind of slicing that FITS (or another >> binary format) can, because you can't know how big something is (or >> even what it is!) without parsing the whole file. That said, given >> the constraint of the file format, minimal memory usage is one of the >> main design features of astropy.io.votable, so I'd recommend trying >> it on large files and seeing how it goes. It shouldn't ever take >> significantly more memory than a binary array of data, i.e. the same >> as the equivalent FITS file loaded entirely into memory. >> >> Cheers, >> Mike >> >> On 05/17/2015 10:11 AM, Jennifer Baldwin wrote: >>> Hi all, >>> >>> I was trying to find an answer to this but could not. I am wondering >>> if parse_single_table will attempt to read an entire VOTable file? >>> Or if it will operate the same way as for FITS files so that when >>> you slice the returned data array, it only loads the part it needs >>> into memory? I'm concerned with how it will perform with extremely >>> large xml files, but could not find a direct answer anywhere in the >>> documentation. >>> >>> Thanks! >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=fqrZPrNFrzwqmHSxKJ-shiCsIXJN8_SWmuwg5yOr9sA&s=m6R7fy7bDIllNOJ0BaVKj5GdN1j87_QtxcNSxOty56I&e= >> > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From mcraig at mnstate.edu Mon May 18 19:31:01 2015 From: mcraig at mnstate.edu (Matthew Craig) Date: Mon, 18 May 2015 23:31:01 +0000 Subject: [AstroPy] [ANN] ccdproc v0.3.1: CCD data reduction Message-ID: <58AF22FE-B00B-45CF-AC40-6E8B881B57D7@mnstate.edu> Dear colleagues, We are pleased to announce the release of ccdproc v0.3.1. This patch release fixes a couple of minor bugs and adds several new tests, but adds no new functionality. Ccdproc is is an affiliated package for the AstroPy package for basic data reductions of CCD images. More Information and repository https://github.com/astropy/ccdproc Documentation http://ccdproc.readthedocs.org/en/latest/ Installation http://ccdproc.readthedocs.org/en/latest/ccdproc/install.html Feedback, contributions, and comments are welcome. We would especially be interested in contributions of examples of using ccdproc and comparisons to other reduction methods. Special thanks to new contributors Jennifer Karr (@JenniferKarr) and Nathan Walker (@walkerna22) for their work on this release! Cheers, Steve Crawford and Matt Craig -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinberoiz at gmail.com Thu May 28 15:48:35 2015 From: martinberoiz at gmail.com (Martin Beroiz) Date: Thu, 28 May 2015 14:48:35 -0500 Subject: [AstroPy] Check if header contains WCS Message-ID: <685C56F1-F032-48D9-A67D-2DD94CDE1467@gmail.com> Hello, Is there a way of checking if a header has WCS info? Right now I?m using try: the_wcs = wcs.WCS(header_in) except WcsError: print("No WCS information found in header?) DoesImageHaveWCS = False But I wonder if WcsError will catch all exceptions regarding WCS headers. Thanks, Martin. From martinberoiz at gmail.com Thu May 28 17:31:19 2015 From: martinberoiz at gmail.com (Martin Beroiz) Date: Thu, 28 May 2015 16:31:19 -0500 Subject: [AstroPy] Check if header contains WCS In-Reply-To: <685C56F1-F032-48D9-A67D-2DD94CDE1467@gmail.com> References: <685C56F1-F032-48D9-A67D-2DD94CDE1467@gmail.com> Message-ID: Actually I just checked that trying to create a WCS from a header that contains none, doesn?t throw an exception. > On May 28, 2015, at 2:48 PM, Martin Beroiz wrote: > > Hello, > > Is there a way of checking if a header has WCS info? > Right now I?m using > > try: > the_wcs = wcs.WCS(header_in) > except WcsError: > print("No WCS information found in header?) > DoesImageHaveWCS = False > > But I wonder if WcsError will catch all exceptions regarding WCS headers. > > Thanks, > Martin. > From mcraig at mnstate.edu Thu May 28 21:38:42 2015 From: mcraig at mnstate.edu (Matthew Craig) Date: Fri, 29 May 2015 01:38:42 +0000 Subject: [AstroPy] Check if header contains WCS In-Reply-To: References: <685C56F1-F032-48D9-A67D-2DD94CDE1467@gmail.com> Message-ID: <646120B9-00B6-4E9E-B8A9-9D2239B64187@mnstate.edu> Hi Martin, You can see how we handle creating the WCS object in ccdproc at https://github.com/astropy/ccdproc/blob/master/ccdproc/ccddata.py#L401 Matt Craig schedule: http://physics.mnstate.edu/craig ?? Professor Department of Physics and Astronomy Minnesota State University Moorhead 1104 7th Ave S, Moorhead MN 56563 phone: (218) 477-2439 fax: (218) 477-2290 On May 28, 2015, at 4:31 PM, Martin Beroiz > wrote: Actually I just checked that trying to create a WCS from a header that contains none, doesn?t throw an exception. On May 28, 2015, at 2:48 PM, Martin Beroiz > wrote: Hello, Is there a way of checking if a header has WCS info? Right now I?m using try: the_wcs = wcs.WCS(header_in) except WcsError: print("No WCS information found in header?) DoesImageHaveWCS = False But I wonder if WcsError will catch all exceptions regarding WCS headers. Thanks, Martin. _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinberoiz at gmail.com Fri May 29 17:08:36 2015 From: martinberoiz at gmail.com (Martin Beroiz) Date: Fri, 29 May 2015 16:08:36 -0500 Subject: [AstroPy] Check if header contains WCS In-Reply-To: <646120B9-00B6-4E9E-B8A9-9D2239B64187@mnstate.edu> References: <685C56F1-F032-48D9-A67D-2DD94CDE1467@gmail.com> <646120B9-00B6-4E9E-B8A9-9D2239B64187@mnstate.edu> Message-ID: <8D3C5442-7A20-4378-88C7-76C9F745F205@gmail.com> Thanks a lot, that actually fixed my problem :) > On May 28, 2015, at 8:38 PM, Matthew Craig wrote: > > Hi Martin, > > You can see how we handle creating the WCS object in ccdproc at https://github.com/astropy/ccdproc/blob/master/ccdproc/ccddata.py#L401 > > Matt Craig > > schedule: http://physics.mnstate.edu/craig > ?? > > Professor > Department of Physics and Astronomy > Minnesota State University Moorhead > 1104 7th Ave S, Moorhead MN 56563 > > phone: (218) 477-2439 > fax: (218) 477-2290 > >> On May 28, 2015, at 4:31 PM, Martin Beroiz > wrote: >> >> Actually I just checked that trying to create a WCS from a header that contains none, doesn?t throw an exception. >> >> >>> On May 28, 2015, at 2:48 PM, Martin Beroiz > wrote: >>> >>> Hello, >>> >>> Is there a way of checking if a header has WCS info? >>> Right now I?m using >>> >>> try: >>> the_wcs = wcs.WCS(header_in) >>> except WcsError: >>> print("No WCS information found in header?) >>> DoesImageHaveWCS = False >>> >>> But I wonder if WcsError will catch all exceptions regarding WCS headers. >>> >>> Thanks, >>> Martin. >>> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: