From deil.christoph at googlemail.com Mon Mar 3 10:01:09 2014 From: deil.christoph at googlemail.com (Christoph Deil) Date: Mon, 3 Mar 2014 16:01:09 +0100 Subject: [AstroPy] Hubble deep field as JPEG or PNG with AVM info anyone? Message-ID: <98CAA848-BCB2-4D4C-8175-E7665AE6C8F8@gmail.com> Hi, as part of a pull request for blob detection [1], the scikit-image guys decided to add a Hubble deep field image [2] to their `skimage.data` example images [3] to their repository, and we?re trying to find the best file to choose. The ideal file would be: 1. an RGB image in PNG or JPEG format 2. not too large (say 100 kB to 500 kB max) 3. include AVM info [4] 4. have a public, permanent URL as a reference Does someone know where to a file in this format? Cheers, Christoph [1] https://github.com/scikit-image/scikit-image/pull/900 [2] http://en.wikipedia.org/wiki/Hubble_Deep_Field [3] http://scikit-image.org/docs/dev/api/skimage.data.html [4] http://en.wikipedia.org/wiki/Astronomy_Visualization_Metadata From sosey at stsci.edu Mon Mar 3 10:06:49 2014 From: sosey at stsci.edu (Megan Sosey) Date: Mon, 3 Mar 2014 15:06:49 +0000 Subject: [AstroPy] Hubble deep field as JPEG or PNG with AVM info anyone? In-Reply-To: <98CAA848-BCB2-4D4C-8175-E7665AE6C8F8@gmail.com> References: <98CAA848-BCB2-4D4C-8175-E7665AE6C8F8@gmail.com> Message-ID: <8C2C7C07-34A0-4594-B184-47A48CDB2AA2@stsci.edu> I don't know about avm info, but check out the available images on hubblesite.org Sent from my iPhone On Mar 3, 2014, at 10:01 AM, "Christoph Deil" wrote: > Hi, > > as part of a pull request for blob detection [1], the scikit-image guys decided to add a Hubble deep field image [2] to their `skimage.data` example images [3] to their repository, and we?re trying to find the best file to choose. > > The ideal file would be: > 1. an RGB image in PNG or JPEG format > 2. not too large (say 100 kB to 500 kB max) > 3. include AVM info [4] > 4. have a public, permanent URL as a reference > > Does someone know where to a file in this format? > > Cheers, > Christoph > > > [1] https://github.com/scikit-image/scikit-image/pull/900 > [2] http://en.wikipedia.org/wiki/Hubble_Deep_Field > [3] http://scikit-image.org/docs/dev/api/skimage.data.html > [4] http://en.wikipedia.org/wiki/Astronomy_Visualization_Metadata > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From august.fly at gmail.com Mon Mar 3 10:07:23 2014 From: august.fly at gmail.com (August (Gus) Muench) Date: Mon, 3 Mar 2014 10:07:23 -0500 Subject: [AstroPy] Hubble deep field as JPEG or PNG with AVM info anyone? In-Reply-To: <98CAA848-BCB2-4D4C-8175-E7665AE6C8F8@gmail.com> References: <98CAA848-BCB2-4D4C-8175-E7665AE6C8F8@gmail.com> Message-ID: All recent Chandra EPO release images fit that criteria (well, very close to the kB limit): e.g., the most recent one: http://chandra.harvard.edu/photo/2014/igrj11014/ http://chandra.harvard.edu/photo/2014/igrj11014/igrj11014.jpg [700kB] perhaps this one comes in under the limit: http://chandra.harvard.edu/photo/2013/ngc2392/ http://chandra.harvard.edu/photo/2013/ngc2392/ngc2392.jpg [498kB] - gus On Mon, Mar 3, 2014 at 10:01 AM, Christoph Deil wrote: > Hi, > > as part of a pull request for blob detection [1], the scikit-image guys decided to add a Hubble deep field image [2] to their `skimage.data` example images [3] to their repository, and we?re trying to find the best file to choose. > > The ideal file would be: > 1. an RGB image in PNG or JPEG format > 2. not too large (say 100 kB to 500 kB max) > 3. include AVM info [4] > 4. have a public, permanent URL as a reference > > Does someone know where to a file in this format? > > Cheers, > Christoph > > > [1] https://github.com/scikit-image/scikit-image/pull/900 > [2] http://en.wikipedia.org/wiki/Hubble_Deep_Field > [3] http://scikit-image.org/docs/dev/api/skimage.data.html > [4] http://en.wikipedia.org/wiki/Astronomy_Visualization_Metadata > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- August (Gus) Muench august.fly at gmail.com @augustmuench From deil.christoph at googlemail.com Mon Mar 3 10:19:30 2014 From: deil.christoph at googlemail.com (Christoph Deil) Date: Mon, 3 Mar 2014 16:19:30 +0100 Subject: [AstroPy] Hubble deep field as JPEG or PNG with AVM info anyone? In-Reply-To: <8C2C7C07-34A0-4594-B184-47A48CDB2AA2@stsci.edu> References: <98CAA848-BCB2-4D4C-8175-E7665AE6C8F8@gmail.com> <8C2C7C07-34A0-4594-B184-47A48CDB2AA2@stsci.edu> Message-ID: <981015C0-B0C3-416A-9A64-0C00B48C4F79@gmail.com> On 03 Mar 2014, at 16:06, Megan Sosey wrote: > I don't know about avm info, but check out the available images on hubblesite.org Hi Megan, thanks for the tip ? found one that fits all criteria ? it even contains AVM info! http://hubblesite.org/newscenter/archive/releases/2012/37/image/a/ $ wget http://imgsrc.hubblesite.org/hu/db/images/hs-2012-37-a-large_web.jpg $ python -c 'from pyavm import AVM; print(AVM.from_image("hs-2012-37-a-large_web.jpg"))' Spatial: ReferencePixel: * 3657.5 * 384.0 Equinox: 2000 ReferenceValue: * 53.122751 * -27.805089 ReferenceDimension: * 3380.0 * 3286.0 Scale: * -1.66666666667e-05 * 1.66666666667e-05 CoordsystemProjection: TAN Rotation: -0.0 Type: Observation FL: BackgroundLevel: * 0.0 StretchFunction: * Arcsinh(x) ScaledBackgroundLevel: * 0.0 WhiteLevel: * 4.84225618744 ScaledPeakLevel: * 1000.0 PeakLevel: * 5.0 BlackLevel: * -0.331921895361 MetadataVersion: 1.1 Thanks for your help! Christoph > > Sent from my iPhone > > On Mar 3, 2014, at 10:01 AM, "Christoph Deil" wrote: > >> Hi, >> >> as part of a pull request for blob detection [1], the scikit-image guys decided to add a Hubble deep field image [2] to their `skimage.data` example images [3] to their repository, and we?re trying to find the best file to choose. >> >> The ideal file would be: >> 1. an RGB image in PNG or JPEG format >> 2. not too large (say 100 kB to 500 kB max) >> 3. include AVM info [4] >> 4. have a public, permanent URL as a reference >> >> Does someone know where to a file in this format? >> >> Cheers, >> Christoph >> >> >> [1] https://github.com/scikit-image/scikit-image/pull/900 >> [2] http://en.wikipedia.org/wiki/Hubble_Deep_Field >> [3] http://scikit-image.org/docs/dev/api/skimage.data.html >> [4] http://en.wikipedia.org/wiki/Astronomy_Visualization_Metadata >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From gb.gabrielebrambilla at gmail.com Mon Mar 3 19:58:14 2014 From: gb.gabrielebrambilla at gmail.com (Gabriele Brambilla) Date: Mon, 3 Mar 2014 19:58:14 -0500 Subject: [AstroPy] numerical problems in sum Message-ID: Hi, I'm calculating the energy spectrum of photons emitted by of a lot of accelerated particles. After I have calculated the amount of energy for a single particle in a particular energy interval I sum them all together in this way: MYMAP[i, j, k] = MYMAP[i, j, k] + www where the 3d matrix is a np.array. I have strong reasons to think that in this operation happens some numerical error...Have you suggestions to discover where it is? My results differs a bit from the one obtained by another guy. thanks Gabriele -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Mar 6 09:02:53 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 6 Mar 2014 15:02:53 +0100 Subject: [AstroPy] Astronomy at SciPy 2014 Message-ID: Hi everyone, As a reminder, the deadline for submitting abstracts for the SciPy 2014 conference is *March 14th*: https://conference.scipy.org/scipy2014/ As the chair of the mini-symposium on Astronomy, which will take place during SciPy 2014, I would like to encourage all of you to consider attending, and applying to present any Python-related Astronomy projects (whether Astropy-related or not) you have been working on! It would be great to have many as many people as possible attending from the astronomy community - in addition to interacting with other members of the general scientific Python community, it will be a great opportunity to have discussions at the Astronomy mini-symposium, and to hold coding sprints to make significant progress on Astronomy packages. Even if you haven't had a chance to contribute to the Astronomy Python development efforts until now, this will be your chance to get involved! I look forward to seeing many of you there! Please don't hesitate to let me know if you have any questions! Cheers, Tom From tvw2pu at virginia.edu Thu Mar 6 10:02:08 2014 From: tvw2pu at virginia.edu (Trey Wenger) Date: Thu, 6 Mar 2014 10:02:08 -0500 Subject: [AstroPy] Interest in developing Message-ID: Hi everyone, My name is Trey and I am currently a graduate student at the University of Virginia. I'm an avid Python programmer and have been using Astropy since it's earliest stages. I've been subscribed to this list for some time and finally decided it was time to introduce myself. I am interested in radio astronomy and spectroscopy, specifically single dish radio astronomy. I would enjoy helping with the development of Astropy if you think there is anything I could do! Cheers, Trey -- Trey V. Wenger Department of Astronomy University of Virginia http://www.astro.virginia.edu/~tvw2pu/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Mar 6 17:15:40 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 6 Mar 2014 23:15:40 +0100 Subject: [AstroPy] Interest in developing In-Reply-To: References: Message-ID: Hi Trey, Great that you are interested in contributing to Astropy! I would recommend joining the developer mailing list: https://groups.google.com/forum/#!forum/astropy-dev rather than this more general list, as you will then be able to follow our current development discussions. There's certainly a lot of work available to do, so your offer to help is very welcome! We've created a generic 'contributing' page that has some useful links: http://astropy.org/contribute In practice if you want to contribute actual code, then I would suggest either starting with small issues on the Astropy issue tracker: https://github.com/astropy/astropy/issues?labels=&milestone=&page=1&state=open (but many of the easy ones are being claimed by google summer of code/GSoC applicants). Or, since you are interested in spectroscopy, you might be interested in helping out with the specutils affiliated package: https://github.com/astropy/specutils You could email Wolfgang Kerzendorf, the maintainer of that package (cc-ed) to ask if you can help there. As a side note, it might be easier to determine where you can make the most impact in a couple of weeks once the GSoC projects have been assigned, but in the mean time you can always work on any open astropy (or affiliated package) issue! Cheers, Tom On 6 March 2014 16:02, Trey Wenger wrote: > Hi everyone, > > My name is Trey and I am currently a graduate student at the University of > Virginia. I'm an avid Python programmer and have been using Astropy since > it's earliest stages. I've been subscribed to this list for some time and > finally decided it was time to introduce myself. I am interested in radio > astronomy and spectroscopy, specifically single dish radio astronomy. I > would enjoy helping with the development of Astropy if you think there is > anything I could do! > > Cheers, > Trey > > -- > Trey V. Wenger > Department of Astronomy > University of Virginia > http://www.astro.virginia.edu/~tvw2pu/ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From erik.tollerud at gmail.com Fri Mar 7 15:37:20 2014 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Fri, 7 Mar 2014 15:37:20 -0500 Subject: [AstroPy] ANN: Astropy 0.3.1 released Message-ID: The Astropy project developers are happy to announce the release of v0.3.1 of the astropy core package. This is a bugfix release, meaning it does not contain new features relative to v0.3, but does contain fixes for 50+ bugs. As usual, you can download the new release from PyPI at https://pypi.python.org/pypi/astropy/0.3.1 and the full changelog can be found at http://docs.astropy.org/en/stable/changelog.html . Further details about Astropy are available at http://www.astropy.org If you use Astropy directly - or as a dependency to another package - for your work, please remember to include the following acknowledgment at the end of papers: "This research made use of Astropy, a community-developed core Python package for Astronomy (Astropy Collaboration, 2013)." where "(Astropy Collaboration, 2013)" is the Astropy paper which was published this year: http://adsabs.harvard.edu/abs/2013A%26A...558A..33A Please feel free to forward this announcement to anyone you think might be interested in this release. We hope that you enjoy using Astropy as much as we enjoyed developing it! Erik Tollerud, Thomas Robitaille, and Perry Greenfield on behalf of The Astropy Collaboration -------------- next part -------------- An HTML attachment was scrubbed... URL: From zdr0001 at uah.edu Tue Mar 11 14:06:56 2014 From: zdr0001 at uah.edu (Zachary Robinson) Date: Tue, 11 Mar 2014 13:06:56 -0500 Subject: [AstroPy] Help with WindowsError: [Error 8] Message-ID: Hello, I'm currently a graduate student working on a project for my adviser. The data we are using has always been in CSV format and we have a bunch of python programs using these CSV formatted files. However, a small portion of the oldest data we have is in FITS formatted files. I've been put in charge of extracting the data from these FITS files and putting them into CSV files so we can use our programs we wrote. I read online that astropy has the ability to do this so I've downloaded it and have started to learn it using the documentation center. However, I have experienced an issue while trying to learn the basics commands of the table data portion of FITS file handling. My code so far is (infil is the location of the FITS files on my computer): HDUlist = fits.open(infil) HDUlist.info() tbdata = HDUlist[1].data print(tbdata[1]) When I try to run the program, I get this error message: Traceback (most recent call last): File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS Files.py", line 52, in tbdata = HDUlist[1].data File "C:\Program Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in __get__ val = self._fget(obj) File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, in data data = self._get_tbdata() File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in _get_tbdata self._data_offset) File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in _get_raw_data return self._file.readarray(offset=offset, dtype=code, shape=shape) File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in readarray shape=shape).view(np.ndarray) File "C:\Program Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) WindowsError: [Error 8] Not enough storage is available to process this command Can anyone explain to me what this error is and how I can go about fixing it? From what I've read online so far, I'm thinking it has something to do with the mmap but I don't know what that is. I'm running this code in Python 2.7 on a 32-bit WinXP machine. Thanks a lot for the help, Zach -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Tue Mar 11 14:09:54 2014 From: stsci.perry at gmail.com (Perry Greenfield) Date: Tue, 11 Mar 2014 14:09:54 -0400 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: References: Message-ID: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> How big is the table? It's quite possible you are running into a 32-bit limitation, particularly if the table is 2GB or bigger. Perry On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: > Hello, > > I'm currently a graduate student working on a project for my adviser. The data we are using has always been in CSV format and we have a bunch of python programs using these CSV formatted files. However, a small portion of the oldest data we have is in FITS formatted files. I've been put in charge of extracting the data from these FITS files and putting them into CSV files so we can use our programs we wrote. I read online that astropy has the ability to do this so I've downloaded it and have started to learn it using the documentation center. > > However, I have experienced an issue while trying to learn the basics commands of the table data portion of FITS file handling. My code so far is (infil is the location of the FITS files on my computer): > > HDUlist = fits.open(infil) > HDUlist.info() > tbdata = HDUlist[1].data > print(tbdata[1]) > > When I try to run the program, I get this error message: > > Traceback (most recent call last): > File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS Files.py", line 52, in > tbdata = HDUlist[1].data > File "C:\Program Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in __get__ > val = self._fget(obj) > File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, in data > data = self._get_tbdata() > File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in _get_tbdata > self._data_offset) > File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in _get_raw_data > return self._file.readarray(offset=offset, dtype=code, shape=shape) > File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in readarray > shape=shape).view(np.ndarray) > File "C:\Program Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ > mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) > WindowsError: [Error 8] Not enough storage is available to process this command > > Can anyone explain to me what this error is and how I can go about fixing it? From what I've read online so far, I'm thinking it has something to do with the mmap but I don't know what that is. > > I'm running this code in Python 2.7 on a 32-bit WinXP machine. > > Thanks a lot for the help, > Zach > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From zdr0001 at uah.edu Tue Mar 11 14:49:44 2014 From: zdr0001 at uah.edu (Zachary Robinson) Date: Tue, 11 Mar 2014 13:49:44 -0500 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> Message-ID: <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> Perry, The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C Zach Sent from my iPhone > On Mar 11, 2014, at 1:09 PM, Perry Greenfield wrote: > > How big is the table? > > It's quite possible you are running into a 32-bit limitation, particularly if the table is 2GB or bigger. > > Perry > > >> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: >> >> Hello, >> >> I'm currently a graduate student working on a project for my adviser. The data we are using has always been in CSV format and we have a bunch of python programs using these CSV formatted files. However, a small portion of the oldest data we have is in FITS formatted files. I've been put in charge of extracting the data from these FITS files and putting them into CSV files so we can use our programs we wrote. I read online that astropy has the ability to do this so I've downloaded it and have started to learn it using the documentation center. >> >> However, I have experienced an issue while trying to learn the basics commands of the table data portion of FITS file handling. My code so far is (infil is the location of the FITS files on my computer): >> >> HDUlist = fits.open(infil) >> HDUlist.info() >> tbdata = HDUlist[1].data >> print(tbdata[1]) >> >> When I try to run the program, I get this error message: >> >> Traceback (most recent call last): >> File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS Files.py", line 52, in >> tbdata = HDUlist[1].data >> File "C:\Program Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in __get__ >> val = self._fget(obj) >> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, in data >> data = self._get_tbdata() >> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in _get_tbdata >> self._data_offset) >> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in _get_raw_data >> return self._file.readarray(offset=offset, dtype=code, shape=shape) >> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in readarray >> shape=shape).view(np.ndarray) >> File "C:\Program Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ >> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) >> WindowsError: [Error 8] Not enough storage is available to process this command >> >> Can anyone explain to me what this error is and how I can go about fixing it? From what I've read online so far, I'm thinking it has something to do with the mmap but I don't know what that is. >> >> I'm running this code in Python 2.7 on a 32-bit WinXP machine. >> >> Thanks a lot for the help, >> Zach >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > From thomas.robitaille at gmail.com Wed Mar 12 08:33:16 2014 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 12 Mar 2014 13:33:16 +0100 Subject: [AstroPy] SciPy 2014 abstract deadline *this friday!* Message-ID: Hi everyone, Just a reminder that the deadline for submitting abstracts for SciPy 2014 is this Friday, 14th March. I would like to encourage all of you who are involved in developing packages (including Astropy-affiliated packages) to consider submitting an abstract for a talk! Note that you don't have to yet formally register for the meeting, so if you are currently unsure if you will be able to make it, I would still encourage you to apply! https://conference.scipy.org/scipy2014/ Since SciPy 2014 includes two days of sprinting, this will be a great chance to make a lot of practical progress, so even if your package is not 'finished', you should apply to present it, and motivate people to help you with it during the sprints! Cheers, Tom From dklaes at astro.uni-bonn.de Wed Mar 12 12:28:06 2014 From: dklaes at astro.uni-bonn.de (Dominik Klaes) Date: Wed, 12 Mar 2014 17:28:06 +0100 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> Message-ID: He sent me one file for testing, and I get a somehow similar error on Linux, so it doesn't seem to be a Windows related problem. When checking the tables with fv (it's part of ftools), I saw that there are at the end of the table many empty lines (so no 0 or NaN or similar, just empty). Could that be a problem? Cheers, Dominik 2014-03-11 19:49 GMT+01:00 Zachary Robinson : > Perry, > > The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C > > Zach > > Sent from my iPhone > > > On Mar 11, 2014, at 1:09 PM, Perry Greenfield > wrote: > > > > How big is the table? > > > > It's quite possible you are running into a 32-bit limitation, > particularly if the table is 2GB or bigger. > > > > Perry > > > > > >> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: > >> > >> Hello, > >> > >> I'm currently a graduate student working on a project for my adviser. > The data we are using has always been in CSV format and we have a bunch of > python programs using these CSV formatted files. However, a small portion > of the oldest data we have is in FITS formatted files. I've been put in > charge of extracting the data from these FITS files and putting them into > CSV files so we can use our programs we wrote. I read online that astropy > has the ability to do this so I've downloaded it and have started to learn > it using the documentation center. > >> > >> However, I have experienced an issue while trying to learn the basics > commands of the table data portion of FITS file handling. My code so far > is (infil is the location of the FITS files on my computer): > >> > >> HDUlist = fits.open(infil) > >> HDUlist.info() > >> tbdata = HDUlist[1].data > >> print(tbdata[1]) > >> > >> When I try to run the program, I get this error message: > >> > >> Traceback (most recent call last): > >> File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic SPE > Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS > Files.py", line 52, in > >> tbdata = HDUlist[1].data > >> File "C:\Program > Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in > __get__ > >> val = self._fget(obj) > >> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, > in data > >> data = self._get_tbdata() > >> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in > _get_tbdata > >> self._data_offset) > >> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in > _get_raw_data > >> return self._file.readarray(offset=offset, dtype=code, shape=shape) > >> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in > readarray > >> shape=shape).view(np.ndarray) > >> File "C:\Program > Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ > >> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) > >> WindowsError: [Error 8] Not enough storage is available to process this > command > >> > >> Can anyone explain to me what this error is and how I can go about > fixing it? From what I've read online so far, I'm thinking it has something > to do with the mmap but I don't know what that is. > >> > >> I'm running this code in Python 2.7 on a 32-bit WinXP machine. > >> > >> Thanks a lot for the help, > >> Zach > >> _______________________________________________ > >> AstroPy mailing list > >> AstroPy at scipy.org > >> http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Dominik Klaes Deputy student representative of the AIfA Argelander-Institut f?r Astronomie Room 2.027a Auf dem H?gel 71 53121 Bonn Telefon: 0228/73-5773 E-Mail: dklaes at astro.uni-bonn.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Wed Mar 12 14:00:21 2014 From: embray at stsci.edu (Erik Bray) Date: Wed, 12 Mar 2014 14:00:21 -0400 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> Message-ID: <5320A0B5.5050108@stsci.edu> On 03/11/2014 02:49 PM, Zachary Robinson wrote: > Perry, > > The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C This raises the question, however, of what the formats are of the columns. Could you please post the entire header(s) of this file and the output of fits.info() on the file? Erik >> On Mar 11, 2014, at 1:09 PM, Perry Greenfield wrote: >> >> How big is the table? >> >> It's quite possible you are running into a 32-bit limitation, particularly if the table is 2GB or bigger. >> >> Perry >> >> >>> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: >>> >>> Hello, >>> >>> I'm currently a graduate student working on a project for my adviser. The data we are using has always been in CSV format and we have a bunch of python programs using these CSV formatted files. However, a small portion of the oldest data we have is in FITS formatted files. I've been put in charge of extracting the data from these FITS files and putting them into CSV files so we can use our programs we wrote. I read online that astropy has the ability to do this so I've downloaded it and have started to learn it using the documentation center. >>> >>> However, I have experienced an issue while trying to learn the basics commands of the table data portion of FITS file handling. My code so far is (infil is the location of the FITS files on my computer): >>> >>> HDUlist = fits.open(infil) >>> HDUlist.info() >>> tbdata = HDUlist[1].data >>> print(tbdata[1]) >>> >>> When I try to run the program, I get this error message: >>> >>> Traceback (most recent call last): >>> File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS Files.py", line 52, in >>> tbdata = HDUlist[1].data >>> File "C:\Program Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in __get__ >>> val = self._fget(obj) >>> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, in data >>> data = self._get_tbdata() >>> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in _get_tbdata >>> self._data_offset) >>> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in _get_raw_data >>> return self._file.readarray(offset=offset, dtype=code, shape=shape) >>> File "C:\Program Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in readarray >>> shape=shape).view(np.ndarray) >>> File "C:\Program Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ >>> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) >>> WindowsError: [Error 8] Not enough storage is available to process this command >>> >>> Can anyone explain to me what this error is and how I can go about fixing it? From what I've read online so far, I'm thinking it has something to do with the mmap but I don't know what that is. >>> >>> I'm running this code in Python 2.7 on a 32-bit WinXP machine. >>> >>> Thanks a lot for the help, >>> Zach >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From zdr0001 at uah.edu Wed Mar 12 14:31:06 2014 From: zdr0001 at uah.edu (Zachary Robinson) Date: Wed, 12 Mar 2014 13:31:06 -0500 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: <5320A0B5.5050108@stsci.edu> References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> <5320A0B5.5050108@stsci.edu> Message-ID: Sure. WARNING: File may have been truncated: actual file length (9947040) is smaller than the expected size (10736640) [astropy.io.fits.file] Filename: C:\Documents and Settings\zach\Desktop\H0638308.FIT No. Name Type Cards Dimensions Format 0 PRIMARY PrimaryHDU 34 () 1 BinTableHDU 221 892800R x 4C [J, J, I, I] This is the output after running fits.info(). Zach On Wed, Mar 12, 2014 at 1:00 PM, Erik Bray wrote: > On 03/11/2014 02:49 PM, Zachary Robinson wrote: > > Perry, > > > > The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C > > This raises the question, however, of what the formats are of the columns. > Could you please post the entire header(s) of this file and the output of > fits.info() on the file? > > Erik > > >> On Mar 11, 2014, at 1:09 PM, Perry Greenfield > wrote: > >> > >> How big is the table? > >> > >> It's quite possible you are running into a 32-bit limitation, > particularly if the table is 2GB or bigger. > >> > >> Perry > >> > >> > >>> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: > >>> > >>> Hello, > >>> > >>> I'm currently a graduate student working on a project for my adviser. > The data we are using has always been in CSV format and we have a bunch of > python programs using these CSV formatted files. However, a small portion > of the oldest data we have is in FITS formatted files. I've been put in > charge of extracting the data from these FITS files and putting them into > CSV files so we can use our programs we wrote. I read online that astropy > has the ability to do this so I've downloaded it and have started to learn > it using the documentation center. > >>> > >>> However, I have experienced an issue while trying to learn the basics > commands of the table data portion of FITS file handling. My code so far > is (infil is the location of the FITS files on my computer): > >>> > >>> HDUlist = fits.open(infil) > >>> HDUlist.info() > >>> tbdata = HDUlist[1].data > >>> print(tbdata[1]) > >>> > >>> When I try to run the program, I get this error message: > >>> > >>> Traceback (most recent call last): > >>> File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic > SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS > Files.py", line 52, in > >>> tbdata = HDUlist[1].data > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in > __get__ > >>> val = self._fget(obj) > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, > in data > >>> data = self._get_tbdata() > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in > _get_tbdata > >>> self._data_offset) > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in > _get_raw_data > >>> return self._file.readarray(offset=offset, dtype=code, shape=shape) > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in > readarray > >>> shape=shape).view(np.ndarray) > >>> File "C:\Program > Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ > >>> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) > >>> WindowsError: [Error 8] Not enough storage is available to process > this command > >>> > >>> Can anyone explain to me what this error is and how I can go about > fixing it? From what I've read online so far, I'm thinking it has something > to do with the mmap but I don't know what that is. > >>> > >>> I'm running this code in Python 2.7 on a 32-bit WinXP machine. > >>> > >>> Thanks a lot for the help, > >>> Zach > >>> _______________________________________________ > >>> AstroPy mailing list > >>> AstroPy at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/astropy > >> > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dklaes at astro.uni-bonn.de Wed Mar 12 14:50:37 2014 From: dklaes at astro.uni-bonn.de (Dominik Klaes) Date: Wed, 12 Mar 2014 19:50:37 +0100 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> <5320A0B5.5050108@stsci.edu> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 As I already wrote: A lot of lines are simply empty, that could explain while the actual file size is smaller than the expected size. Cheers, Dominik On 12. M?rz 2014 19:31:06 MEZ, Zachary Robinson wrote: >Sure. > >WARNING: File may have been truncated: actual file length (9947040) is >smaller than the expected size (10736640) [astropy.io.fits.file] >Filename: C:\Documents and Settings\zach\Desktop\H0638308.FIT >No. Name Type Cards Dimensions Format >0 PRIMARY PrimaryHDU 34 () >1 BinTableHDU 221 892800R x 4C [J, J, I, I] > >This is the output after running fits.info(). > >Zach > > > >On Wed, Mar 12, 2014 at 1:00 PM, Erik Bray wrote: > >> On 03/11/2014 02:49 PM, Zachary Robinson wrote: >> > Perry, >> > >> > The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C >> >> This raises the question, however, of what the formats are of the >columns. >> Could you please post the entire header(s) of this file and the >output of >> fits.info() on the file? >> >> Erik >> >> >> On Mar 11, 2014, at 1:09 PM, Perry Greenfield > >> wrote: >> >> >> >> How big is the table? >> >> >> >> It's quite possible you are running into a 32-bit limitation, >> particularly if the table is 2GB or bigger. >> >> >> >> Perry >> >> >> >> >> >>> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: >> >>> >> >>> Hello, >> >>> >> >>> I'm currently a graduate student working on a project for my >adviser. >> The data we are using has always been in CSV format and we have a >bunch of >> python programs using these CSV formatted files. However, a small >portion >> of the oldest data we have is in FITS formatted files. I've been put >in >> charge of extracting the data from these FITS files and putting them >into >> CSV files so we can use our programs we wrote. I read online that >astropy >> has the ability to do this so I've downloaded it and have started to >learn >> it using the documentation center. >> >>> >> >>> However, I have experienced an issue while trying to learn the >basics >> commands of the table data portion of FITS file handling. My code so >far >> is (infil is the location of the FITS files on my computer): >> >>> >> >>> HDUlist = fits.open(infil) >> >>> HDUlist.info() >> >>> tbdata = HDUlist[1].data >> >>> print(tbdata[1]) >> >>> >> >>> When I try to run the program, I get this error message: >> >>> >> >>> Traceback (most recent call last): >> >>> File "C:\Documents and >Settings\zach\Desktop\Dropbox\Probabilistic >> SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading >FITS >> Files.py", line 52, in >> >>> tbdata = HDUlist[1].data >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in >> __get__ >> >>> val = self._fget(obj) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line >285, >> in data >> >>> data = self._get_tbdata() >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line >93, in >> _get_tbdata >> >>> self._data_offset) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line >509, in >> _get_raw_data >> >>> return self._file.readarray(offset=offset, dtype=code, >shape=shape) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, >in >> readarray >> >>> shape=shape).view(np.ndarray) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in >__new__ >> >>> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) >> >>> WindowsError: [Error 8] Not enough storage is available to >process >> this command >> >>> >> >>> Can anyone explain to me what this error is and how I can go >about >> fixing it? From what I've read online so far, I'm thinking it has >something >> to do with the mmap but I don't know what that is. >> >>> >> >>> I'm running this code in Python 2.7 on a 32-bit WinXP machine. >> >>> >> >>> Thanks a lot for the help, >> >>> Zach >> >>> _______________________________________________ >> >>> AstroPy mailing list >> >>> AstroPy at scipy.org >> >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> > _______________________________________________ >> > AstroPy mailing list >> > AstroPy at scipy.org >> > http://mail.scipy.org/mailman/listinfo/astropy >> > >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > >------------------------------------------------------------------------ > >_______________________________________________ >AstroPy mailing list >AstroPy at scipy.org >http://mail.scipy.org/mailman/listinfo/astropy - -- Dominik Klaes Student representative of the AIfA Argelander-Institut f?r Astronomie Room 2.027a Auf dem H?gel 71 53121 Bonn Telefon: 0228/73-5773 E-Mail: dklaes at astro.uni-bonn.de -----BEGIN PGP SIGNATURE----- Version: APG v1.0.9 iQJGBAEBCgAwBQJTIKx9KRxEb21pbmlrIEtsYWVzIDxka2xhZXNAYXN0cm8udW5p LWJvbm4uZGU+AAoJEG4xBfJ3OX2tN4AP/Rs0jssHIaQDD37DPqHUxZzBaqnvUpiq 17XqhQKWuos35vUEcYS4LKbD2r5XfF3V183ttEMkVIEp35Y4E6EvCrJWEJlBDYJf uCWsQkLFcw1WxNLahNTVG5Y0EUxsYPlcRPu74EVJ/1YO21CvGzfrmjVHrLLXwta1 wFL0zTStaOZx83FSDZ7W4kOWiEWBNMlxDMvzK5bbCpi0VOCsQemCZ+1+zRqp+yyB +nwhzGEILl5FFrYgj602vWjW0gyNDyENVBkJEk3kzYCUo1QaY4hSEaRSGJjdpuRg s0/jItpLHbR1Y3R84tfDIuIHhoADOpRKsHoRpjy+OUkhi5FGE3i0ZCttEnAZiw+5 fuKrbFgfYtoYXmypao9MZpSEf6YqRXnEXYPTJIW0RJnz2UW0KEa9pmJaMK9HV1ef 3P+f/bgzf07+jSWU+AKfKxZ0MaES5IOcVIerMGUKU6l7hDfH4ZU3RGYt7uE0t184 EJHGb7ZerTTmaKHVOkE+Uwp12YH+maFpjqu5o7HnTPZxXxPIR1XOufugN3GdlJSl WqR9WB79iKk74go8nj8L50HVHZ8pMrrchnpxsmN860EF4AD7BZ+bLJlKLUL2FfZJ dguuTcaBSbpGxO594cvDCtB4EbNEW2/hWAEmtnkzOBV5QRJbp0thp21YrRbUGNtE NPBeqAh/lg02 =oA0r -----END PGP SIGNATURE----- From embray at stsci.edu Wed Mar 12 15:12:22 2014 From: embray at stsci.edu (Erik Bray) Date: Wed, 12 Mar 2014 15:12:22 -0400 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> <5320A0B5.5050108@stsci.edu> Message-ID: <5320B196.30006@stsci.edu> On 03/12/2014 02:31 PM, Zachary Robinson wrote: > Sure. > > WARNING: File may have been truncated: actual file length (9947040) is smaller > than the expected size (10736640) [astropy.io.fits.file] > Filename: C:\Documents and Settings\zach\Desktop\H0638308.FIT > No. Name Type Cards Dimensions Format > 0 PRIMARY PrimaryHDU 34 () > 1 BinTableHDU 221 892800R x 4C [J, J, I, I] > > This is the output after running fits.info (). As I suspected, it sounds like the file is corrupted. When that "File may have been truncated" warning occurs pyfits doesn't outright *crash* because it may still be possible to get some information out of the file. But it still means all bets are off. For what it's worth, I did open an issue to try to improve reading data from corrupt files, so that it can at least be partially read (it sounds like most of your file is at least there, and I think we can do better than just crashing with an error from the OS): https://github.com/spacetelescope/PyFITS/issues/50 Erik > Zach > > > > On Wed, Mar 12, 2014 at 1:00 PM, Erik Bray > wrote: > > On 03/11/2014 02:49 PM, Zachary Robinson wrote: > > Perry, > > > > The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C > > This raises the question, however, of what the formats are of the columns. > Could you please post the entire header(s) of this file and the output of > fits.info () on the file? > > Erik > > >> On Mar 11, 2014, at 1:09 PM, Perry Greenfield > wrote: > >> > >> How big is the table? > >> > >> It's quite possible you are running into a 32-bit limitation, > particularly if the table is 2GB or bigger. > >> > >> Perry > >> > >> > >>> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: > >>> > >>> Hello, > >>> > >>> I'm currently a graduate student working on a project for my adviser. > The data we are using has always been in CSV format and we have a bunch of > python programs using these CSV formatted files. However, a small portion > of the oldest data we have is in FITS formatted files. I've been put in > charge of extracting the data from these FITS files and putting them into > CSV files so we can use our programs we wrote. I read online that astropy > has the ability to do this so I've downloaded it and have started to learn > it using the documentation center. > >>> > >>> However, I have experienced an issue while trying to learn the basics > commands of the table data portion of FITS file handling. My code so far is > (infil is the location of the FITS files on my computer): > >>> > >>> HDUlist = fits.open(infil) > >>> HDUlist.info() > >>> tbdata = HDUlist[1].data > >>> print(tbdata[1]) > >>> > >>> When I try to run the program, I get this error message: > >>> > >>> Traceback (most recent call last): > >>> File "C:\Documents and Settings\zach\Desktop\Dropbox\Probabilistic > SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading FITS > Files.py", line 52, in > >>> tbdata = HDUlist[1].data > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, in __get__ > >>> val = self._fget(obj) > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 285, in > data > >>> data = self._get_tbdata() > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line 93, in > _get_tbdata > >>> self._data_offset) > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line 509, in > _get_raw_data > >>> return self._file.readarray(offset=offset, dtype=code, shape=shape) > >>> File "C:\Program > Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, in > readarray > >>> shape=shape).view(np.ndarray) > >>> File "C:\Program > Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in __new__ > >>> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) > >>> WindowsError: [Error 8] Not enough storage is available to process this > command > >>> > >>> Can anyone explain to me what this error is and how I can go about > fixing it? From what I've read online so far, I'm thinking it has something > to do with the mmap but I don't know what that is. > >>> > >>> I'm running this code in Python 2.7 on a 32-bit WinXP machine. > >>> > >>> Thanks a lot for the help, > >>> Zach > >>> _______________________________________________ > >>> AstroPy mailing list > >>> AstroPy at scipy.org > >>> http://mail.scipy.org/mailman/listinfo/astropy > >> > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > From zdr0001 at uah.edu Wed Mar 12 15:41:25 2014 From: zdr0001 at uah.edu (Zachary Robinson) Date: Wed, 12 Mar 2014 14:41:25 -0500 Subject: [AstroPy] Help with WindowsError: [Error 8] In-Reply-To: <5320B196.30006@stsci.edu> References: <264C44E1-F835-49FF-A47E-5D0569F700E7@gmail.com> <41649E35-CB36-4B2C-A1F7-FEEE430DD31F@uah.edu> <5320A0B5.5050108@stsci.edu> <5320B196.30006@stsci.edu> Message-ID: Erik, Well I downloaded the data straight from the satellite's website and this happens for all the files I've tried so far. I guess I will try to get in touch with the satellite operators to see if they have different versions of the data offline that they could send me. Thank you guys very much, Zach On Wed, Mar 12, 2014 at 2:12 PM, Erik Bray wrote: > On 03/12/2014 02:31 PM, Zachary Robinson wrote: > >> Sure. >> >> WARNING: File may have been truncated: actual file length (9947040) is >> smaller >> than the expected size (10736640) [astropy.io.fits.file] >> Filename: C:\Documents and Settings\zach\Desktop\H0638308.FIT >> No. Name Type Cards Dimensions Format >> 0 PRIMARY PrimaryHDU 34 () >> 1 BinTableHDU 221 892800R x 4C [J, J, I, I] >> >> This is the output after running fits.info (). >> > > As I suspected, it sounds like the file is corrupted. When that "File may > have been truncated" warning occurs pyfits doesn't outright *crash* because > it may still be possible to get some information out of the file. But it > still means all bets are off. > > For what it's worth, I did open an issue to try to improve reading data > from corrupt files, so that it can at least be partially read (it sounds > like most of your file is at least there, and I think we can do better than > just crashing with an error from the OS): https://github.com/ > spacetelescope/PyFITS/issues/50 > > > Erik > > Zach >> >> >> >> On Wed, Mar 12, 2014 at 1:00 PM, Erik Bray > > wrote: >> >> On 03/11/2014 02:49 PM, Zachary Robinson wrote: >> > Perry, >> > >> > The FITS file is 3.66MB. The dimensions of the table are 892800Rx4C >> >> This raises the question, however, of what the formats are of the >> columns. >> Could you please post the entire header(s) of this file and the >> output of >> fits.info () on the file? >> >> >> Erik >> >> >> On Mar 11, 2014, at 1:09 PM, Perry Greenfield < >> stsci.perry at gmail.com >> > wrote: >> >> >> >> How big is the table? >> >> >> >> It's quite possible you are running into a 32-bit limitation, >> particularly if the table is 2GB or bigger. >> >> >> >> Perry >> >> >> >> >> >>> On Mar 11, 2014, at 2:06 PM, Zachary Robinson wrote: >> >>> >> >>> Hello, >> >>> >> >>> I'm currently a graduate student working on a project for my >> adviser. >> The data we are using has always been in CSV format and we have a >> bunch of >> python programs using these CSV formatted files. However, a small >> portion >> of the oldest data we have is in FITS formatted files. I've been put >> in >> charge of extracting the data from these FITS files and putting them >> into >> CSV files so we can use our programs we wrote. I read online that >> astropy >> has the ability to do this so I've downloaded it and have started to >> learn >> it using the documentation center. >> >>> >> >>> However, I have experienced an issue while trying to learn the >> basics >> commands of the table data portion of FITS file handling. My code so >> far is >> (infil is the location of the FITS files on my computer): >> >>> >> >>> HDUlist = fits.open(infil) >> >>> HDUlist.info() >> >>> tbdata = HDUlist[1].data >> >>> print(tbdata[1]) >> >>> >> >>> When I try to run the program, I get this error message: >> >>> >> >>> Traceback (most recent call last): >> >>> File "C:\Documents and Settings\zach\Desktop\Dropbox\ >> Probabilistic >> SPE Model\GOES\Python Programs\Astropy and FIT Files Tutorial\Reading >> FITS >> Files.py", line 52, in >> >>> tbdata = HDUlist[1].data >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\utils\misc.py", line 279, >> in __get__ >> >>> val = self._fget(obj) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line >> 285, in >> data >> >>> data = self._get_tbdata() >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\hdu\table.py", line >> 93, in >> _get_tbdata >> >>> self._data_offset) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\hdu\base.py", line >> 509, in >> _get_raw_data >> >>> return self._file.readarray(offset=offset, dtype=code, >> shape=shape) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\astropy\io\fits\file.py", line 257, >> in >> readarray >> >>> shape=shape).view(np.ndarray) >> >>> File "C:\Program >> Files\Python27\lib\site-packages\numpy\core\memmap.py", line 237, in >> __new__ >> >>> mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) >> >>> WindowsError: [Error 8] Not enough storage is available to >> process this >> command >> >>> >> >>> Can anyone explain to me what this error is and how I can go >> about >> fixing it? From what I've read online so far, I'm thinking it has >> something >> to do with the mmap but I don't know what that is. >> >>> >> >>> I'm running this code in Python 2.7 on a 32-bit WinXP machine. >> >>> >> >>> Thanks a lot for the help, >> >>> Zach >> >>> _______________________________________________ >> >>> AstroPy mailing list >> >>> AstroPy at scipy.org >> >> >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> > _______________________________________________ >> > AstroPy mailing list >> > AstroPy at scipy.org >> >> > http://mail.scipy.org/mailman/listinfo/astropy >> > >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gb.gabrielebrambilla at gmail.com Fri Mar 14 15:57:14 2014 From: gb.gabrielebrambilla at gmail.com (Gabriele Brambilla) Date: Fri, 14 Mar 2014 15:57:14 -0400 Subject: [AstroPy] problems with pyfits Message-ID: Hi, I'm trying to run a code that starts with import numpy, pyfits but the pc told me that ImportError: No module named pyfits I have installed astropy (I use Anaconda) but in this case it seems that it doesn't work... Anyone has an idea? thanks Gabriele -------------- next part -------------- An HTML attachment was scrubbed... URL: From jzuhone at gmail.com Fri Mar 14 15:59:24 2014 From: jzuhone at gmail.com (John ZuHone) Date: Fri, 14 Mar 2014 15:59:24 -0400 Subject: [AstroPy] problems with pyfits In-Reply-To: References: Message-ID: <90BC7AB2-BE61-44C7-880F-EA4945ADF81E@gmail.com> Hi Gabriele, If you installed AstroPy using anaconda, then you will get the pyFITS modules as part of AstroPy, but the module name is different. You can import import astropy.io.fits as pyfits and you will get the same effect. For more reference about FITS in AstroPy: http://astropy.readthedocs.org/en/latest/io/fits/ Best, John Z On Mar 14, 2014, at 3:57 PM, Gabriele Brambilla wrote: > Hi, > I'm trying to run a code that starts with > > import numpy, pyfits > > but the pc told me that > > ImportError: No module named pyfits > > I have installed astropy (I use Anaconda) but in this case it seems that it doesn't work... > > Anyone has an idea? > > thanks > > Gabriele > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From gb.gabrielebrambilla at gmail.com Fri Mar 14 16:15:24 2014 From: gb.gabrielebrambilla at gmail.com (Gabriele Brambilla) Date: Fri, 14 Mar 2014 16:15:24 -0400 Subject: [AstroPy] problems with pyfits In-Reply-To: <90BC7AB2-BE61-44C7-880F-EA4945ADF81E@gmail.com> References: <90BC7AB2-BE61-44C7-880F-EA4945ADF81E@gmail.com> Message-ID: thanks. it works Gabriele 2014-03-14 15:59 GMT-04:00 John ZuHone : > Hi Gabriele, > > If you installed AstroPy using anaconda, then you will get the pyFITS > modules as part of AstroPy, but the module name is different. > > You can import > > import astropy.io.fits as pyfits > > and you will get the same effect. > > For more reference about FITS in AstroPy: > > http://astropy.readthedocs.org/en/latest/io/fits/ > > Best, > > John Z > > On Mar 14, 2014, at 3:57 PM, Gabriele Brambilla < > gb.gabrielebrambilla at gmail.com> wrote: > > > Hi, > > I'm trying to run a code that starts with > > > > import numpy, pyfits > > > > but the pc told me that > > > > ImportError: No module named pyfits > > > > I have installed astropy (I use Anaconda) but in this case it seems that > it doesn't work... > > > > Anyone has an idea? > > > > thanks > > > > Gabriele > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > http://mail.scipy.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lrodriguez at iafe.uba.ar Mon Mar 17 10:00:08 2014 From: lrodriguez at iafe.uba.ar (Leticia Rodriguez) Date: Mon, 17 Mar 2014 11:00:08 -0300 Subject: [AstroPy] Heliocentric Julian Date HJD Message-ID: Hi, I need to find out the Observational Date (timestamp) from a HJD value. I have a function that it converts from MJD to Timestamp. I could use this function converting first from HJD to MJD. Or just use a function that it converts from HJD to Timestamp. Does anyone know any function to convert from HJD to MJD or HJD to Timestamp in any python library? Thanks Leticia. -- *Leticia Lorena Rodr?guez* Technical Contact NOVA - Virtual Observatory Argentina http://nova.conicet.gov.ar -------------- next part -------------- An HTML attachment was scrubbed... URL: From jslavin at cfa.harvard.edu Mon Mar 17 13:33:02 2014 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Mon, 17 Mar 2014 13:33:02 -0400 Subject: [AstroPy] Heliocentric Julian Date HJD Message-ID: Hi Leticia, It looks like the astrolibpy package (http://code.google.com/p/astrolibpy/) has what you want in it. See helio_jd.py. (Caveat: I haven't tried it myself.) Jon On Mon, Mar 17, 2014 at 1:00 PM, wrote: > Date: Mon, 17 Mar 2014 11:00:08 -0300 > From: Leticia Rodriguez > Subject: [AstroPy] Heliocentric Julian Date HJD > To: "" > Message-ID: > ZfYnA at mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi, > > I need to find out the Observational Date (timestamp) from a HJD value. > > I have a function that it converts from MJD to Timestamp. I could use this > function converting first from HJD to MJD. > > Or just use a function that it converts from HJD to Timestamp. > > Does anyone know any function to convert from HJD to MJD or HJD to > Timestamp in any python library? > > Thanks > Leticia. > -- ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 fax: (617) 496-7577 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From lrodriguez at iafe.uba.ar Mon Mar 17 13:38:39 2014 From: lrodriguez at iafe.uba.ar (Leticia Rodriguez) Date: Mon, 17 Mar 2014 14:38:39 -0300 Subject: [AstroPy] Heliocentric Julian Date HJD In-Reply-To: References: Message-ID: Jon, Thanks for your answer!. I'm not sure. It seems that helio_jd.py goes from JD to HJD and I need from HJD to JD. Am I right? 2014-03-17 14:33 GMT-03:00 Slavin, Jonathan : > Hi Leticia, > > It looks like the astrolibpy package (http://code.google.com/p/astrolibpy/) > has what you want in it. See helio_jd.py. (Caveat: I haven't tried it > myself.) > > Jon > > On Mon, Mar 17, 2014 at 1:00 PM, wrote: > >> Date: Mon, 17 Mar 2014 11:00:08 -0300 >> From: Leticia Rodriguez >> Subject: [AstroPy] Heliocentric Julian Date HJD >> To: "" >> Message-ID: >> > ZfYnA at mail.gmail.com> >> Content-Type: text/plain; charset="iso-8859-1" >> >> >> Hi, >> >> I need to find out the Observational Date (timestamp) from a HJD value. >> >> I have a function that it converts from MJD to Timestamp. I could use >> this >> function converting first from HJD to MJD. >> >> Or just use a function that it converts from HJD to Timestamp. >> >> Does anyone know any function to convert from HJD to MJD or HJD to >> Timestamp in any python library? >> >> Thanks >> Leticia. >> > > > > > -- > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > fax: (617) 496-7577 USA > ________________________________________________________ > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- *Leticia Lorena Rodr?guez* Technical Contact NOVA - Virtual Observatory Argentina http://nova.conicet.gov.ar -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergio.pasra at gmail.com Thu Mar 20 19:38:29 2014 From: sergio.pasra at gmail.com (Sergio Pascual) Date: Fri, 21 Mar 2014 00:38:29 +0100 Subject: [AstroPy] Problems reading CDS catalogue Message-ID: Hi, I'm having problems reading the catalog in http://cdsarc.u-strasbg.fr/viz-bin/Cat?J/PASP/110/863 (file J_PASP_110_863.tar.gz) The files inside are compressed, so I uncompress them before. I create a reader object, (I'm using astropy 0.3, BTW) readme = 'ReadMe' r = ascii.get_reader(ascii.Cds, readme=readme) This reader works only for some specta. For example, m2v and goi works, but the star 'o9v' doesn't work. Inserting this last star generates the following exception >>> r.read('o9v.dat') /usr/lib64/python2.7/site-packages/astropy/io/ascii/core.pyc in read(self, table) 831 832 # Get the table column definitions --> 833 self.header.get_cols(self.lines) 834 835 cols = self.header.cols /usr/lib64/python2.7/site-packages/astropy/io/ascii/cds.pyc in get_cols(self, lines) 164 cols[-1].description += line.strip() 165 else: --> 166 raise ValueError('Line "%s" not parsable as CDS header' % line) 167 168 self.names = [x.name for x in cols] ValueError: Line "Bytes Format Units Label Explanations" not parsable as CDS header Any hint? Sergio -------------- next part -------------- An HTML attachment was scrubbed... URL: From aldcroft at head.cfa.harvard.edu Thu Mar 20 21:31:15 2014 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 20 Mar 2014 21:31:15 -0400 Subject: [AstroPy] Problems reading CDS catalogue In-Reply-To: References: Message-ID: I believe this is a bug in the CDS reader, which isn't correctly handling a ReadMe that has multiple lines of the Byte-by-byte Description of file like this: Byte-by-byte Description of file: o9v.dat, b57v.dat, a2v.dat Byte-by-byte Description of file: b12iii.dat, b3iii.dat, b5iii.dat, b9iii.dat Byte-by-byte Description of file: a0iii.dat, b0i.dat, b5i.dat, b8i.dat -------------------------------------------------------------------------------- Bytes Format Units Label Explanations -------------------------------------------------------------------------------- 1- 7 F7.1 0.1nm lambda wavelength (in Angstroms) 8- 17 F10.6 --- nflam component-averaged f(lambda)/f(5556 A) 19- 27 F9.6 --- sdnflam std. dev. of avg. at this wavelength 28- 37 F10.6 --- fi f(lambda)/f(5556 A) from Cat. 39- 47 F9.6 --- fg f(lambda)/f(5556 A) from Cat. 49- 57 F9.6 --- fj f(lambda)/f(5556 A) from Cat. 59- 67 F9.6 --- fd f(lambda)/f(5556 A) from 1994PASP..106..382D -------------------------------------------------------------------------------- As a workaround you can modify your local ReadMe to have only one such line corresponding to the file you want. This is now an astropy issue: https://github.com/astropy/astropy/issues/2218 . Cheers, Tom On Thu, Mar 20, 2014 at 7:38 PM, Sergio Pascual wrote: > Hi, I'm having problems reading the catalog in > http://cdsarc.u-strasbg.fr/viz-bin/Cat?J/PASP/110/863 (file > J_PASP_110_863.tar.gz) > > The files inside are compressed, so I uncompress them before. > > I create a reader object, (I'm using astropy 0.3, BTW) > > readme = 'ReadMe' > > r = ascii.get_reader(ascii.Cds, readme=readme) > > This reader works only for some specta. For example, m2v and goi works, > but the star 'o9v' doesn't work. Inserting this last star generates the > following exception > > >>> r.read('o9v.dat') > > /usr/lib64/python2.7/site-packages/astropy/io/ascii/core.pyc in read(self, > table) > 831 > 832 # Get the table column definitions > --> 833 self.header.get_cols(self.lines) > 834 > 835 cols = self.header.cols > > /usr/lib64/python2.7/site-packages/astropy/io/ascii/cds.pyc in > get_cols(self, lines) > 164 cols[-1].description += line.strip() > 165 else: > --> 166 raise ValueError('Line "%s" not parsable as > CDS header' % line) > 167 > 168 self.names = [x.name for x in cols] > > ValueError: Line "Bytes Format Units Label Explanations" not > parsable as CDS header > > > > Any hint? > > Sergio > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From demitri.muna at gmail.com Fri Mar 21 19:30:43 2014 From: demitri.muna at gmail.com (Demitri Muna) Date: Fri, 21 Mar 2014 19:30:43 -0400 Subject: [AstroPy] sample FITS file with variable-length array column? Message-ID: Hi, Does anyone have a sample FITS file that contains one or more variable length array columns? I'd prefer something that comes from an archive rather than something made by hand (e.g. astropy.fits). Cheers, Demitri _________________________________________ Demitri Muna Department of Astronomy Il Ohio State University http://scicoder.org/ From kylebarbary at gmail.com Fri Mar 28 11:25:33 2014 From: kylebarbary at gmail.com (Kyle Barbary) Date: Fri, 28 Mar 2014 10:25:33 -0500 Subject: [AstroPy] [ANN] sncosmo v0.4.0 Message-ID: Hi everyone, I'd like to announce the release of v0.4.0 of sncosmo, a Python package for supernova cosmology. sncosmo provides functionality for using empirical models of supernovae, such as generating synthetic photometry from models and fitting models to photometric light curves. Although several previous versions of sncosmo have been available, this is the first widely advertised release. landing page: https://sncosmo.github.io/ documentation: http://sncosmo.readthedocs.org/ code: https://github.com/sncosmo/sncosmo - Kyle Barbary -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Fri Mar 28 16:13:07 2014 From: embray at stsci.edu (Erik Bray) Date: Fri, 28 Mar 2014 16:13:07 -0400 Subject: [AstroPy] [ANN] [PyFITS] Re-release of versions 3.2.2 and 3.1.5 Message-ID: <5335D7D3.7070707@stsci.edu> Hello, Three days ago I released two patch releases for PyFITS: versions 3.2.2 and 3.1.5. I did not announce them because they only contained a minor bug fix. As usual the PyFITS release notes can be read at: http://pyfits.readthedocs.org/en/latest/appendix/changelog.html Unfortunately, due to an error on my part, the release distributions for those versions did not actually contain the fix they purported to contain! Therefore I have reuploaded new packages for PyFITS 3.2.2 and 3.1.5. The versions numbers are the same but the source distributions will have different md5 sums. Normally I would just increase the version number again, but in this case it was a matter of the intended patches just not being included in the uploaded files. Just thought I should announce this in the hopes of reducing confusion. If you upgraded to either PyFITS 3.2.2 or 3.1.5 before today please re-upgrade. Thanks, Erik