[AstroPy] astropy and photutils slow
Stephen Bailey
stephenbailey at lbl.gov
Tue Sep 13 16:17:18 EDT 2016
What version of numpy and astropy do you have? (numpy.__version__ and
astropy.__version__).
As another data point, using numpy/1.11.1 and astropy/1.2.1 I get much
faster results than you:
On Tue, Sep 13, 2016 at 12:05 PM, David Nidever <davidnidever at gmail.com>
wrote:
> Hi all,
>
> I'm pretty new to astropy and to python in general. I've been trying out
> some of the astropy and photutils photometry tools, but I'm finding that
> they
> are running pretty slow. I wanted to double-check that I'm not doing
> something
> wrong.
>
> Below is an example of running the astropy sigma clipped statistics routine
> on a fake 2Kx4K image (since I'm working on DECam data). This takes
> almost 20s to run even though similar statistics using numpy take ~0.1s
> or so.
>
> >>> import numpy as np
> >>> from astropy.stats import sigma_clipped_stats
> >>> im = np.random.rand(4096,2048)*10.0+100.0
> >>> timeit mean, median, std = sigma_clipped_stats(im, sigma=3.0, iters=5)
> 1 loop, best of 3: 18.9 s per loop
>
1.84s per loop for me. Still slow, but not 18.9s.
> It seems like it might be the use of the numpy masked array module that is
> slowing things down. Just using the masked array median versus the
> normal numpy median takes ~30x longer.
>
> >>> timeit np.median(im)
> 10 loops, best of 3: 109 ms per loop
>
103 ms for me; i.e. this isn't just that I have a super fast machine and
you have a super slow one.
> >>> timeit np.ma.median(im)
> 1 loop, best of 3: 2.71 s per loop
>
104 ms for me. i.e. with numpy 1.11.1 I don't see a radical difference
between masked array or not.
Stephen
>
> I'm finding similar speed issues with the photutils routines (e.g.
> background2D)
> that I believe also use masked arrays.
>
> Is there something I can do to speed things up?
>
> Thanks!
> David
>
> --
> Dr. David Lee Nidever
> Survey Data Scientist, NOAO
> 950 N. Cherry Ave.
> Tucson, AZ 85719
> (520) 318-8368
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> https://mail.scipy.org/mailman/listinfo/astropy
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20160913/f65a6a52/attachment.html>
More information about the AstroPy
mailing list