[SciPy-Dev] SciPy-Dev Digest, Vol 92, Issue 9

Sun Jun 12 22:32:31 EDT 2011

Hi again Chuck and thanks for helping me.

>Can you be a bit more specific about the
>grid and how you want to resample it? That is, do you need to sample it a
>random points, or just resample it on a different grid, etc.

Our task is not about resampling but (the simpler) task of interpolating
from a regular grid (e.g. earthquake ground shaking) to a collection of
arbitrary points (e.g. location of critical infrastructure) - see e.g the
illustration at http://en.wikipedia.org/wiki/Bilinear_interpolation

so we need

   1. A simple and robust way of interpolating and from a regular grid to
   points without overshoot and ringing. Smoothing is not important.
   2. Ability to hand NaN sensibly. E.g. if the is one grid point which is
   NaN only interpolation points near it should be NaN.

It is interesting how implementations of the hard problems (e.g. high order
splines on variable resolution) are abundant, whereas something as basic as
this is harder to find :-)

I am quite keen to have a go at implementing a simple bilinear interpolator
as a Python C-extension using numpy arrays if there is interest in it.
Meanwhile, if you could tell me where the LinearNDInterpolator package can
be found?

Cheers and thanks
Ole

On Sat, Jun 11, 2011 at 10:36 AM, <scipy-dev-request at scipy.org> wrote:

> Send SciPy-Dev mailing list submissions to
>        scipy-dev at scipy.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://mail.scipy.org/mailman/listinfo/scipy-dev
> or, via email, send a message with subject or body 'help' to
>        scipy-dev-request at scipy.org
>
> You can reach the person managing the list at
>        scipy-dev-owner at scipy.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-Dev digest..."
>
>
> Today's Topics:
>
>   1. Re: SciPy-Dev Digest, Vol 92, Issue 8 (Ole Nielsen)
>   2. Re: SciPy-Dev Digest, Vol 92, Issue 8 (Pauli Virtanen)
>   3. Re: SciPy-Dev Digest, Vol 92, Issue 8 (Charles R Harris)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 11 Jun 2011 08:04:14 +0700
> From: Ole Nielsen <ole.moller.nielsen at gmail.com>
> Subject: Re: [SciPy-Dev] SciPy-Dev Digest, Vol 92, Issue 8
> To: scipy-dev at scipy.org
> Message-ID: <BANLkTi=t-jMdfo5QJgNat1kqjZVup1u+dQ at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Thank you everyone for your suggestions. I didn't realise that
> RectBivariateSpline is a global method, but now it makes sense what I see
> in
> terms of ringing and handling of NaN.
>
> I am really interested in the local methods suggested by Chuck (bi-cubic
> interpolation or bilinear. However, looking in scipy.ndimage I couldn't
> find
> this kind of thing. Looking at GDAL this seems to be available only for
> resampling.
> I couldn't find LinearNDInterpolator either. Can anyone tell me where to
> find these packages please.
>
> Just to recab, what we need is a local bilinear interpolation from regular
> grid data to a collection of points which handles NaN in a sensible way.
>
> Many thanks
> Ole
>
>
> On Sat, Jun 11, 2011 at 12:00 AM, <scipy-dev-request at scipy.org> wrote:
>
> > Send SciPy-Dev mailing list submissions to
> >        scipy-dev at scipy.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >        http://mail.scipy.org/mailman/listinfo/scipy-dev
> > or, via email, send a message with subject or body 'help' to
> >        scipy-dev-request at scipy.org
> >
> > You can reach the person managing the list at
> >        scipy-dev-owner at scipy.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of SciPy-Dev digest..."
> >
> >
> > Today's Topics:
> >
> >   1. Bivariate interpolation and NaN (Ole Nielsen)
> >   2. Re: Bivariate interpolation and NaN (Charles R Harris)
> >   3. Re: Bivariate interpolation and NaN (Gael Varoquaux)
> >   4. Re: Bivariate interpolation and NaN (Charles R Harris)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Fri, 10 Jun 2011 13:26:01 +0700
> > From: Ole Nielsen <ole.moller.nielsen at gmail.com>
> > Subject: [SciPy-Dev] Bivariate interpolation and NaN
> > To: scipy-dev at scipy.org
> > Message-ID: <BANLkTi=fiCLzawFixTSN0GW+G7WTyUXRtQ at mail.gmail.com>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Dear Scipy developers
> >
> > I am working on a project where we need to interpolate from gridded data
> to
> > individual points.
> > We want it to be fast, bilinear (i.e. smooting is not important) and be
> > able
> > to deal with NaN in a sensible way.
> >
> > I looked at a few and settled for RectBivariateSpline which is part of
> > scipy.interpolate.
> > It works well but, we have encountered two problems:
> >
> >   1. If there is a single NaN in the grid, all interpolated points become
> >   NaN even if the surrounding pixels are valid floating point numbers. I
> > would
> >   have expected NaNs only for points whose immediate neighbours contain
> > NaN.
> >   2. We have noticed small 'overshoots', i.e. interpolated values may be
> >   outside the range of the gridded data. Can anyone tell me if this is
> >   expected?
> >
> > I am interested in thoughts on whether it is possible to achieve this. If
> > there is a need to reproduce my observations, I can provide that.
> >
> > Cheers and thanks
> > Ole Nielsen
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> >
> http://mail.scipy.org/pipermail/scipy-dev/attachments/20110610/0c475125/attachment-0001.html
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Fri, 10 Jun 2011 01:01:24 -0600
> > From: Charles R Harris <charlesr.harris at gmail.com>
> > Subject: Re: [SciPy-Dev] Bivariate interpolation and NaN
> > To: SciPy Developers List <scipy-dev at scipy.org>
> > Message-ID: <BANLkTi=DZrBc6G6HbOcHjJGPg+R1o4=XAA at mail.gmail.com>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > On Fri, Jun 10, 2011 at 12:26 AM, Ole Nielsen
> > <ole.moller.nielsen at gmail.com>wrote:
> >
> > > Dear Scipy developers
> > >
> > > I am working on a project where we need to interpolate from gridded
> data
> > to
> > > individual points.
> > > We want it to be fast, bilinear (i.e. smooting is not important) and be
> > > able to deal with NaN in a sensible way.
> > >
> > > I looked at a few and settled for RectBivariateSpline which is part of
> > > scipy.interpolate.
> > > It works well but, we have encountered two problems:
> > >
> > >    1. If there is a single NaN in the grid, all interpolated points
> > become
> > >    NaN even if the surrounding pixels are valid floating point numbers.
> I
> > would
> > >    have expected NaNs only for points whose immediate neighbours
> contain
> > NaN.
> > >    2. We have noticed small 'overshoots', i.e. interpolated values may
> be
> > >    outside the range of the gridded data. Can anyone tell me if this is
> > >    expected?
> > >
> > > I think both are expected. Splines are a global fit and nans will cause
> > global trouble. Likewise, splines can exhibit ringing. You can use a
> > smoothing spline to get around that, but it won't interpolate the data
> > points exactly. It sounds to me like you want something local, for
> instance
> > bi-cubic interpolation or bilinear (the algorithm name). There are some
> > tools for this sort of thing in scipy.ndimage, and tools like gdal or
> > imagemagick might also do what you want depending on the specifics of the
> > problem.
> >
> > Chuck
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> >
> http://mail.scipy.org/pipermail/scipy-dev/attachments/20110610/3be014a5/attachment-0001.html
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Fri, 10 Jun 2011 09:13:35 +0200
> > From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> > Subject: Re: [SciPy-Dev] Bivariate interpolation and NaN
> > To: SciPy Developers List <scipy-dev at scipy.org>
> > Message-ID: <20110610071335.GC11586 at phare.normalesup.org>
> > Content-Type: text/plain; charset=us-ascii
> >
> > On Fri, Jun 10, 2011 at 01:01:24AM -0600, Charles R Harris wrote:
> > >    I think both are expected. Splines are a global fit and nans will
> > cause
> > >    global trouble. Likewise, splines can exhibit ringing. You can use a
> > >    smoothing spline to get around that, but it won't interpolate the
> data
> > >    points exactly. It sounds to me like you want something local, for
> > >    instance bi-cubic interpolation or bilinear (the algorithm name).
> > There
> > >    are some tools for this sort of thing in scipy.ndimage, and tools
> like
> > >    gdal or imagemagick might also do what you want depending on the
> > specifics
> > >    of the problem.
> >
> > WENO interpolation (weighted essentially non-oscillatory):
> >
> >
> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.148.8505&rep=rep1&type=pdf
> >
> > http://www.scholarpedia.org/article/WENO_methods
> >
> > Adrian Townsend presented such interpolation scheme at scipy a couple of
> > years ago:
> > http://www.archive.org/details/scipy09_day1_14-Adrian_Townsend
> > He slides might actually be amongst the most readable material around:
> > http://conference.scipy.org/static/wiki/townsend_weno.pdf
> > anhd he has Python code:
> > http://memmett.github.com/PyWENO/
> >
> > Gael
> >
> >
> > ------------------------------
> >
> > Message: 4
> > Date: Fri, 10 Jun 2011 01:14:59 -0600
> > From: Charles R Harris <charlesr.harris at gmail.com>
> > Subject: Re: [SciPy-Dev] Bivariate interpolation and NaN
> > To: SciPy Developers List <scipy-dev at scipy.org>
> > Message-ID: <BANLkTikTVFHk9_1Jp6_VMAigJDb0nfXN+g at mail.gmail.com>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > On Fri, Jun 10, 2011 at 1:01 AM, Charles R Harris <
> > charlesr.harris at gmail.com
> > > wrote:
> >
> > >
> > >
> > > On Fri, Jun 10, 2011 at 12:26 AM, Ole Nielsen <
> > > ole.moller.nielsen at gmail.com> wrote:
> > >
> > >> Dear Scipy developers
> > >>
> > >> I am working on a project where we need to interpolate from gridded
> data
> > >> to individual points.
> > >> We want it to be fast, bilinear (i.e. smooting is not important) and
> be
> > >> able to deal with NaN in a sensible way.
> > >>
> > >> I looked at a few and settled for RectBivariateSpline which is part of
> > >> scipy.interpolate.
> > >> It works well but, we have encountered two problems:
> > >>
> > >>    1. If there is a single NaN in the grid, all interpolated points
> > >>    become NaN even if the surrounding pixels are valid floating point
> > numbers.
> > >>    I would have expected NaNs only for points whose immediate
> neighbours
> > >>    contain NaN.
> > >>    2. We have noticed small 'overshoots', i.e. interpolated values may
> > be
> > >>    outside the range of the gridded data. Can anyone tell me if this
> is
> > >>    expected?
> > >>
> > >> I think both are expected. Splines are a global fit and nans will
> cause
> > > global trouble. Likewise, splines can exhibit ringing. You can use a
> > > smoothing spline to get around that, but it won't interpolate the data
> > > points exactly. It sounds to me like you want something local, for
> > instance
> > > bi-cubic interpolation or bilinear (the algorithm name). There are some
> > > tools for this sort of thing in scipy.ndimage, and tools like gdal or
> > > imagemagick might also do what you want depending on the specifics of
> the
> > > problem.
> > >
> > >
> > I should say that I interpreted gridded as evenly spaced points on a
> square
> > grid. If that is not the case the LinearNDInterpolator might be your best
> > bet.
> >
> > Chuck
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> >
> http://mail.scipy.org/pipermail/scipy-dev/attachments/20110610/be546dcd/attachment-0001.html
> >
> > ------------------------------
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >
> > End of SciPy-Dev Digest, Vol 92, Issue 8
> > ****************************************
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/scipy-dev/attachments/20110611/79acf212/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Sat, 11 Jun 2011 01:48:51 +0000 (UTC)
> From: Pauli Virtanen <pav at iki.fi>
> Subject: Re: [SciPy-Dev] SciPy-Dev Digest, Vol 92, Issue 8
> To: scipy-dev at scipy.org
> Message-ID: <isuhi3$io4$1 at dough.gmane.org>
> Content-Type: text/plain; charset=UTF-8
>
> On Sat, 11 Jun 2011 08:04:14 +0700, Ole Nielsen wrote:
>
> > I am really interested in the local methods suggested by Chuck (bi-cubic
> > interpolation or bilinear. However, looking in scipy.ndimage I couldn't
> > find this kind of thing. Looking at GDAL this seems to be available only
> > for resampling.
> > I couldn't find LinearNDInterpolator either. Can anyone tell me where to
> > find these packages please.
>
> http://docs.scipy.org/doc/scipy/reference/ndimage.html
>
>
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.interpolation.map_coordinates.html
>
> http://docs.scipy.org/doc/scipy/reference/interpolate.html
>
>
>
> ------------------------------
>
> Message: 3
> Date: Fri, 10 Jun 2011 21:36:22 -0600
> From: Charles R Harris <charlesr.harris at gmail.com>
> Subject: Re: [SciPy-Dev] SciPy-Dev Digest, Vol 92, Issue 8
> To: SciPy Developers List <scipy-dev at scipy.org>
> Message-ID: <BANLkTikizWHwbEEdp=DJObyGnAbFODSCTA at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Fri, Jun 10, 2011 at 7:04 PM, Ole Nielsen
> <ole.moller.nielsen at gmail.com>wrote:
>
> > Thank you everyone for your suggestions. I didn't realise that
> > RectBivariateSpline is a global method, but now it makes sense what I see
> in
> > terms of ringing and handling of NaN.
> >
> > I am really interested in the local methods suggested by Chuck (bi-cubic
> > interpolation or bilinear. However, looking in scipy.ndimage I couldn't
> find
> > this kind of thing. Looking at GDAL this seems to be available only for
> > resampling.
> > I couldn't find LinearNDInterpolator either. Can anyone tell me where to
> > find these packages please.
> >
> > Just to recab, what we need is a local bilinear interpolation from
> regular
> > grid data to a collection of points which handles NaN in a sensible way.
> >
>
> Well, I looked a bit more closely at the ndimage interpolation routines and
> they are all spline based, so that leaves you with splines of order 1 as
> the
> only option there, which I think is equivalent to bilinear. The
> LinearNDInterpolator uses triangulation and will give a linear
> interpolation, but such depends on the triangulation. It's a shame that the
> offerings are so sparse, matplotlib's imshow has a lot more options.
> Scikits.image doesn't seem to concern itself with interpolation (yet), so I
> don't know where else you can look. Can you be a bit more specific about
> the
> grid and how you want to resample it? That is, do you need to sample it a
> random points, or just resample it on a different grid, etc.
>
> <snip>
>
> Chuck
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/scipy-dev/attachments/20110610/9a4ddd0f/attachment.html
>
> ------------------------------
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
> End of SciPy-Dev Digest, Vol 92, Issue 9
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110613/19f4530a/attachment.html>