[Numpy-discussion] Numpy-discussion Digest, Vol 22, Issue 109
Frank Lagor
dfranci at seas.upenn.edu
Fri Jul 25 16:01:32 EDT 2008
Thanks so much for your help on the '*' confusion. It makes sense now.
Thanks,
Frank
On Fri, Jul 25, 2008 at 3:57 PM, <numpy-discussion-request at scipy.org> wrote:
> Send Numpy-discussion mailing list submissions to
> numpy-discussion at scipy.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
> or, via email, send a message with subject or body 'help' to
> numpy-discussion-request at scipy.org
>
> You can reach the person managing the list at
> numpy-discussion-owner at scipy.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Numpy-discussion digest..."
>
> Today's Topics:
>
> 1. Re: [Cdat-discussion] Arrays containing NaNs (Charles Doutriaux)
> 2. numpy-1.1.1rc2 Mac binary - Please Test. (Christopher Burns)
> 3. Re: numpy-1.1.1rc2 Mac binary - Please Test. (Alan McIntyre)
> 4. curious problem with SVD (Frank Lagor)
> 5. Re: curious problem with SVD (Keith Goodman)
> 6. Re: curious problem with SVD (Keith Goodman)
> 7. Re: curious problem with SVD (Robert Kern)
> 8. Re: curious problem with SVD (Arnar Flatberg)
>
>
> ---------- Forwarded message ----------
> From: Charles Doutriaux <doutriaux1 at llnl.gov>
> To: Pierre GM <pgmdevlist at gmail.com>
> Date: Fri, 25 Jul 2008 11:43:41 -0700
> Subject: Re: [Numpy-discussion] [Cdat-discussion] Arrays containing NaNs
> Hi Pierre,
>
> Thanks for the answer, I'm ccing cdat's discussion list.
>
> It makes sense, that's also the way we develop things here NEVER assume
> what the user is going to do with the data BUT give the user the necessary
> tools to do what you're assuming he/she wants to do (as simple as possible)
>
> Thanks again for the answer.
>
> C.
>
>
> Pierre GM wrote:
>
>> Oh, I guess this one's for me...
>>
>> On Thursday 01 January 1970 04:21:03 Charles Doutriaux wrote:
>>
>>
>>
>>> Basically it was suggested to automarically mask NaN (and Inf ?) when
>>> creating ma.
>>> I'm sure you already thought of this on this list and was curious to
>>> know why you decided not to do it.
>>>
>>>
>>
>> Because it's always best to let the user decide what to do with his/her
>> data and not impose anything ?
>>
>> Masking a point doesn't necessarily mean that the point is invalid (in the
>> sense of NaNs/Infs), just that it doesn't satisfy some particular condition.
>> In that sense, masks act as selecting tools.
>>
>> By forcing invalid data to be masked at the creation of an array, you run
>> the risk to tamper with the (potential) physical meaning of the mask you
>> have given as input, and/or miss the fact that some data are actually
>> invalid when you don't expect it to be.
>>
>> Let's take an example: I want to analyze sea surface temperatures at the
>> world scale. The data comes as a regular 2D ndarray, with NaNs for missing
>> or invalid data. In a first step, I create a masked array of this data,
>> filtering out the land masses by a predefined geographical mask. The
>> remaining NaNs in the masked array indicate areas where the sensor failed...
>> It's an important information I would probably have missed by masking all
>> the NaNs at first...
>>
>>
>> As Eric F. suggested, you can use numpy.ma.masked_invalid to create a
>> masked array with NaNs/Infs filtered out:
>>
>>
>>
>>> import numpy as np,. numpy.ma as ma
>>>>> x = np.array([1,2,None,4], dtype=float)
>>>>> x
>>>>>
>>>>>
>>>> array([ 1., 2., NaN, 4.])
>>
>>
>>> mx = ma.masked_invalid(x)
>>>>> mx
>>>>>
>>>>>
>>>> masked_array(data = [1.0 2.0 -- 4.0],
>> mask = [False False True False],
>> fill_value=1e+20)
>>
>> Note that the underlying data still has NaNs/Infs:
>>
>>
>>> mx._data
>>>>>
>>>>>
>>>> array([ 1., 2., NaN, 4.])
>>
>> You can also use the ma.fix_invalid function: it creates a mask where the
>> data is not finite (NaNs/Infs), and set the corresponding points to
>> fill_value.
>>
>>
>>> mx = ma.fix_invalid(x, fill_value=999)
>>>>> mx
>>>>>
>>>>>
>>>> masked_array(data = [1.0 2.0 -- 4.0],
>> mask = [False False True False],
>> fill_value=1e+20)
>>
>>
>>> mx._data
>>>>>
>>>>>
>>>> array([ 1., 2., 999., 4.])
>>
>>
>> The advantage of the second approach is that you no longer have NaNs/Infs
>> in the underlying data, which speeds things up during computation. The
>> obvious disadvantage is that you no longer know where the data was
>> invalid...
>>
>>
>>
>
>
>
>
> ---------- Forwarded message ----------
> From: "Christopher Burns" <cburns at berkeley.edu>
> To: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Date: Fri, 25 Jul 2008 11:48:13 -0700
> Subject: [Numpy-discussion] numpy-1.1.1rc2 Mac binary - Please Test.
> Reminder, please test the Mac installer for rc2 so we have time to fix any
> bugs before the release next week.
>
> Also, I committed my build script to the trunk/tools/osxbuild. bdist_mpkg
> 0.4.3 is required.
>
> Thank you,
> Chris
>
> On Thu, Jul 24, 2008 at 11:03 AM, Jarrod Millman <millman at berkeley.edu>
> wrote:
>
>> Hello,
>>
>> The 1.1.1rc2 is now available:
>> http://svn.scipy.org/svn/numpy/tags/1.1.1rc2
>>
>> The source tarball is here:
>> http://cirl.berkeley.edu/numpy/numpy-1.1.1rc2.tar.gz
>>
>> Here is the universal Mac binary:
>> http://cirl.berkeley.edu/numpy/numpy-1.1.1rc2-py2.5-macosx10.5.dmg
>>
>> David Cournapeau will be creating a 1.1.1rc2 Windows binary in next few
>> days.
>>
>> Please test this release ASAP and let us know if there are any
>> problems. If there are no show stoppers, this will likely become the
>> 1.1.1 release.
>>
>> Thanks,
>>
>> --
>> Jarrod Millman
>> Computational Infrastructure for Research Labs
>> 10 Giannini Hall, UC Berkeley
>> phone: 510.643.4014
>> http://cirl.berkeley.edu/
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at scipy.org
>> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
>
> ---------- Forwarded message ----------
> From: "Alan McIntyre" <alan.mcintyre at gmail.com>
> To: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Date: Fri, 25 Jul 2008 15:00:42 -0400
> Subject: Re: [Numpy-discussion] numpy-1.1.1rc2 Mac binary - Please Test.
> On Fri, Jul 25, 2008 at 2:48 PM, Christopher Burns <cburns at berkeley.edu>
> wrote:
> > Reminder, please test the Mac installer for rc2 so we have time to fix
> any
> > bugs before the release next week.
>
> I just tried it; it installs with no problems and tests run with no
> failures.
>
>
>
> ---------- Forwarded message ----------
> From: "Frank Lagor" <dfranci at seas.upenn.edu>
> To: numpy-discussion at scipy.org
> Date: Fri, 25 Jul 2008 15:32:56 -0400
> Subject: [Numpy-discussion] curious problem with SVD
> Perhaps I do not understand something properly, if so could someone please
> explain the behavior I notice with numpy.linalg.svd when acting on arrays.
> It gives the incorrect answer, but works fine with matrices. My numpy is
> 1.1.0.
>
> >>> R = n.array([[3.6,.35],[.35,1.8]])
> >>> V,D,W = n.linalg.svd(R)
> >>> V*n.diag(D)*W.transpose()
> array([[ 3.5410365 , 0. ],
> [ 0. , 1.67537611]])
> >>> R = n.matrix([[3.6,.35],[.35,1.8]])
> >>> V,D,W = n.linalg.svd(R)
> >>> V*n.diag(D)*W.transpose()
> matrix([[ 3.6 , 0.35],
> [ 0.35, 1.8 ]])
>
> Thanks in advance,
> Frank
> --
> Frank Lagor
> Ph.D. Candidate
> Mechanical Engineering and Applied Mechanics
> University of Pennsylvania
>
>
> ---------- Forwarded message ----------
> From: "Keith Goodman" <kwgoodman at gmail.com>
> To: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Date: Fri, 25 Jul 2008 12:36:28 -0700
> Subject: Re: [Numpy-discussion] curious problem with SVD
> On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <dfranci at seas.upenn.edu>
> wrote:
> > Perhaps I do not understand something properly, if so could someone
> please
> > explain the behavior I notice with numpy.linalg.svd when acting on
> arrays.
> > It gives the incorrect answer, but works fine with matrices. My numpy is
> > 1.1.0.
> >
> >>>> R = n.array([[3.6,.35],[.35,1.8]])
> >>>> V,D,W = n.linalg.svd(R)
> >>>> V*n.diag(D)*W.transpose()
> > array([[ 3.5410365 , 0. ],
> > [ 0. , 1.67537611]])
> >>>> R = n.matrix([[3.6,.35],[.35,1.8]])
> >>>> V,D,W = n.linalg.svd(R)
> >>>> V*n.diag(D)*W.transpose()
> > matrix([[ 3.6 , 0.35],
> > [ 0.35, 1.8 ]])
>
> '*' does element-by-element multiplication for arrays but matrix
> multiplication for matrices.
>
>
>
> ---------- Forwarded message ----------
> From: "Keith Goodman" <kwgoodman at gmail.com>
> To: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Date: Fri, 25 Jul 2008 12:39:23 -0700
> Subject: Re: [Numpy-discussion] curious problem with SVD
> On Fri, Jul 25, 2008 at 12:36 PM, Keith Goodman <kwgoodman at gmail.com>
> wrote:
> > On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <dfranci at seas.upenn.edu>
> wrote:
> >> Perhaps I do not understand something properly, if so could someone
> please
> >> explain the behavior I notice with numpy.linalg.svd when acting on
> arrays.
> >> It gives the incorrect answer, but works fine with matrices. My numpy
> is
> >> 1.1.0.
> >>
> >>>>> R = n.array([[3.6,.35],[.35,1.8]])
> >>>>> V,D,W = n.linalg.svd(R)
> >>>>> V*n.diag(D)*W.transpose()
> >> array([[ 3.5410365 , 0. ],
> >> [ 0. , 1.67537611]])
> >>>>> R = n.matrix([[3.6,.35],[.35,1.8]])
> >>>>> V,D,W = n.linalg.svd(R)
> >>>>> V*n.diag(D)*W.transpose()
> >> matrix([[ 3.6 , 0.35],
> >> [ 0.35, 1.8 ]])
> >
> > '*' does element-by-element multiplication for arrays but matrix
> > multiplication for matrices.
>
> As a check (for the array case):
>
> >> n.dot(V, n.dot(n.diag(D), W.transpose())) # That's hard to read!
>
> array([[ 3.6 , 0.35],
> [ 0.35, 1.8 ]])
>
>
>
> ---------- Forwarded message ----------
> From: "Robert Kern" <robert.kern at gmail.com>
> To: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Date: Fri, 25 Jul 2008 14:50:24 -0500
> Subject: Re: [Numpy-discussion] curious problem with SVD
> On Fri, Jul 25, 2008 at 14:32, Frank Lagor <dfranci at seas.upenn.edu> wrote:
> > Perhaps I do not understand something properly, if so could someone
> please
> > explain the behavior I notice with numpy.linalg.svd when acting on
> arrays.
> > It gives the incorrect answer, but works fine with matrices. My numpy is
> > 1.1.0.
> >
> >>>> R = n.array([[3.6,.35],[.35,1.8]])
> >>>> V,D,W = n.linalg.svd(R)
> >>>> V*n.diag(D)*W.transpose()
>
> For regular arrays, * is element-wise multiplication, not matrix
> multiplication. For matrix objects, * is matrix multiplication.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> -- Umberto Eco
>
>
>
> ---------- Forwarded message ----------
> From: "Arnar Flatberg" <arnar.flatberg at gmail.com>
> To: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Date: Fri, 25 Jul 2008 21:53:09 +0200
> Subject: Re: [Numpy-discussion] curious problem with SVD
>
> On Fri, Jul 25, 2008 at 9:39 PM, Keith Goodman <kwgoodman at gmail.com>
> wrote:
>
>> On Fri, Jul 25, 2008 at 12:36 PM, Keith Goodman <kwgoodman at gmail.com>
>> wrote:
>> > On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <dfranci at seas.upenn.edu>
>> wrote:
>> >> Perhaps I do not understand something properly, if so could someone
>> please
>> >> explain the behavior I notice with numpy.linalg.svd when acting on
>> arrays.
>> >> It gives the incorrect answer, but works fine with matrices. My numpy
>> is
>>
>> > '*' does element-by-element multiplication for arrays but matrix
>> > multiplication for mat
>> >> n.dot(V, n.dot(n.diag(D), W.transpose())) # That's hard to read!
>
>
> Just two small points:
>
> 1.) Broadcasting may be easier on the eye ... well, atleast when you have
> gotten used to it
> Then the above is np.dot(V*D, W)
>
> 2.) Also, note that the right hand side eigenvectors in numpy's svd routine
> is ordered by rows!
> Yes, I know this is confusing as it is different from just about any other
> linear algebra software out there, but the documentation is clear. It is
> also a little inconsistent with eig and eigh, some more experienced user can
> probably answer on why it is like that?
>
> Arnar
>
>
>
>> <http://projects.scipy.org/mailman/listinfo/numpy-discussion>
>>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
--
Frank Lagor
Ph.D. Candidate
Mechanical Engineering and Applied Mechanics
University of Pennsylvania
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20080725/308a53f6/attachment.html>
More information about the NumPy-Discussion
mailing list