From elias.pampalk at gmail.com  Wed Jul  1 12:18:51 2009
From: elias.pampalk at gmail.com (Elias Pampalk)
Date: Wed, 1 Jul 2009 17:18:51 +0100
Subject: [SciPy-user] scipy.stats Documentation?
In-Reply-To: <43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>
References: <43958ee60906301516s637b180fm35b96d5b61a1b549@mail.gmail.com>
	<4A4A92A6.9070503@apstat.com>
	<43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>
Message-ID: <4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>

If I don't want to be searching through the source code - where should I be
looking for up-to-date documentation?

Searching via Google for the documentation on scipy.stats Mann Whitney U
test I found:

http://www.scipy.org/doc/api_docs/SciPy.stats.stats.html#mannwhitneyu
http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.mstats.mannw
hitneyu.html

Both seem outdated compared to what I found here:
http://svn.scipy.org/svn/scipy/trunk/scipy/stats/stats.py
http://projects.scipy.org/scipy/browser/tags/0.7.1/scipy/stats/stats.py

Thank you,
Elias






From cournape at gmail.com  Wed Jul  1 12:33:22 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 2 Jul 2009 01:33:22 +0900
Subject: [SciPy-user] scipy.stats Documentation?
In-Reply-To: <4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>
References: <43958ee60906301516s637b180fm35b96d5b61a1b549@mail.gmail.com>
	<4A4A92A6.9070503@apstat.com>
	<43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>
	<4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>
Message-ID: <5b8d13220907010933j7463ec93tb0bb92f02fbf9308@mail.gmail.com>

On Thu, Jul 2, 2009 at 1:18 AM, Elias Pampalk<elias.pampalk at gmail.com> wrote:
> If I don't want to be searching through the source code - where should I be
> looking for up-to-date documentation?

The most uptodate documentation is almost always the docstring. If you
use an advanced interpreter such as ipython, you get the docstring
without looking at the sources:

import numpy as np
help np.mean

There is a current effort to bring the official documentation on par
with the docstring, but I don't think it has been done completely for
scipy.stats yet.

David


From Scott.Daniels at Acm.Org  Wed Jul  1 13:21:22 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 01 Jul 2009 10:21:22 -0700
Subject: [SciPy-user] assign to diagonal values?
In-Reply-To: <3d375d730905231433p5003abfak9dca94a975825a39@mail.gmail.com>
References: <47A128A5.7010406@sci.utah.edu>	<ce557a360801302237l15c7ab81ub84f2aec38654115@mail.gmail.com>
	<4A18647D.50207@american.edu>	<3d375d730905231405t474f1e63n67d68d55f317e20a@mail.gmail.com>
	<4A186A21.5040102@american.edu>
	<3d375d730905231433p5003abfak9dca94a975825a39@mail.gmail.com>
Message-ID: <h2g5hc$khj$1@ger.gmane.org>

Robert Kern wrote:
> On Sat, May 23, 2009 at 16:26, Alan G Isaac <aisaac at american.edu> wrote:
>>> On Sat, May 23, 2009 at 16:02, Alan G Isaac wrote:
>>>> On 1/31/2008 1:37 AM Anne Archibald apparently wrote:
>>>>> m[range(n),range(n)]=new_diagonal
>>>> Will that work with range objects (in Python 3)?
>> On 5/23/2009 5:05 PM Robert Kern apparently wrote:
>>> No. The automatic conversion to arrays does not consume iterators (nor
>>> will it when we port to Python 3).
>> Sure, but range objects are not iterators.
>> They are "almost" sequences.
> 
> The answer is still no. Perhaps someone will write special support for
> that type when we do the Python 3 port, but there's nothing in numpy
> that would make it work automatically. For example, xrange() does not
> work as an index with the current numpy.

Well, ranges are more capable than you think in Python 3:
     v = range(25)
     print (v[3], v[0], v[22], v)
prints:
     3 0 22 range(0, 25)

-Scott



From pav at iki.fi  Wed Jul  1 13:57:09 2009
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 1 Jul 2009 17:57:09 +0000 (UTC)
Subject: [SciPy-user] scipy.stats Documentation?
References: <43958ee60906301516s637b180fm35b96d5b61a1b549@mail.gmail.com>
	<4A4A92A6.9070503@apstat.com>
	<43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>
	<4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>
Message-ID: <h2g81l$skn$1@ger.gmane.org>

On 2009-07-01, Elias Pampalk <elias.pampalk at gmail.com> wrote:
> If I don't want to be searching through the source code - where should I be
> looking for up-to-date documentation?
>
> Searching via Google for the documentation on scipy.stats Mann Whitney U
> test I found:
>
> http://www.scipy.org/doc/api_docs/SciPy.stats.stats.html#mannwhitneyu
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.mstats.mannwhitneyu.html

The second one is up-to-date but it's for 
scipy.stats.mstats.mannwhitneyu, not scipy.stats.mannwhitneyu.

Apparently, the main stats.mannwhitneyu function was not included 
in the documentation. Fixed, should appear tomorrow.

-- 
Pauli Virtanen



From robert.kern at gmail.com  Wed Jul  1 14:38:49 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 1 Jul 2009 13:38:49 -0500
Subject: [SciPy-user] assign to diagonal values?
In-Reply-To: <h2g5hc$khj$1@ger.gmane.org>
References: <47A128A5.7010406@sci.utah.edu>
	<ce557a360801302237l15c7ab81ub84f2aec38654115@mail.gmail.com> 
	<4A18647D.50207@american.edu>
	<3d375d730905231405t474f1e63n67d68d55f317e20a@mail.gmail.com> 
	<4A186A21.5040102@american.edu>
	<3d375d730905231433p5003abfak9dca94a975825a39@mail.gmail.com> 
	<h2g5hc$khj$1@ger.gmane.org>
Message-ID: <3d375d730907011138g204ffb57kfc7cd12b9a8b45ce@mail.gmail.com>

On Wed, Jul 1, 2009 at 12:21, Scott David Daniels<Scott.Daniels at acm.org> wrote:
> Robert Kern wrote:
>> On Sat, May 23, 2009 at 16:26, Alan G Isaac <aisaac at american.edu> wrote:
>>>> On Sat, May 23, 2009 at 16:02, Alan G Isaac wrote:
>>>>> On 1/31/2008 1:37 AM Anne Archibald apparently wrote:
>>>>>> m[range(n),range(n)]=new_diagonal
>>>>> Will that work with range objects (in Python 3)?
>>> On 5/23/2009 5:05 PM Robert Kern apparently wrote:
>>>> No. The automatic conversion to arrays does not consume iterators (nor
>>>> will it when we port to Python 3).
>>> Sure, but range objects are not iterators.
>>> They are "almost" sequences.
>>
>> The answer is still no. Perhaps someone will write special support for
>> that type when we do the Python 3 port, but there's nothing in numpy
>> that would make it work automatically. For example, xrange() does not
>> work as an index with the current numpy.
>
> Well, ranges are more capable than you think in Python 3:
> ? ? v = range(25)
> ? ? print (v[3], v[0], v[22], v)
> prints:
> ? ? 3 0 22 range(0, 25)

No, they are exactly as capable as I think, i.e. as capable as
xrange() is in Python 2:

  In [10]: v = xrange(25)

  In [11]: print v[3], v[0], v[22], v
  3 0 22 xrange(25)

Quite simply, numpy does not support arbitrary sequences and
sequence-like objects as indices. If the eventual numpy port to Python
3 supports range() objects as indices, it will be because someone will
have written special code for it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From fperez.net at gmail.com  Wed Jul  1 15:38:12 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 1 Jul 2009 12:38:12 -0700
Subject: [SciPy-user] assign to diagonal values?
In-Reply-To: <2bc7a5a50801302128g79da7ea6jc95ef9abb00b3785@mail.gmail.com>
References: <47A128A5.7010406@sci.utah.edu>
	<Mahogany-0.67.0-1804-20080130-220801.00@american.edu> 
	<DD23E9F4-9327-4371-8767-6DFB77FADAD7@cs.toronto.edu>
	<2bc7a5a50801302128g79da7ea6jc95ef9abb00b3785@mail.gmail.com>
Message-ID: <db6b5ecc0907011238r5e611f51y2c01052bbc0ae83b@mail.gmail.com>

On Wed, Jan 30, 2008 at 10:28 PM, Anand
Patil<anand.prabhakar.patil at gmail.com> wrote:
> Yeah, that's awesome. I have so many for-loops littering my code for
> setting diagonals. Here's an nd-version:
>
> def setdiag(a, d):
> ? ?assert(all([s == len(d) for s in a.shape]))
> ? ?stride = 1+sum(cumprod(a.shape[:-1]))
> ? ?a.flat[::stride] = d
>

If someone feels like reviewing this ticket:

http://projects.scipy.org/numpy/attachment/ticket/1132/numpy-index-funcs.diff

it has the above and a few more utilities, with docs and tests.

Cheers,

f


From lorenzo.isella at gmail.com  Thu Jul  2 09:25:42 2009
From: lorenzo.isella at gmail.com (Lorenzo Isella)
Date: Thu, 02 Jul 2009 15:25:42 +0200
Subject: [SciPy-user] Creating a Matrix from a Sum
Message-ID: <4A4CB556.6090203@gmail.com>

Dear All,
I need some help to efficiently write an array manipulation. I suspect 
this could be a one-liner.
Assume that you have a histogram of observations that you store in a 
vector x.
Let us say that its i-th entry, x_i, corresponds to the number of 
observations in the i-th channel, for i=1,2...N. (or 0,1...N-1, please 
let me know if there is any potential 0/1 pitfall in the following).
Now, for any two channels i and j, I want to calculate the probability 
of having an observation  in any other channel k, where  k>=max(i,j).
That is to say

P(i,j)=sum_{k=max(i,j)}^N x_k /C,
where C=sum_{i=1}^N x_i is just a normalization factor.

Does anyone know a good way of writing this in a code to get the P(i,j) 
matrix?
Many thanks

Lorenzo


From Dharhas.Pothina at twdb.state.tx.us  Thu Jul  2 11:58:43 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Thu, 02 Jul 2009 10:58:43 -0500
Subject: [SciPy-user] Add a column to masked array.
Message-ID: <4A4C92E3.63BA.009B.0@twdb.state.tx.us>

Hi,

I have a masked array 'data' where data.shape = (1805,4) and another masked array 'a' where a.shape = (1805,) 

I would like to generate a new masked array with the 'a' as the last column , i.e. the final shape should be (1805,5). 

I've tried np.hstack which gave an error 'arrays must have same number of dimensions' and ma.hstack just combines both masked arrays into a list.

how would I go about doing what I need?

thanks

- dharhas






From pgmdevlist at gmail.com  Thu Jul  2 12:57:34 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 2 Jul 2009 12:57:34 -0400
Subject: [SciPy-user] Add a column to masked array.
In-Reply-To: <4A4C92E3.63BA.009B.0@twdb.state.tx.us>
References: <4A4C92E3.63BA.009B.0@twdb.state.tx.us>
Message-ID: <6E2CE761-723F-4CB4-8452-256115F18D27@gmail.com>


On Jul 2, 2009, at 11:58 AM, Dharhas Pothina wrote:

> Hi,
>
> I have a masked array 'data' where data.shape = (1805,4) and another  
> masked array 'a' where a.shape = (1805,)
>
> I would like to generate a new masked array with the 'a' as the last  
> column , i.e. the final shape should be (1805,5).

Try ma.column_stack

 >>> x=ma.masked_all((10,3))
 >>> y=np.random.rand(10)
 >>> ma.column_stack((x,y)).shape
(10,4)


> I've tried np.hstack which gave an error 'arrays must have same  
> number of dimensions'

The corresponding standard numpy functions don't know how to process  
the mask and tend to fail miserably.

> and ma.hstack just combines both masked arrays into a list.

I've never been able to remember what vstack, hstack and stack do  
respectively, so I use error/trial and the online help...



From elias.pampalk at gmail.com  Thu Jul  2 13:43:22 2009
From: elias.pampalk at gmail.com (Elias Pampalk)
Date: Thu, 2 Jul 2009 18:43:22 +0100
Subject: [SciPy-user]  scipy.stats.stats mannwhitneyu vs ranksums?
In-Reply-To: <5b8d13220907010933j7463ec93tb0bb92f02fbf9308@mail.gmail.com>
References: <43958ee60906301516s637b180fm35b96d5b61a1b549@mail.gmail.com>	<4A4A92A6.9070503@apstat.com>	<43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>	<4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>
	<5b8d13220907010933j7463ec93tb0bb92f02fbf9308@mail.gmail.com>
Message-ID: <4a4cf1bf.05a4100a.7834.ffffcf2e@mx.google.com>

Why does stats.py contain both mannwhitneyu and ranksums?

See also:
http://en.wikipedia.org/wiki/Mann-Whitney-Wilcoxon_test

"Mann-Whitney U test (also called the Mann-Whitney-Wilcoxon (MWW), Wilcoxon
rank-sum test, or Wilcoxon-Mann-Whitney test)"

http://www.mathworks.com/access/helpdesk/help/toolbox/stats/index.html?/acce
ss/helpdesk/help/toolbox/stats/ranksum.html

"ranksum [...] The test is equivalent to a Mann-Whitney U-test."

Thanks,
Elias

PS. @David, @Pauli: Thank you for answering my question wrt documentation
yesterday!



From Dharhas.Pothina at twdb.state.tx.us  Thu Jul  2 13:52:17 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Thu, 02 Jul 2009 12:52:17 -0500
Subject: [SciPy-user] Add a column to masked array.
In-Reply-To: <6E2CE761-723F-4CB4-8452-256115F18D27@gmail.com>
References: <4A4C92E3.63BA.009B.0@twdb.state.tx.us>
	<6E2CE761-723F-4CB4-8452-256115F18D27@gmail.com>
Message-ID: <4A4CAD81.63BA.009B.0@twdb.state.tx.us>


thank you. ma.column_stack works great.

- d

>>> Pierre GM <pgmdevlist at gmail.com> 7/2/2009 11:57 AM >>>

On Jul 2, 2009, at 11:58 AM, Dharhas Pothina wrote:

> Hi,
>
> I have a masked array 'data' where data.shape = (1805,4) and another  
> masked array 'a' where a.shape = (1805,)
>
> I would like to generate a new masked array with the 'a' as the last  
> column , i.e. the final shape should be (1805,5).

Try ma.column_stack

 >>> x=ma.masked_all((10,3))
 >>> y=np.random.rand(10)
 >>> ma.column_stack((x,y)).shape
(10,4)


> I've tried np.hstack which gave an error 'arrays must have same  
> number of dimensions'

The corresponding standard numpy functions don't know how to process  
the mask and tend to fail miserably.

> and ma.hstack just combines both masked arrays into a list.

I've never been able to remember what vstack, hstack and stack do  
respectively, so I use error/trial and the online help...

_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org 
http://mail.scipy.org/mailman/listinfo/scipy-user



From dwf at cs.toronto.edu  Thu Jul  2 19:54:33 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Thu, 2 Jul 2009 19:54:33 -0400
Subject: [SciPy-user] scipy.stats.stats mannwhitneyu vs ranksums?
In-Reply-To: <4a4cf1bf.05a4100a.7834.ffffcf2e@mx.google.com>
References: <43958ee60906301516s637b180fm35b96d5b61a1b549@mail.gmail.com>	<4A4A92A6.9070503@apstat.com>	<43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>	<4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>
	<5b8d13220907010933j7463ec93tb0bb92f02fbf9308@mail.gmail.com>
	<4a4cf1bf.05a4100a.7834.ffffcf2e@mx.google.com>
Message-ID: <4F46BA01-652B-4AEA-84CE-277D54F26693@cs.toronto.edu>

There's also wilcoxon, which is related but subtly different. There's  
an open ticket for clarifying these docs:

	http://projects.scipy.org/scipy/ticket/901

The discussion there might illuminate things.

David

On 2-Jul-09, at 1:43 PM, Elias Pampalk wrote:

> Why does stats.py contain both mannwhitneyu and ranksums?
>
> See also:
> http://en.wikipedia.org/wiki/Mann-Whitney-Wilcoxon_test
>
> "Mann-Whitney U test (also called the Mann-Whitney-Wilcoxon (MWW),  
> Wilcoxon
> rank-sum test, or Wilcoxon-Mann-Whitney test)"
>
> http://www.mathworks.com/access/helpdesk/help/toolbox/stats/index.html?/acce
> ss/helpdesk/help/toolbox/stats/ranksum.html
>
> "ranksum [...] The test is equivalent to a Mann-Whitney U-test."
>
> Thanks,
> Elias
>
> PS. @David, @Pauli: Thank you for answering my question wrt  
> documentation
> yesterday!
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From oanjao at yahoo.com  Thu Jul  2 22:54:09 2009
From: oanjao at yahoo.com (Craig Finch)
Date: Thu, 2 Jul 2009 19:54:09 -0700 (PDT)
Subject: [SciPy-user] Scipy weave errors: undefined symbols when importing
	compiled extension
Message-ID: <524620.24680.qm@web30402.mail.mud.yahoo.com>


I have two computers that have almost identical Python installations,
but one of them has problems with inline Weave code.  Here is a simple
test script that I am using:

#!/usr/bin/env python
from scipy import weave
input_val = 10

code=r"""return_val = 10*input_val;"""

print weave.inline(code, ['input_val'], headers=[], compiler='gcc')

It runs successfully on one computer with the following configuration:
Python 2.5.2 (r252:60911, Apr 17 2009, 18:42:17) 
[GCC 4.1.1 (Gentoo 4.1.1-r3)] on linux2
Scipy 0.7.0
Numpy 1.3.0

On the other computer, it fails with the following message:
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    print weave.inline(code, ['input_val'], headers=[], compiler='gcc')
  File "/home/cfinch/lib/python2.5/site-packages/scipy/weave/inline_tools.py", line 335, in inline
    **kw)
  File "/home/cfinch/lib/python2.5/site-packages/scipy/weave/inline_tools.py", line 468, in compile_function
    exec 'import ' + module_name
  File "<string>", line 1, in <module>
ImportError: /home/cfinch/.python25_compiled/sc_71b2502f9a0b0ca9f89b0cdc7ad3819e0.so: undefined symbol: _ZNSt8ios_base4InitD1Ev

I used nm to check, and that symbol is indeed present in the compiled .so file.

The configuration of this computer is:
Python 2.5.4 (r254:67916, Apr 22 2009, 15:52:10) 
[GCC 4.1.1 20070105 (Red Hat 4.1.1-52)] on linux2
Scipy 0.7.0
Numpy 1.3.0
One
complication is that the default compiler on this system is icc, and
numpy distutils doesn't offer a way to force it to use gcc for
everything.

I have tried everything I can think of--deleting the
files in .python25_compiled, rebuilding Scipy to ensure it was built
with gcc, even hacking Numpy's distutils to force it to compile the
extension with gcc.

I also ran the unit tests that come with
weave.  Running weave.test() produces no errors, but running
weave.test("full") produces lots of errors:

Ran 390 tests in 145.246s
FAILED (KNOWNFAIL=6, errors=196)
<nose.result.TextTestResult run=390 errors=196 failures=0>

There are no errors on the other system.

Can anyone suggest a way to fix this?

    Craig



      


From david at ar.media.kyoto-u.ac.jp  Fri Jul  3 03:19:36 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 03 Jul 2009 16:19:36 +0900
Subject: [SciPy-user] Scipy weave errors: undefined symbols when
 importing compiled extension
In-Reply-To: <524620.24680.qm@web30402.mail.mud.yahoo.com>
References: <524620.24680.qm@web30402.mail.mud.yahoo.com>
Message-ID: <4A4DB108.8050407@ar.media.kyoto-u.ac.jp>

Craig Finch wrote:
> I have two computers that have almost identical Python installations,
> but one of them has problems with inline Weave code.  Here is a simple
> test script that I am using:
>
> #!/usr/bin/env python
> from scipy import weave
> input_val = 10
>
> code=r"""return_val = 10*input_val;"""
>
> print weave.inline(code, ['input_val'], headers=[], compiler='gcc')
>
> It runs successfully on one computer with the following configuration:
> Python 2.5.2 (r252:60911, Apr 17 2009, 18:42:17) 
> [GCC 4.1.1 (Gentoo 4.1.1-r3)] on linux2
> Scipy 0.7.0
> Numpy 1.3.0
>
> On the other computer, it fails with the following message:
> Traceback (most recent call last):
>   File "test.py", line 12, in <module>
>     print weave.inline(code, ['input_val'], headers=[], compiler='gcc')
>   File "/home/cfinch/lib/python2.5/site-packages/scipy/weave/inline_tools.py", line 335, in inline
>     **kw)
>   File "/home/cfinch/lib/python2.5/site-packages/scipy/weave/inline_tools.py", line 468, in compile_function
>     exec 'import ' + module_name
>   File "<string>", line 1, in <module>
> ImportError: /home/cfinch/.python25_compiled/sc_71b2502f9a0b0ca9f89b0cdc7ad3819e0.so: undefined symbol: _ZNSt8ios_base4InitD1Ev
>
> I used nm to check, and that symbol is indeed present in the compiled .so file.
>
> The configuration of this computer is:
> Python 2.5.4 (r254:67916, Apr 22 2009, 15:52:10) 
> [GCC 4.1.1 20070105 (Red Hat 4.1.1-52)] on linux2
> Scipy 0.7.0
> Numpy 1.3.0
> One
> complication is that the default compiler on this system is icc, and
> numpy distutils doesn't offer a way to force it to use gcc for
> everything.
>   

Mixing C++ compilers is difficult. Icc goes through great length to be
compatible with g++ on Linux, but it is highly version dependent. In
particular, the C++ library often causes problem, which is the case here
it seems. So you're right that you should use gcc to compile the extension.

One thing you could do is to have a script called icc which calls gcc, I
think this works if the script comes first in your path. That's an
horrible hack, though.

cheers,

David


From elias.pampalk at gmail.com  Fri Jul  3 12:06:58 2009
From: elias.pampalk at gmail.com (Elias Pampalk)
Date: Fri, 3 Jul 2009 17:06:58 +0100
Subject: [SciPy-user] scipy.stats.stats mannwhitneyu vs ranksums?
In-Reply-To: <4F46BA01-652B-4AEA-84CE-277D54F26693@cs.toronto.edu>
References: <43958ee60906301516s637b180fm35b96d5b61a1b549@mail.gmail.com>	<4A4A92A6.9070503@apstat.com>	<43958ee60906301624l7fd7f445sebd0d69b4ac32e61@mail.gmail.com>	<4a4b8c6f.1a135e0a.7a2d.0b82@mx.google.com>	<5b8d13220907010933j7463ec93tb0bb92f02fbf9308@mail.gmail.com>	<4a4cf1bf.05a4100a.7834.ffffcf2e@mx.google.com>
	<4F46BA01-652B-4AEA-84CE-277D54F26693@cs.toronto.edu>
Message-ID: <4a4e2ca7.0c92100a.2f71.0392@mx.google.com>

Thanks David! 

 

I did a quick comparison between Matlab/stats (R14SP3), R (2.8.1), and
Python/SciPy (0.7). Maybe this is somehow useful for others too.

 

(I'm intentionally violating the continuous distribution assumptions.)

 

Samples:

A1 <-> B: not paired with ties

A2 <-> B: not paired without ties

 

A1 <-> C: paired with zeros 

A2 <-> C: paired without zeros

 

- Matlab

      A1 = 0:19

      A2 = A1 + (1:20)./100

      B = 0:39

      C = [0:14,16:20]

 

- R

      A1 <- 0:19

      A2 <- A1 + 1:20/100

      B <- 0:39

      C <- c(0:14,16:20)

 

- SciPy

A1 = numpy.arange(20)

A2 = A1 + numpy.arange(1,21)/100.0

B = numpy.arange(40)

C = numpy.array(range(15) + range(16,21))

 

 

2 Samples, Not Paired

=====================

 

(from scipy.stats import stats)

 

Kruskal-Wallis Test 

-------------------

 

Same p-values for all.

 

Samples contain ties:

 

- Matlab: kruskalwallis([A1,B],[A1*0,B*0+1]) = 0.00170615101265

- R: kruskal.test(list(A1,B)) = 0.00170615101265

- R: wilcox.test(A1,B, correct=FALSE) = 0.00170615101265 (+warning: ties)

- SciPy: stats.kruskal(A1,B) = 0.00170615101265

 

(R: kruskal = wilcox without correction for continuity)

 

Samples without ties:

 

- Matlab: kruskalwallis([A2,B], [A2*0,B*0+1]) = 0.00288777919292

- R: kruskal.test(list(A2,B)) = 0.00288777919292

- SciPy: stats.kruskal(A2,B) = 0.00288777919292

      

 

Wilcoxon Rank Sum (aka Mann Whitney U) Test

-------------------------------------------

 

Matlab and R identical (but different defaults wrt exact/approximate), 

SciPy computes approximate results and does not correct for continuity
(changed in version 7.1 for stats.mannwhitneyu?).

 

Samples contain ties:

 

- Matlab: ranksum(A1,B) = 0.00175235702866

- R: wilcox.test(A1,B) = 0.00175235702866 (+warning: ties)

 

- R: wilcox.test(A1,B,correct=FALSE) = 0.001706151012654 (+warning: ties)

 

- SciPy: stats.mannwhitneyu(A1,B)[1]*2 = 0.0017086895586986284

 

- SciPy: stats.ranksums(A1,B) = 0.0017112312247389294

 

Samples without ties:

 

- Matlab: ranksum(A2,B) = 0.00296255173431

- R: wilcox.test(A2,B, exact=FALSE) = 0.00296255173431

 

- Matlab: ranksum(A2,B,'method','exact') = 0.00246078580826

- R: wilcox.test(A2,B) = 0.00246078580826

 

- R: wilcox.test(A2,B, exact=FALSE, correct=FALSE) = 0.00288777919292

- SciPy: stats.mannwhitneyu(A2,B)[1]*2 = 0.00288777919292

- SciPy: stats.ranksums(A2,B) = 0.00288777919292

 

(SciPy: mannwhitneyu = ranksums = kruskal if no ties)

 

 

2 Samples, Paired, Wilcoxon Sign Rank Test

==========================================

 

(from scipy.stats import wilcoxon)

 

Matlab and SciPy do not correct for continuity and R does.

Matlab and R have different defaults for exact/approximate.

Matlab computes exact results also if ties/zeros exist.

 

With zeros:

 

- Matlab: signrank(A1,C,'method','approximate') = 0.02534731867747

- R: wilcox.test(A1 - C, correct=FALSE) = 0.02534731867747 (+warnings: ties
+ zeros)

 

- Matlab: signrank(A1,C) = 0.06250000000000

 

- R: wilcox.test(A1 - C) = 0.0368884257070 (+warnings: ties + zeros)

 

- SciPy: wilcoxon(A1,C) = nan (+error: sample size too small)

      

Without zeros:

 

- Matlab: signrank(A2,C,'method','exact') = 0.59581947326660

- R: wilcox.test(A2 - C) = 0.59581947326660

 

- Matlab: signrank(A2,C) = 0.57548622813650     

- R: wilcox.test(A2 - C, exact=FALSE, correct=FALSE) = 0.57548622813650

- SciPy: wilcoxon(A2,C) = 0.57548622813650

 

- R: wilcox.test(A2 - C, exact=FALSE) = 0.5882844808893

 

Elias

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090703/6d26bbf6/attachment.html>

From tpk at kraussfamily.org  Fri Jul  3 13:35:51 2009
From: tpk at kraussfamily.org (Tom K.)
Date: Fri, 3 Jul 2009 10:35:51 -0700 (PDT)
Subject: [SciPy-user] Creating a Matrix from a Sum
In-Reply-To: <4A4CB556.6090203@gmail.com>
References: <4A4CB556.6090203@gmail.com>
Message-ID: <24326539.post@talk.nabble.com>



Lorenzo Isella wrote:
> 
> I need some help to efficiently write an array manipulation. I suspect 
> this could be a one-liner.
> Assume that you have a histogram of observations that you store in a 
> vector x.
> Let us say that its i-th entry, x_i, corresponds to the number of 
> observations in the i-th channel, for i=1,2...N. (or 0,1...N-1, please 
> let me know if there is any potential 0/1 pitfall in the following).
> Now, for any two channels i and j, I want to calculate the probability 
> of having an observation  in any other channel k, where  k>=max(i,j).
> That is to say
> 
> P(i,j)=sum_{k=max(i,j)}^N x_k /C,
> where C=sum_{i=1}^N x_i is just a normalization factor.
> 
> Does anyone know a good way of writing this in a code to get the P(i,j) 
> matrix?
> 

I'm not sure about a 1 liner.

Here are two ways that work:

1) Accumulate the array starting with the last element.
P = np.zeros((len(x), len(x)))
i = len(x)
for xi in x[::-1]:
    P[:i, :i] += xi
    i -= 1

2) Index into the reversed cumulative sum with a max_ij array.
i = np.arange(len(x))[:,np.newaxis]
j = np.arange(len(x))
max_ij = np.maximum(i,j)
P = x[::-1].cumsum()[::-1][max_ij]



-- 
View this message in context: http://www.nabble.com/Creating-a-Matrix-from-a-Sum-tp24306698p24326539.html
Sent from the Scipy-User mailing list archive at Nabble.com.



From oanjao at yahoo.com  Fri Jul  3 14:38:06 2009
From: oanjao at yahoo.com (Craig Finch)
Date: Fri, 3 Jul 2009 11:38:06 -0700 (PDT)
Subject: [SciPy-user] Scipy weave errors: undefined symbols when
	importing compiled extension
In-Reply-To: <4A4DB108.8050407@ar.media.kyoto-u.ac.jp>
References: <524620.24680.qm@web30402.mail.mud.yahoo.com>
	<4A4DB108.8050407@ar.media.kyoto-u.ac.jp>
Message-ID: <829316.46979.qm@web30408.mail.mud.yahoo.com>


Thank you for the suggestion.  I have some new information, but unfortunately no solution.  I re-built Numpy and Scipy with gcc and gfortran, and did some hacking
to ensure that my Weave extension was also being built with gcc.  I get the same error when I try to import the Weave extension.  I do have some new information that might help--I found that I have other library problems in my Scipy build, rather than a Weave-specific issue.  Here is another manifestation of the problem:


>>> from scipy.interpolate import UnivariateSpline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cfinch/lib/python2.5/site-packages/scipy/interpolate/__init__.py",
 line 13, in <module>
    from rbf import Rbf
  File "/home/cfinch/lib/python2.5/site-packages/scipy/interpolate/rbf.py", line 47, in <module>
    from scipy import linalg
  File "/home/cfinch/lib/python2.5/site-packages/scipy/linalg/__init__.py", line 13, in <module>
    from iterative import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/linalg/iterative.py", line 5, in <module>
    from scipy.sparse.linalg import isolve
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/__init__.py", line 6, in <module>
    from csr import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/csr.py", line 12, in <module>
    from sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/__init__.py", line 4, in <module>
    from csr import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/csr.py", line 7, in <module>
    import _csr
ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: __gxx_personality_v0

As far as I know, everything Python-related on this machine has now been built with GNU tools, but something is still broken.  One more thing--Python, Numpy, and Scipy are all locally installed in my /home/cfinch directory because the system Python is frozen at 2.4.  Any more ideas?

  Craig



----- Original Message ----
From: David Cournapeau <david at ar.media.kyoto-u.ac.jp>
To: Craig Finch <cfinch at ieee.org>; SciPy Users List <scipy-user at scipy.org>
Sent: Friday, July 3, 2009 3:19:36 AM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing compiled extension

Craig Finch wrote:
> I have two computers that have almost identical Python installations,
> but one of them has problems with inline Weave code.  Here is a simple
> test script that I am using:
>
> #!/usr/bin/env python
> from scipy import weave
> input_val = 10
>
> code=r"""return_val = 10*input_val;"""
>
> print weave.inline(code, ['input_val'], headers=[], compiler='gcc')
>
> It runs successfully on one computer with the following configuration:
> Python 2.5.2 (r252:60911, Apr 17 2009, 18:42:17) 
> [GCC 4.1.1 (Gentoo 4.1.1-r3)] on linux2
> Scipy 0.7.0
> Numpy 1.3.0
>
> On the other computer, it fails with the following message:
> Traceback (most recent call last):
>   File "test.py", line 12, in <module>
>     print weave.inline(code, ['input_val'], headers=[], compiler='gcc')
>   File "/home/cfinch/lib/python2.5/site-packages/scipy/weave/inline_tools.py", line 335, in inline
>     **kw)
>   File "/home/cfinch/lib/python2.5/site-packages/scipy/weave/inline_tools.py", line 468, in compile_function
>     exec 'import ' + module_name
>   File "<string>", line 1, in <module>
> ImportError: /home/cfinch/.python25_compiled/sc_71b2502f9a0b0ca9f89b0cdc7ad3819e0.so: undefined symbol: _ZNSt8ios_base4InitD1Ev
>
> I used nm to check, and that symbol is indeed present in the compiled .so file.
>
> The configuration of this computer is:
> Python 2.5.4 (r254:67916, Apr 22 2009, 15:52:10) 
> [GCC 4.1.1 20070105 (Red Hat 4.1.1-52)] on linux2
> Scipy 0.7.0
> Numpy 1.3.0
> One
> complication is that the default compiler on this system is icc, and
> numpy distutils doesn't offer a way to force it to use gcc for
> everything.
>  

Mixing C++ compilers is difficult. Icc goes through great length to be
compatible with g++ on Linux, but it is highly version dependent. In
particular, the C++ library often causes problem, which is the case here
it seems. So you're right that you should use gcc to compile the extension.

One thing you could do is to have a script called icc which calls gcc, I
think this works if the script comes first in your path. That's an
horrible hack, though.

cheers,

David



      


From dwf at cs.toronto.edu  Fri Jul  3 14:45:23 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Fri, 3 Jul 2009 14:45:23 -0400
Subject: [SciPy-user] Creating a Matrix from a Sum
In-Reply-To: <4A4CB556.6090203@gmail.com>
References: <4A4CB556.6090203@gmail.com>
Message-ID: <E7C27F49-BB78-4727-B874-92796AD1A765@cs.toronto.edu>

On 2-Jul-09, at 9:25 AM, Lorenzo Isella wrote:

> Dear All,
> I need some help to efficiently write an array manipulation. I suspect
> this could be a one-liner.
> Assume that you have a histogram of observations that you store in a
> vector x.
> Let us say that its i-th entry, x_i, corresponds to the number of
> observations in the i-th channel, for i=1,2...N. (or 0,1...N-1, please
> let me know if there is any potential 0/1 pitfall in the following).
> Now, for any two channels i and j, I want to calculate the probability
> of having an observation  in any other channel k, where  k>=max(i,j).
> That is to say
>
> P(i,j)=sum_{k=max(i,j)}^N x_k /C,
> where C=sum_{i=1}^N x_i is just a normalization factor.

Hm, there might be a simpler/more efficient way but this is what I  
came up with as far as 1-liners:

P = np.max(np.concatenate(np.broadcast_arrays(x[np.newaxis, :,  
np.newaxis], x[:, np.newaxis, np.newaxis]),axis=2),axis=2); P /= x.sum()





From dwf at cs.toronto.edu  Fri Jul  3 15:01:44 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Fri, 3 Jul 2009 15:01:44 -0400
Subject: [SciPy-user] Creating a Matrix from a Sum
In-Reply-To: <E7C27F49-BB78-4727-B874-92796AD1A765@cs.toronto.edu>
References: <4A4CB556.6090203@gmail.com>
	<E7C27F49-BB78-4727-B874-92796AD1A765@cs.toronto.edu>
Message-ID: <7F843EB2-C294-4F1C-A1B6-19C1CF7BEDFA@cs.toronto.edu>


On 3-Jul-09, at 2:45 PM, David Warde-Farley wrote:

> On 2-Jul-09, at 9:25 AM, Lorenzo Isella wrote:
>
>> Dear All,
>> I need some help to efficiently write an array manipulation. I  
>> suspect
>> this could be a one-liner.
>> Assume that you have a histogram of observations that you store in a
>> vector x.
>> Let us say that its i-th entry, x_i, corresponds to the number of
>> observations in the i-th channel, for i=1,2...N. (or 0,1...N-1,  
>> please
>> let me know if there is any potential 0/1 pitfall in the following).
>> Now, for any two channels i and j, I want to calculate the  
>> probability
>> of having an observation  in any other channel k, where  k>=max(i,j).
>> That is to say
>>
>> P(i,j)=sum_{k=max(i,j)}^N x_k /C,
>> where C=sum_{i=1}^N x_i is just a normalization factor.
>
> Hm, there might be a simpler/more efficient way but this is what I
> came up with as far as 1-liners:
>
> P = np.max(np.concatenate(np.broadcast_arrays(x[np.newaxis, :,
> np.newaxis], x[:, np.newaxis, np.newaxis]),axis=2),axis=2); P /=  
> x.sum()

Oops, nevermind, that's wrong.

Here's how I would do it.

You want the sum from k to the end (N) so I'd extract a vector like  
this (the cumulative sum of the reversed array, reversed again

	sums = np.cumsum(x[::-1])[::-1]

Then you want the max of i and j at each position in the matrix. For  
this, though there may be a more efficient way, you can use mgrid to  
get i and j indices at each position in the matrix and then use  
concatenate and max on them to concatenate along a third axis and then  
do a max along that same axis to recover a 2D array:

	idx = np.concatenate([M[:,:,np.newaxis] for M in mgrid[0:len(x),  
0:len(x)]],axis=2).max(axis=2)

Finally, use fancy indexing and divide by your normalizer:

	P = sums[idx]  / x.sum()

Or, as a oneliner, much less readable:

	P = cumsum(x[::-1])[::-1][np.concatenate([M[:,:,np.newaxis] for M in  
mgrid[0:len(x), 0:len(x)]],axis=2).max(axis=2)] / x.sum()

HTH,

David


From tpk at kraussfamily.org  Fri Jul  3 21:18:40 2009
From: tpk at kraussfamily.org (Tom K.)
Date: Fri, 3 Jul 2009 18:18:40 -0700 (PDT)
Subject: [SciPy-user] Creating a Matrix from a Sum
In-Reply-To: <7F843EB2-C294-4F1C-A1B6-19C1CF7BEDFA@cs.toronto.edu>
References: <4A4CB556.6090203@gmail.com>
	<E7C27F49-BB78-4727-B874-92796AD1A765@cs.toronto.edu>
	<7F843EB2-C294-4F1C-A1B6-19C1CF7BEDFA@cs.toronto.edu>
Message-ID: <24330442.post@talk.nabble.com>




David Warde-Farley-2 wrote:
> 
> 
> On 3-Jul-09, at 2:45 PM, David Warde-Farley wrote:
> 
>> On 2-Jul-09, at 9:25 AM, Lorenzo Isella wrote:
> 
> Then you want the max of i and j at each position in the matrix. For  
> this, though there may be a more efficient way, you can use mgrid to  
> get i and j indices at each position in the matrix and then use  
> concatenate and max on them to concatenate along a third axis and then  
> do a max along that same axis to recover a 2D array:
> 
> 	idx = np.concatenate([M[:,:,np.newaxis] for M in mgrid[0:len(x),  
> 0:len(x)]],axis=2).max(axis=2)
> 

David this is interesting however I think the use of mgrid followed by
concatenation into a 3D array here is overkill.  The idx array is the same
as the max_ij array that I suggested - a one liner for it is 
  max_ij = np.maximum(np.arange(len(x))[:,np.newaxis], np.arange(len(x)))

This problem involves O(N) sums and O(N^2) assignments.   The vectorized
versions that David and I posted (my version 2) are doing just that much
work - however with overhead of several intermediate arrays.  A more
straight-forward approach in python is just a double for-loop, which may be
the most readable and perfectly sufficient unless your N is huge and/or this
function becomes the most significant item in a profile of your application. 

sums = np.cumsum(x[::-1])[::-1]
P = np.zeros((len(x), len(x))
for i in range(len(x)):
  for j in  range(len(x)):
    P[i, j] = sums[np.max((i, j))]

#Here it is with list comprehensions:
  P = np.array([[sums[np.max((i, j))] for j in range(len(x))] for i in
range(len(x))])

By the way I do NOT recommend using the first method I posted in my earlier
post - I think it is doing O(N^3) adds and O(N^3) assigns!
-- 
View this message in context: http://www.nabble.com/Creating-a-Matrix-from-a-Sum-tp24306698p24330442.html
Sent from the Scipy-User mailing list archive at Nabble.com.



From cournape at gmail.com  Fri Jul  3 22:21:28 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 4 Jul 2009 11:21:28 +0900
Subject: [SciPy-user] Scipy weave errors: undefined symbols when
	importing compiled extension
In-Reply-To: <829316.46979.qm@web30408.mail.mud.yahoo.com>
References: <524620.24680.qm@web30402.mail.mud.yahoo.com>
	<4A4DB108.8050407@ar.media.kyoto-u.ac.jp>
	<829316.46979.qm@web30408.mail.mud.yahoo.com>
Message-ID: <5b8d13220907031921v24626b17hc353705f766435d6@mail.gmail.com>

On Sat, Jul 4, 2009 at 3:38 AM, Craig Finch<oanjao at yahoo.com> wrote:

> ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: __gxx_personality_v0
>
> As far as I know, everything Python-related on this machine has now been built with GNU tools, but something is still broken. ?One more thing--Python, Numpy, and Scipy are all locally installed in my /home/cfinch directory because the system Python is frozen at 2.4.

This is still a C++ problem, I have sometimes noticed this problem on
some machines. Normally this is caused by a C++ object file being
linked with gcc (instead of g++). Can you post the build log of scipy
?

David


From dwf at cs.toronto.edu  Sat Jul  4 06:56:27 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sat, 4 Jul 2009 06:56:27 -0400
Subject: [SciPy-user] Creating a Matrix from a Sum
In-Reply-To: <24330442.post@talk.nabble.com>
References: <4A4CB556.6090203@gmail.com>
	<E7C27F49-BB78-4727-B874-92796AD1A765@cs.toronto.edu>
	<7F843EB2-C294-4F1C-A1B6-19C1CF7BEDFA@cs.toronto.edu>
	<24330442.post@talk.nabble.com>
Message-ID: <9483AA21-7F65-4D53-ABD2-3FD31339C9D7@cs.toronto.edu>

On 3-Jul-09, at 9:18 PM, Tom K. wrote:

> A more
> straight-forward approach in python is just a double for-loop, which  
> may be
> the most readable and perfectly sufficient unless your N is huge and/ 
> or this
> function becomes the most significant item in a profile of your  
> application.
>
> sums = np.cumsum(x[::-1])[::-1]
> P = np.zeros((len(x), len(x))
> for i in range(len(x)):
>  for j in  range(len(x)):
>    P[i, j] = sums[np.max((i, j))]
>
> #Here it is with list comprehensions:
>  P = np.array([[sums[np.max((i, j))] for j in range(len(x))] for i in
> range(len(x))])


Before I begin, a quick tip: replace range() with xrange() to do the  
same with less memory (range() causes a list allocation, xrange()  
provides a constant-size generator).

This is an instance where the intuition gleaned from asymptotic  
notation fails to really address one critical piece of the puzzle:  
both the implementation above and the implementation I posted are  
doing roughly the same amount of work (mine is doing a little more in  
the form of copies and allocations), but one is doing it in C whereas  
the other is doing it in interpreted Python. It also depends on your  
definition of a "huge" problem. Speed wise, Python-vs-C starts to  
matter very, very quickly, i.e. on the order of hundreds of elements,  
and by 1000 things get really horrible:

In [144]: def python_impl(x):
     P = np.zeros((len(x), len(x)))
     sums = np.cumsum(x[::-1])[::-1]    for i in range(len(x)):
         for j in range(len(x)):
             P[i, j] = sums[np.max((i,j))]
    .....:

In [150]: def numpy_impl(x):
     sums = np.cumsum(x[::-1])[::-1]
     idx = np.concatenate([M[:,:,np.newaxis] for M in  
np.mgrid[0:len(x), 0:len(x)]],axis=2).max(axis=2)
     P = sums[idx] / x.sum()
    .....:

In [154]: x = randn(1000)

In [155]: timeit numpy_impl(x)
1 loops, best of 3: 200 ms per loop

In [156]: timeit python_impl(x)
1 loops, best of 3: 37.7 s per loop

In [157]: x = randn(5000)

In [158]: timeit numpy_impl(x)
1 loops, best of 3: 5.41 s per loop

I did start running python_impl with len(x) = 5000 but my back of the  
envelope calculation is that it will be taking at least 1 hour per  
try, so timeit will be done in about 3 hours. :(

The overhead of the Python nested loops may be a showstopper even for  
moderately sized problems. In a lot of these situations I find I run  
out of patience far before I run out of memory to throw around. :)

David


From stefan at sun.ac.za  Sat Jul  4 08:23:01 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Sat, 4 Jul 2009 14:23:01 +0200
Subject: [SciPy-user] assign to diagonal values?
In-Reply-To: <db6b5ecc0907011238r5e611f51y2c01052bbc0ae83b@mail.gmail.com>
References: <47A128A5.7010406@sci.utah.edu>
	<Mahogany-0.67.0-1804-20080130-220801.00@american.edu> 
	<DD23E9F4-9327-4371-8767-6DFB77FADAD7@cs.toronto.edu>
	<2bc7a5a50801302128g79da7ea6jc95ef9abb00b3785@mail.gmail.com> 
	<db6b5ecc0907011238r5e611f51y2c01052bbc0ae83b@mail.gmail.com>
Message-ID: <9457e7c80907040523g734aa3c0nab686af1476a731d@mail.gmail.com>

2009/7/1 Fernando Perez <fperez.net at gmail.com>:
> If someone feels like reviewing this ticket:
>
> http://projects.scipy.org/numpy/attachment/ticket/1132/numpy-index-funcs.diff
>
> it has the above and a few more utilities, with docs and tests.

Thanks! Reviewed and applied in r7101 through r7103.

Cheers
St?fan


From tpk at kraussfamily.org  Sat Jul  4 09:31:18 2009
From: tpk at kraussfamily.org (Tom K.)
Date: Sat, 4 Jul 2009 06:31:18 -0700 (PDT)
Subject: [SciPy-user]  Access to IPython's timeit result
Message-ID: <24334619.post@talk.nabble.com>


Hi, 
 Looking at timeit in IPython, you get a nice output e.g.

In [14]: timeit x=range(100000)
100 loops, best of 3: 5.75 ms per loop

but what if I want to get access to that 5.75 ms number so I can write a
program to capture these over a list of "N" values and make a plot of time
versus "N"?

 - Tom K.
-- 
View this message in context: http://www.nabble.com/Access-to-IPython%27s-timeit-result-tp24334619p24334619.html
Sent from the Scipy-User mailing list archive at Nabble.com.



From zunzun at zunzun.com  Sat Jul  4 12:22:23 2009
From: zunzun at zunzun.com (James Phillips)
Date: Sat, 4 Jul 2009 11:22:23 -0500
Subject: [SciPy-user] Additional Kern-ification of zunzun.com
Message-ID: <268756d30907040922k76cfdde1gc73cfc7cdbab4fcd@mail.gmail.com>

ODR is now one of the available fitting targets at http://zunzun.com, and
again my thanks to Robert Kern.

    James Phillips
    http://zunzun.com

On Fri, Jun 03, 2005 at 02:40:09AM -0700, Robert Kern wrote:
>
> I always use the full-blown nonlinear implementation,
> which is not yet integrated into scipy.
>
> http://starship.python.net/crew/kernr/source/odr-0.6.1.tar.gz

Say, that is very interesting work.  May I add it to my
web site as a fitting method?  I'm also curious as to when
it might make it into the SciPy distribution.

    James Phillips
    http://zunzun.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090704/cc49854e/attachment.html>

From jkington at wisc.edu  Sat Jul  4 14:44:34 2009
From: jkington at wisc.edu (Joe Kington)
Date: Sat, 4 Jul 2009 13:44:34 -0500
Subject: [SciPy-user] Access to IPython's timeit result
In-Reply-To: <f6951adf0907041002x43d50d89mc9cb5ceb75bcb2a7@mail.gmail.com>
References: <24334619.post@talk.nabble.com>
	<f6951adf0907041002x43d50d89mc9cb5ceb75bcb2a7@mail.gmail.com>
Message-ID: <f6951adf0907041144g2039125cv22cca3bff18029fa@mail.gmail.com>

Hi Tom,

Have you looked at the timeit module?

http://docs.python.org/library/timeit.html

At least in 2.6, timeit.timeit() is builtin.  In older versions, you just
need to build a timeit.Time class instance and call it's timeit() function,
which returns an average over a number of executions (default=1000000).

If you really want the "best of 3" like ipython gives you, you can use the
repeat function of the timeit.Time module to return a list of the execution
times for each run.

Hope that helps,
-Joe

On Sat, Jul 4, 2009 at 12:02 PM, Joe Kington <jkington at geology.wisc.edu>wrote:

> Hi Tom,
>
> Have you looked at the timeit module?
>
> http://docs.python.org/library/timeit.html
>
> At least in 2.6, timeit.timeit() is builtin.  In older versions, you just
> need to build a timeit.Time class instance and call it's timeit() function,
> which returns an average over a number of executions (default=1000000).
>
> If you really want the "best of 3" like ipython gives you, you can use the
> repeat function of the timeit.Time module to return a list of the execution
> times for each run.
>
> Hope that helps,
> -Joe
>
> On Sat, Jul 4, 2009 at 8:31 AM, Tom K. <tpk at kraussfamily.org> wrote:
>
>>
>> Hi,
>>  Looking at timeit in IPython, you get a nice output e.g.
>>
>> In [14]: timeit x=range(100000)
>> 100 loops, best of 3: 5.75 ms per loop
>>
>> but what if I want to get access to that 5.75 ms number so I can write a
>> program to capture these over a list of "N" values and make a plot of time
>> versus "N"?
>>
>>  - Tom K.
>> --
>> View this message in context:
>> http://www.nabble.com/Access-to-IPython%27s-timeit-result-tp24334619p24334619.html
>> Sent from the Scipy-User mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090704/1add1c77/attachment.html>

From dwf at cs.toronto.edu  Sat Jul  4 16:25:03 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sat, 4 Jul 2009 16:25:03 -0400
Subject: [SciPy-user] Creating a Matrix from a Sum
In-Reply-To: <9483AA21-7F65-4D53-ABD2-3FD31339C9D7@cs.toronto.edu>
References: <4A4CB556.6090203@gmail.com>
	<E7C27F49-BB78-4727-B874-92796AD1A765@cs.toronto.edu>
	<7F843EB2-C294-4F1C-A1B6-19C1CF7BEDFA@cs.toronto.edu>
	<24330442.post@talk.nabble.com>
	<9483AA21-7F65-4D53-ABD2-3FD31339C9D7@cs.toronto.edu>
Message-ID: <F79780C8-326B-4824-9569-AD32FA9506A0@cs.toronto.edu>

> In [157]: x = randn(5000)
>
> In [158]: timeit numpy_impl(x)
> 1 loops, best of 3: 5.41 s per loop
>
> I did start running python_impl with len(x) = 5000 but my back of the
> envelope calculation is that it will be taking at least 1 hour per
> try, so timeit will be done in about 3 hours. :(


So, things didn't scale quite as badly as I thought they would (I was  
running on very little sleep, heh)

In [157]: x = randn(5000)

In [158]: timeit numpy_impl(x)
1 loops, best of 3: 5.41 s per loop

In [159]: timeit python_impl(x)
1 loops, best of 3: 1.01e+12 ns per loop

Why IPython gave me an answer in nanoseconds I'm not sure...  But  
that's about 17 minutes.

In either case,  then, you're looking at roughly a 200x speed up of  
the numpy version over the python loops, which is quite significant.  
If you can spare the  ~2N^2 intermediate memory usage, it's definitely  
worth it.

If you can't, but you still can't tolerate the speed hit, you can run  
the Python implementation Tom gave through Cython and add type  
specifiers for i, j, P and x, see http://docs.cython.org/docs/numpy_tutorial.html 
  - that should achieve similar speedups without the memory waste.

David


From oanjao at yahoo.com  Sun Jul  5 14:30:18 2009
From: oanjao at yahoo.com (Craig Finch)
Date: Sun, 5 Jul 2009 11:30:18 -0700 (PDT)
Subject: [SciPy-user] Scipy weave errors: undefined symbols when
	importing compiled extension
In-Reply-To: <5b8d13220907031921v24626b17hc353705f766435d6@mail.gmail.com>
References: <524620.24680.qm@web30402.mail.mud.yahoo.com>
	<4A4DB108.8050407@ar.media.kyoto-u.ac.jp>
	<829316.46979.qm@web30408.mail.mud.yahoo.com>
	<5b8d13220907031921v24626b17hc353705f766435d6@mail.gmail.com>
Message-ID: <783243.63027.qm@web30408.mail.mud.yahoo.com>


By "build log" I assume you mean the output to stderr and stdout when when building Scipy.  It's a huge amount of output, and I'm not sure exactly what to look for.  Do you want me to post a particular section?  Below is the output from sections where g++ is called.  It seems to be linked with g++ everywhere.

One more potential problem area: I'm linking with the Intel Math Kernel Library (MKL) which was almost certainly built with Intel compilers.  Could that be causing this problem?

   Craig
  
-----------------------
building 'scipy.interpolate._interpolate' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-Iscipy/interpolate/src -I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/interpolate/src/_interpolate.cpp
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/interpolate/src/_interpolate.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/interpolate/_interpolate.so
-----------------------
building 'scipy.sparse.sparsetools._csr' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

creating build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools
compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/csr_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/csr_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_csr.so
building 'scipy.sparse.sparsetools._csc' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/csc_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/csc_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_csc.so
building 'scipy.sparse.sparsetools._coo' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/coo_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/coo_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_coo.so
building 'scipy.sparse.sparsetools._bsr' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/bsr_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/bsr_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_bsr.so
building 'scipy.sparse.sparsetools._dia' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/dia_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/dia_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_dia.so

-----------------------




----- Original Message ----
From: David Cournapeau <cournape at gmail.com>
To: Craig Finch <cfinch at ieee.org>; SciPy Users List <scipy-user at scipy.org>
Sent: Friday, July 3, 2009 10:21:28 PM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing  compiled extension

On Sat, Jul 4, 2009 at 3:38 AM, Craig Finch<oanjao at yahoo.com> wrote:

> ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: __gxx_personality_v0
>
> As far as I know, everything Python-related on this machine has now been built with GNU tools, but something is still broken.  One more thing--Python, Numpy, and Scipy are all locally installed in my /home/cfinch directory because the system Python is frozen at 2.4.

This is still a C++ problem, I have sometimes noticed this problem on
some machines. Normally this is caused by a C++ object file being
linked with gcc (instead of g++). Can you post the build log of scipy
?

David



      


From d_l_goldsmith at yahoo.com  Sun Jul  5 23:19:45 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 5 Jul 2009 20:19:45 -0700 (PDT)
Subject: [SciPy-user] The baffling behavior that just won't die
Message-ID: <753940.28516.qm@web52101.mail.re2.yahoo.com>


Hi, folks.  I'm having a problem using numpy.lookfor() that is very reminiscent of this thread:

http://mail.scipy.org/pipermail/scipy-user/2008-June/017294.html

The simplest reproduction of the problem is illustrated thusly:

c:\Python25\Lib\site-packages\numpy>dir random.py
 Volume in drive C is SQ004816V03
 Volume Serial Number is 8E67-90B7

 Directory of c:\Python25\Lib\site-packages\numpy

File Not Found

c:\Python25\Lib\site-packages\numpy>python
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> 'lookfor' in dir(np)
True
>>> help(np.lookfor)
cannot import name Random

And that's all she wrote.  

I read the thread cited above and it led me to examine the goings-on using python -v; as warned, there was a waterfall of imports, but at the end of it all was:
                 .
                 :
# C:\Python25\lib\repr.pyc matches C:\Python25\lib\repr.py
import repr # precompiled from C:\Python25\lib\repr.pyc
import collections # builtin
# C:\Python25\lib\tempfile.pyc matches C:\Python25\lib\tempfile.py
import tempfile # precompiled from C:\Python25\lib\tempfile.pyc
import random # directory random
# random\__init__.pyc matches random\__init__.py
import random # precompiled from random\__init__.pyc
# random\info.pyc matches random\info.py
import random.info # precompiled from random\info.pyc
import random.mtrand # dynamically loaded from random\mtrand.pyd
cannot import name Random

Please help.  Thanks!

DG


      


From lorenzo.isella at gmail.com  Mon Jul  6 04:18:07 2009
From: lorenzo.isella at gmail.com (Lorenzo Isella)
Date: Mon, 06 Jul 2009 10:18:07 +0200
Subject: [SciPy-user] Finding Gaps in an Array
Message-ID: <4A51B33F.7060306@gmail.com>

Dear All,
Suppose you have an integer array whose entries are like the one below 
(no entry is ever repeated and the array is sorted in increasing order)

1240070020
1240070040
1240070060
1240070080
1240070100
1240070180
1240070200
1240070620
1240070640
1240070880
1240070900
1240070940
1240070980
1240071000
1240071020
1240071040
1240071060
1240071080
1240071100
1240071120
1240071140
1240071160
1240071180
1240071200
1240071220
1240071240
1240071260
1240071280
1240071300

Let us assume that the entries represent contact times (in seconds) 
between 2 objects (one does not need to know anything else for this 
post). The value of the initial time does not matter (i.e. you could 
subtract it to all the elements in the array to have a list starting 
from zero). Now, you define the contact duration as the time spanned by 
consecutive entries evenly spaced by 20 seconds.
E.g. in the array above, the first 5 elements define a contact of 
duration 1240070100-1240070020=80. Then there is an 80-sec gap 
(1240070180-1240070100) followed by a contact of duration 20 
(1240070200-1240070180).
How can you automatically calculate these intervals?
Any suggestion is welcome.
Cheers

Lorenzo


From david_baddeley at yahoo.com.au  Mon Jul  6 08:01:14 2009
From: david_baddeley at yahoo.com.au (David Baddeley)
Date: Mon, 6 Jul 2009 05:01:14 -0700 (PDT)
Subject: [SciPy-user] Finding Gaps in an Array
In-Reply-To: <4A51B33F.7060306@gmail.com>
References: <4A51B33F.7060306@gmail.com>
Message-ID: <672254.30337.qm@web33004.mail.mud.yahoo.com>


Hi Lorenzo,

numpy.diff(your_array)

will give you the times between entries,

numpy.where(numpy.diff(your_array) > 20)

will give the indices where the gap was longer than 20

cheers,
David



----- Original Message ----
From: Lorenzo Isella <lorenzo.isella at gmail.com>
To: scipy-user at scipy.org
Sent: Monday, 6 July, 2009 8:18:07 PM
Subject: [SciPy-user] Finding Gaps in an Array

Dear All,
Suppose you have an integer array whose entries are like the one below 
(no entry is ever repeated and the array is sorted in increasing order)

1240070020
1240070040
1240070060
1240070080
1240070100
1240070180
1240070200
1240070620
1240070640
1240070880
1240070900
1240070940
1240070980
1240071000
1240071020
1240071040
1240071060
1240071080
1240071100
1240071120
1240071140
1240071160
1240071180
1240071200
1240071220
1240071240
1240071260
1240071280
1240071300

Let us assume that the entries represent contact times (in seconds) 
between 2 objects (one does not need to know anything else for this 
post). The value of the initial time does not matter (i.e. you could 
subtract it to all the elements in the array to have a list starting 
from zero). Now, you define the contact duration as the time spanned by 
consecutive entries evenly spaced by 20 seconds.
E.g. in the array above, the first 5 elements define a contact of 
duration 1240070100-1240070020=80. Then there is an 80-sec gap 
(1240070180-1240070100) followed by a contact of duration 20 
(1240070200-1240070180).
How can you automatically calculate these intervals?
Any suggestion is welcome.
Cheers

Lorenzo
_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user



      


From adrian.prw at gmail.com  Mon Jul  6 10:18:31 2009
From: adrian.prw at gmail.com (Adrian Price-Whelan)
Date: Mon, 6 Jul 2009 10:18:31 -0400
Subject: [SciPy-user] Misunderstanding scipy.interpolate.interp2d?
In-Reply-To: <mailman.0.1246889566.29528.scipy-user@scipy.org>
References: <mailman.0.1246889566.29528.scipy-user@scipy.org>
Message-ID: <A679FCA3-EB48-462A-A8DA-788E011B1C7D@gmail.com>

Hey all --

Perhaps I am misunderstanding the documentation for the function  
scipy.interpolate.interp2d, or maybe I just don't understand the math  
of how it works. If it is a true 2-D interpolating function, why would  
it take 2 1-dimensional arrays as input? For instance, say I had image  
data in a 2-D array that I wanted to shift by some sub-pixel amount,  
like 0.3 pixels, to some arbitrary direction. How would I take an  
array ( something like [[1,2,3],[4,5,6],[7,8,9]] ) and break this into  
2 1-D arrays x and y to use this function to interpolate new values  
based on the sub-pixel shift?

Thanks!!
-Adrian


From lorenzo.isella at gmail.com  Mon Jul  6 13:22:07 2009
From: lorenzo.isella at gmail.com (Lorenzo Isella)
Date: Mon, 06 Jul 2009 19:22:07 +0200
Subject: [SciPy-user] Finding Gaps in an Array
In-Reply-To: <mailman.5.1246899602.3411.scipy-user@scipy.org>
References: <mailman.5.1246899602.3411.scipy-user@scipy.org>
Message-ID: <4A5232BF.5080802@gmail.com>


> Date: Mon, 06 Jul 2009 10:18:07 +0200
> From: Lorenzo Isella <lorenzo.isella at gmail.com>
> Subject: [SciPy-user] Finding Gaps in an Array
> To: scipy-user at scipy.org
> Message-ID: <4A51B33F.7060306 at gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Dear All,
> Suppose you have an integer array whose entries are like the one below 
> (no entry is ever repeated and the array is sorted in increasing order)
>
> 1240070020
> 1240070040
> 1240070060
> 1240070080
> 1240070100
> 1240070180
> 1240070200
> 1240070620
> 1240070640
> 1240070880
> 1240070900
> 1240070940
> 1240070980
> 1240071000
> 1240071020
> 1240071040
> 1240071060
> 1240071080
> 1240071100
> 1240071120
> 1240071140
> 1240071160
> 1240071180
> 1240071200
> 1240071220
> 1240071240
> 1240071260
> 1240071280
> 1240071300
>
> Let us assume that the entries represent contact times (in seconds) 
> between 2 objects (one does not need to know anything else for this 
> post). The value of the initial time does not matter (i.e. you could 
> subtract it to all the elements in the array to have a list starting 
> from zero). Now, you define the contact duration as the time spanned by 
> consecutive entries evenly spaced by 20 seconds.
> E.g. in the array above, the first 5 elements define a contact of 
> duration 1240070100-1240070020=80. Then there is an 80-sec gap 
> (1240070180-1240070100) followed by a contact of duration 20 
> (1240070200-1240070180).
> How can you automatically calculate these intervals?
> Any suggestion is welcome.
> Cheers
>
> Lorenzo
>
>
>
>
> Date: Mon, 6 Jul 2009 05:01:14 -0700 (PDT)
> From: David Baddeley <david_baddeley at yahoo.com.au>
> Subject: Re: [SciPy-user] Finding Gaps in an Array
> To: SciPy Users List <scipy-user at scipy.org>
> Message-ID: <672254.30337.qm at web33004.mail.mud.yahoo.com>
> Content-Type: text/plain; charset=utf-8
>
>
> Hi Lorenzo,
>
> numpy.diff(your_array)
>
> will give you the times between entries,
>
> numpy.where(numpy.diff(your_array) > 20)
>
> will give the indices where the gap was longer than 20
>
> cheers,
> David
>
>
>
>   


Thanks David,
But I also need something more.
Let us make it real easy.  Consider the array

import numpy as n
f=n.array([0,1,2,4,9,22,23,24,32,33,59,60,76])

I want to calculate the length of the subarrays consisting of numbers 
evenly spaced by 1 (like in the example, but now the gap is 1).
In the case of f, these subarrays are:

[0,1,2], [22,23,24], [32,33] and [59,60].

Hence, I would like to get back an array giving me the counts
[3,3,2,2].

Does anyone know how to do this efficiently?
Cheers

Lorenzo



From pgmdevlist at gmail.com  Mon Jul  6 13:45:33 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Mon, 6 Jul 2009 13:45:33 -0400
Subject: [SciPy-user] Finding Gaps in an Array
In-Reply-To: <4A5232BF.5080802@gmail.com>
References: <mailman.5.1246899602.3411.scipy-user@scipy.org>
	<4A5232BF.5080802@gmail.com>
Message-ID: <687BC23D-6850-44AC-B008-1C404498E1E5@gmail.com>


On Jul 6, 2009, at 1:22 PM, Lorenzo Isella wrote:
>
> Thanks David,
> But I also need something more.
> Let us make it real easy.  Consider the array
>
> import numpy as n
> f=n.array([0,1,2,4,9,22,23,24,32,33,59,60,76])
>
> I want to calculate the length of the subarrays consisting of numbers
> evenly spaced by 1 (like in the example, but now the gap is 1).
> In the case of f, these subarrays are:
>
> [0,1,2], [22,23,24], [32,33] and [59,60].
>
> Hence, I would like to get back an array giving me the counts
> [3,3,2,2].
>
> Does anyone know how to do this efficiently?
> Cheers
>
> Lorenzo

Lorenzo,
I have implemented an object that should help you in  
scikits.hydroclimpy:
http://hydroclimpy.sourceforge.net/
More specifically:
http://hydroclimpy.sourceforge.net/core.objects.html#cluster-object

Roughly, given an array and an increment parameter, Cluster computes a  
sequence of slices that differ by less than increment (or more than  
increment, depending on yet another parameter).
Here's the relevant portion of the code.
Note : in an upcoming version, this code will be implemented in  
Cython, and renamed to Clumps (as the name Cluster is misleading).


##########
class Cluster(object):
     """
     Groups consecutive data from an array according to a clustering  
condition.

     A cluster is defined as a group of consecutive values differing  
by at most the
     increment value.

     Missing values are **not** handled: the input sequence must  
therefore be free
     of missing values.

     Parameters
     ----------
     darray : ndarray
         Input data array to clusterize.
     increment : {float}, optional
         Increment between two consecutive values to group.
         By default, use a value of 1.
     operator : {function}, optional
         Comparison operator for the definition of clusters.
         By default, use :func:`numpy.less_equal`.


     Attributes
     ----------
     inishape
         Shape of the argument array (stored for resizing).
     inisize
         Size of the argument array.
     uniques : sequence
         List of unique cluster values, as they appear in  
chronological order.
     slices : sequence
         List of the slices corresponding to each cluster of data.
     starts : ndarray
         Array of the indices at which the clusters start.
     clustered : list
         List of clustered data.


     Examples
     --------
     >>> A = [0, 0, 1, 2, 2, 2, 3, 4, 3, 4, 4, 4]
     >>> klust = cluster(A,0)
     >>> [list(_) for _ in klust.clustered]
     [[0, 0], [1], [2, 2, 2], [3], [4], [3], [4, 4, 4]]
     >>> klust.uniques
     array([0, 1, 2, 3, 4, 3, 4])

     >>> x = [ 1.8, 1.3, 2.4, 1.2, 2.5, 3.9, 1. , 3.8, 4.2, 3.3,
     ...       1.2, 0.2, 0.9, 2.7, 2.4, 2.8, 2.7, 4.7, 4.2, 0.4]
     >>> Cluster(x,1).starts
     array([ 0,  2,  3,  4,  5,  6,  7, 10, 11, 13, 17, 19])
     >>> Cluster(x,1.5).starts
     array([ 0,  6,  7, 10, 13, 17, 19])
     >>> Cluster(x,2.5).starts
     array([ 0,  6,  7, 19])
     >>> Cluster(x,2.5,greater).starts
     array([ 0,  1,  2,  3,  4,  5,  8,  9, 10,
     ...    11, 12, 13, 14, 15, 16, 17, 18])
     >>> y = [ 0, -1, 0, 0, 0, 1, 1, -1, -1, -1, 1, 1, 0, 0, 0, 0, 1,  
1, 0, 0]
     >>> Cluster(y,1).starts
     array([ 0,  1,  2,  5,  7, 10, 12, 16, 18])

     """
     def __init__(self, darray, increment=1, operator=np.less_equal):
         """
     Initializes instance.

     Parameters
     ----------
     darray : ndarray
         Input data array to clusterize.
     increment : {float}, optional
         Increment between two consecutive values to group.
         By default, use a value of 1.
     operator : {function}, optional
         Comparison operator for the definition of clusters.
         By default, use :func:`numpy.less_equal`

         """
         if hasattr(darray,'mask') and darray.mask.any():
             raise ma.MAError("Masked arrays should be filled prior  
clustering.")
         else:
             darray = np.asanyarray(darray)
         n = darray.size
         self.inishape = darray.shape
         self.inisize = darray.size
         darray = darray.ravel()
         clustercond = 1 - operator(np.absolute(np.diff(darray)),  
increment)
         sid = np.concatenate(([0, ],
                               np.arange(1, n).compress(clustercond),
                               [n, ]))
         slobj = np.asarray([slice(i, d)
                             for (i, d) in np.broadcast(sid[:-1],  
sid[1:])])
         #
         self.uniques = darray[sid[:-1]]
         self.clustered = [darray[k] for k in slobj]
         self.sizes = np.asarray(np.diff(sid))
         self.slices = slobj
         self.starts = sid[:-1]

     def markonsize(self, operator, sizethresh):
         """
     Creates a **mask** for the clusters that do not meet a size  
requirement.
     Thus, outputs ``False`` if the size requirement is met, ``True``  
otherwise.

     Parameters
     ----------
     operator : function
         Comparison operator
     sizethresh : float
         Requirement for the sizes of the clusters

         """
         resmask = np.empty(self.inisize, dtype=bool)
         resmask[:] = True
#        for k in self.slices.compress(operator(self.sizes,sizethresh)):
         for k in self.slices[operator(self.sizes, sizethresh)]:
             resmask[k] = False
         return resmask.reshape(self.inishape)

     def mark_greaterthan(self, sizemin):
         """
     Shortcut for :meth:`markonsize(greater_equal,sizemin)`.
     Thus, the command outputs ``False`` for clusters larger than  
``sizemin``, and
     ``True`` for clusters smaller than ``sizemin``.

     Parameters
     ----------
     sizemin : int
         Minimum size of the clusters.

     See Also
     --------
     :meth:`markonsize`
         Creates a **mask** for the clusters that do not meet a size  
requirement.
     """
         return self.markonsize(np.greater_equal, sizemin)

     def grouped_slices(self):
         """
     Returns a dictionary with the unique values of ``self`` as keys,  
and a list
     of slices for the corresponding values.

     See Also
     --------
     Cluster.grouped_limits
         that does the same thing
         """
         #
         uniques = self.uniques.view(np.ndarray)
         output = dict([(k, []) for k in np.unique1d(uniques)])
         for (k, v) in zip(self.uniques, self.slices):
             output[k].append(v)
         return output

     def grouped_limits(self):
         """
     Returns a dictionary with the unique values of ``self`` as keys,  
and a list
     of tuples (starting index, ending index) for the corresponding  
values.

     See Also
     --------
     Cluster.grouped_slices
         """
         output = dict([(k, []) for k in np.unique1d(self.uniques)])
         for (k, v) in zip(self.uniques, self.slices):
             output[k].append((v.start, v.stop))
         for k in output:
             output[k] = np.array(output[k])
         return output








From jturner at gemini.edu  Mon Jul  6 15:22:56 2009
From: jturner at gemini.edu (James Turner)
Date: Mon, 06 Jul 2009 15:22:56 -0400
Subject: [SciPy-user] Scipy.org down?
In-Reply-To: <4A4A74DB.50002@stsci.edu>
References: <rowen-F74741.10311424062009@news.gmane.org>	<h1tpn0$sb9$1@ger.gmane.org>
	<4A4A74DB.50002@stsci.edu>
Message-ID: <4A524F10.5000206@gemini.edu>

I can't load it or ping it. Same thing from a remote ping web site.

Thanks!

James.


From josef.pktd at gmail.com  Mon Jul  6 17:01:51 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 6 Jul 2009 17:01:51 -0400
Subject: [SciPy-user] Misunderstanding scipy.interpolate.interp2d?
In-Reply-To: <A679FCA3-EB48-462A-A8DA-788E011B1C7D@gmail.com>
References: <mailman.0.1246889566.29528.scipy-user@scipy.org>
	<A679FCA3-EB48-462A-A8DA-788E011B1C7D@gmail.com>
Message-ID: <1cd32cbb0907061401s588e756eid9df01ded1db7021@mail.gmail.com>

On Mon, Jul 6, 2009 at 10:18 AM, Adrian
Price-Whelan<adrian.prw at gmail.com> wrote:
> Hey all --
>
> Perhaps I am misunderstanding the documentation for the function
> scipy.interpolate.interp2d, or maybe I just don't understand the math
> of how it works. If it is a true 2-D interpolating function, why would
> it take 2 1-dimensional arrays as input? For instance, say I had image
> data in a 2-D array that I wanted to shift by some sub-pixel amount,
> like 0.3 pixels, to some arbitrary direction. How would I take an
> array ( something like [[1,2,3],[4,5,6],[7,8,9]] ) and break this into
> 2 1-D arrays x and y to use this function to interpolate new values
> based on the sub-pixel shift?
>
> Thanks!!
> -Adrian

x and y are the coordinates, z are the values.
If I understand your example correctly then something like this should work

import numpy as np
from scipy import interpolate

z = np.array([[1,2,3],[4,5,6],[7,8,9]])
#t,x = np.mgrid[0:3,0:3] #for full grid values
x = np.arange(3)
y = np.arange(3)

ip = interpolate.interp2d(x,y,z)
xn = np.linspace(0,2,7)
yn = np.linspace(0,2,7)
print ip(xn,yn)  # interpolated value with call

Josef


From oanjao at yahoo.com  Mon Jul  6 19:16:42 2009
From: oanjao at yahoo.com (Craig Finch)
Date: Mon, 6 Jul 2009 16:16:42 -0700 (PDT)
Subject: [SciPy-user] Scipy weave errors: undefined symbols when
	importing compiled extension
Message-ID: <823916.80562.qm@web30402.mail.mud.yahoo.com>


I've been dealing with issues like this, on this one system, for a year now.  At one point, I had a working combination of python, numpy, and scipy.  I tried so many things that I can't really remember how I got to that point.  I think I used gcc to build scipy 0.6 because I couldn't get it to work with icc, and I linked it with the Intel MKL successfully.  However, when I upgraded to scipy-0.7.0 I am again facing this problem.

The latest thing I tried was to build both Numpy and Scipy with the Intel C and Fortran compilers (icc and ifort).  Numpy builds fine and passes all its unit tests.  Scipy builds without errors or warnings, but I still have problems when trying to import any Scipy module that depends on "sparse."  

Build command:

python setup.py config --compiler=intel --fcompiler=intelem install --prefix=/home/cfinch 2>&1 > scipy_build_log.txt

Results:
>>> from scipy import sparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/__init__.py", line 6, in <module>
    from csr import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/csr.py", line 12, in <module>
    from sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/__init__.py", line 4, in <module>
    from csr import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/csr.py", line 7, in <module>
    import _csr
ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: _ZNSt8ios_base4InitD1Ev

When I use nm to look at the symbols in _csr.so, I see that these are undefined, along with several dozen other Python-related symbols:
...
00000000000d2810 W _ZNSt6vectorIxSaIxEE6resizeEmx
00000000000d2fdc W _ZNSt6vectorIySaIyEE6resizeEmy
                 U _ZNSt8ios_base4InitC1Ev
                 U _ZNSt8ios_base4InitD1Ev

I have no idea what this symbol is, or where it's supposed to come from.  Has anyone seen an error like this before?  I am going to see what kind of support I can get from Intel.

   Craig


----- Original Message ----
From: Craig Finch <oanjao at yahoo.com>
To: David Cournapeau <cournape at gmail.com>; SciPy Users List <scipy-user at scipy.org>
Sent: Sunday, July 5, 2009 2:30:18 PM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing  compiled extension

By "build log" I assume you mean the output to stderr and stdout when when building Scipy.  It's a huge amount of output, and I'm not sure exactly what to look for.  Do you want me to post a particular section?  Below is the output from sections where g++ is called.  It seems to be linked with g++ everywhere.

One more potential problem area: I'm linking with the Intel Math Kernel Library (MKL) which was almost certainly built with Intel compilers.  Could that be causing this problem?

   Craig
  
-----------------------
building 'scipy.interpolate._interpolate' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-Iscipy/interpolate/src -I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/interpolate/src/_interpolate.cpp
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/interpolate/src/_interpolate.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/interpolate/_interpolate.so
-----------------------
building 'scipy.sparse.sparsetools._csr' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

creating build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools
compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/csr_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/csr_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_csr.so
building 'scipy.sparse.sparsetools._csc' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/csc_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/csc_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_csc.so
building 'scipy.sparse.sparsetools._coo' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/coo_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/coo_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_coo.so
building 'scipy.sparse.sparsetools._bsr' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/bsr_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/bsr_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_bsr.so
building 'scipy.sparse.sparsetools._dia' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/dia_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/dia_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_dia.so

-----------------------




----- Original Message ----
From: David Cournapeau <cournape at gmail.com>
To: Craig Finch <cfinch at ieee.org>; SciPy Users List <scipy-user at scipy.org>
Sent: Friday, July 3, 2009 10:21:28 PM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing  compiled extension

On Sat, Jul 4, 2009 at 3:38 AM, Craig Finch<oanjao at yahoo.com> wrote:

> ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: __gxx_personality_v0
>
> As far as I know, everything Python-related on this machine has now been built with GNU tools, but something is still broken.  One more thing--Python, Numpy, and Scipy are all locally installed in my /home/cfinch directory because the system Python is frozen at 2.4.

This is still a C++ problem, I have sometimes noticed this problem on
some machines. Normally this is caused by a C++ object file being
linked with gcc (instead of g++). Can you post the build log of scipy
?

David


      


From eadrogue at gmx.net  Tue Jul  7 05:36:41 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Tue, 7 Jul 2009 11:36:41 +0200
Subject: [SciPy-user] Scipy.org down?
In-Reply-To: <4A524F10.5000206@gemini.edu>
References: <rowen-F74741.10311424062009@news.gmane.org>
	<h1tpn0$sb9$1@ger.gmane.org> <4A4A74DB.50002@stsci.edu>
	<4A524F10.5000206@gemini.edu>
Message-ID: <20090707093641.GA31531@doriath.local>

 6/07/09 @ 15:22 (-0400), thus spake James Turner:
> I can't load it or ping it. Same thing from a remote ping web site.

It's working now.


Ernest


From joshua.stults at gmail.com  Tue Jul  7 06:28:45 2009
From: joshua.stults at gmail.com (Joshua Stults)
Date: Tue, 7 Jul 2009 06:28:45 -0400
Subject: [SciPy-user] Bootstrap?
Message-ID: <c5ae9cf00907070328t1a820490od6943f42c1c92b26@mail.gmail.com>

Hello,

I was wondering if scipy had something similar to Octave/Matlab's
empricial_rnd().  Here's the blurb from Octave's help describing the
function:

 -- Function File:  empirical_rnd (N, DATA)
 -- Function File:  empirical_rnd (DATA, R, C)
 -- Function File:  empirical_rnd (DATA, SZ)
     Generate a bootstrap sample of size N from the empirical
     distribution obtained from the univariate sample DATA.

     If R and C are given create a matrix with R rows and C columns. Or
     if SZ is a vector, create a matrix of size SZ.

So basically you pass it an array of data, and it returns bootstrap
samples (resampling from the array with replacement).

I did a quick search on 'scipy bootstrap', 'scipy distributions' and
'scipy empirical_rnd', but didn't turn up anything promising.  Any
help / pointers greatly appreciated.  Thanks.

-- 
Joshua Stults
Website: http://j-stults.blogspot.com


From josef.pktd at gmail.com  Tue Jul  7 07:25:49 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 7 Jul 2009 07:25:49 -0400
Subject: [SciPy-user] Bootstrap?
In-Reply-To: <c5ae9cf00907070328t1a820490od6943f42c1c92b26@mail.gmail.com>
References: <c5ae9cf00907070328t1a820490od6943f42c1c92b26@mail.gmail.com>
Message-ID: <1cd32cbb0907070425j4572c688r3f49b7e6ea2154bb@mail.gmail.com>

On Tue, Jul 7, 2009 at 6:28 AM, Joshua Stults<joshua.stults at gmail.com> wrote:
> Hello,
>
> I was wondering if scipy had something similar to Octave/Matlab's
> empricial_rnd(). ?Here's the blurb from Octave's help describing the
> function:
>
> ?-- Function File: ?empirical_rnd (N, DATA)
> ?-- Function File: ?empirical_rnd (DATA, R, C)
> ?-- Function File: ?empirical_rnd (DATA, SZ)
> ? ? Generate a bootstrap sample of size N from the empirical
> ? ? distribution obtained from the univariate sample DATA.
>
> ? ? If R and C are given create a matrix with R rows and C columns. Or
> ? ? if SZ is a vector, create a matrix of size SZ.
>
> So basically you pass it an array of data, and it returns bootstrap
> samples (resampling from the array with replacement).
>
> I did a quick search on 'scipy bootstrap', 'scipy distributions' and
> 'scipy empirical_rnd', but didn't turn up anything promising. ?Any
> help / pointers greatly appreciated. ?Thanks.
>
> --
> Joshua Stults
> Website: http://j-stults.blogspot.com
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>

I was looking for bootstrap in python a while ago and didn't find much
except for one blog post.
Drawing from the data array can be done with random integers as indices:

d is data array

        sample_size = len(d)
        # Choose #sample_size members of d at random, with replacement
        choices = numpy.random.random_integers(0, sample_size-1, sample_size)
        sample = d[choices]

Josef


From eadrogue at gmx.net  Tue Jul  7 08:07:52 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Tue, 7 Jul 2009 14:07:52 +0200
Subject: [SciPy-user] Bootstrap?
In-Reply-To: <c5ae9cf00907070328t1a820490od6943f42c1c92b26@mail.gmail.com>
References: <c5ae9cf00907070328t1a820490od6943f42c1c92b26@mail.gmail.com>
Message-ID: <20090707120752.GA31860@doriath.local>

 7/07/09 @ 06:28 (-0400), thus spake Joshua Stults:
> So basically you pass it an array of data, and it returns bootstrap
> samples (resampling from the array with replacement).

You can use the random module from Python:

In [40]: d = (1,2,3,4)

In [41]: [random.choice(d) for i in range(len(d))]
Out[41]: [4, 4, 2, 1]

In [42]: [random.choice(d) for i in range(len(d))]
Out[42]: [4, 2, 4, 2]

In [43]: [random.choice(d) for i in range(len(d))]
Out[43]: [2, 1, 2, 3]


Bye.

Ernest


From V.vanBeveren at rijnhuizen.nl  Tue Jul  7 10:03:07 2009
From: V.vanBeveren at rijnhuizen.nl (Vincent van Beveren)
Date: Tue, 7 Jul 2009 16:03:07 +0200
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
Message-ID: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>

Hello everyone,

I'm an engineer at Rijnhuizen, which is a research institute for plasma physics. We use Python to drive one of our main research projects, however, in the scientific circles in this institute Python (and SciPy) are largely unknown. Time to change this, I think :). However, since I am an engineer and not a scientist I my viewpoint on Python and SciPy are more of an engineering perspective, like its Open Source, Free, modern language, functional programming, etc... I'm not entirely sure these are compelling arguments for a scientist to start working with Python (or atleast not on it self). So I was wondering, if I was to promote Python in the scientific community here at Rijnhuizen. So I have a few questions:

-          In what aspects does SciPy excel, compared to say IDL or matlab?
-          In what ways allows it a scientist to be more effective?
-          How usable is SciPy for Plasma physics, molucular dynamics and nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
-          How stable is it compared to other packages. (bugs, computation)?

Any comments and insights are welcome!

Regards,
Vincent

PS. Posted in scipy-dev by accident.
___
Ing. V. van Beveren
Software Engineer, FOM Rijnhuizen
E: V.vanBeveren at rijnhuizen.nl<mailto:V.vanBeveren at rijnhuizen.nl>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090707/14446ce4/attachment.html>

From devicerandom at gmail.com  Tue Jul  7 10:33:49 2009
From: devicerandom at gmail.com (ms)
Date: Tue, 07 Jul 2009 15:33:49 +0100
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <4A535CCD.9060503@gmail.com>

Vincent van Beveren ha scritto:
> -          In what aspects does SciPy excel, compared to say IDL or matlab?

I would say that its main strength is being a library of a
full-featured, well-supported, cross-platform programming language. This
means you can build arbitrarily complex applications using Python and
Scipy, while it would be very painful with Matlab.

> -          In what ways allows it a scientist to be more effective?

See above.

> -          How usable is SciPy for Plasma physics, molucular dynamics and nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?

I don't think one could write molecular dynamics engines in python, it's
anyway too slow (I hope someone can tell me I'm wrong!)

But it can be very useful for all kinds of data analysis and
not-enormously-heavy computations (that's what I use it for).

m.


From gokhansever at gmail.com  Tue Jul  7 10:58:51 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 7 Jul 2009 09:58:51 -0500
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <49d6b3500907070758t7d9907b8ob9b893ded99b4ff6@mail.gmail.com>

On Tue, Jul 7, 2009 at 9:03 AM, Vincent van Beveren <
V.vanBeveren at rijnhuizen.nl> wrote:

>  Hello everyone,
>
>
>
> I?m an engineer at Rijnhuizen, which is a research institute for plasma
> physics. We use Python to drive one of our main research projects, however,
> in the scientific circles in this institute Python (and SciPy) are largely
> unknown. Time to change this, I think J. However, since I am an engineer
> and not a scientist I my viewpoint on Python and SciPy are more of an
> engineering perspective, like its Open Source, Free, modern language,
> functional programming, etc... I?m not entirely sure these are compelling
> arguments for a scientist to start working with Python (or atleast not on it
> self). So I was wondering, if I was to promote Python in the scientific
> community here at Rijnhuizen. So I have a few questions:
>
>
>
> -          In what aspects does SciPy excel, compared to say IDL or matlab
> ?
>
> -          In what ways allows it a scientist to be more effective?
>
> -          How usable is SciPy for Plasma physics, molucular dynamics and
> nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
>
> -          How stable is it compared to other packages. (bugs,
> computation)?
>
>
>
> Any comments and insights are welcome!
>
>
>
> Regards,
>
> Vincent
>
>
>
> PS. Posted in scipy-dev by accident.
>
> ___
>
> Ing. V. van Beveren
>
> Software Engineer, FOM Rijnhuizen
>
> E: V.vanBeveren at rijnhuizen.nl
>
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
Vincent,

Here a couple of sources that has inspired me to choose Python for my
scientific data analysis and visualization tasks:

http://conference.scipy.org/proceedings/SciPy2008/

https://cirl.berkeley.edu/fperez/

A Demonstration of the 'IPython' Interactive
Shell<http://video.google.com/videoplay?docid=2025411588241625522>

Scientific Computing with Python <http://nanohub.org/resources/99>

Matplotlib by John D. Hunter
<http://videolectures.net/mloss08_hunter_mat/>(watched on April 12,
2009)

''Sage: Unifying Mathematical
Software'''<http://wstein.org/talks/20090622-sagedays16-thematic/>,
video part 1<http://wstein.org/edu/2009/sd16/00021-stein-unifying_open_source_math_software-part1.m4v>,
video part 2<http://wstein.org/edu/2009/sd16/00022-stein-unifying_open_source_math_software-part2.m4v>

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090707/0645eeec/attachment.html>

From hoytak at cs.ubc.ca  Tue Jul  7 11:29:30 2009
From: hoytak at cs.ubc.ca (Hoyt Koepke)
Date: Tue, 7 Jul 2009 08:29:30 -0700
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <4db580fd0907070829mc6ce8fet1e4f211b92fc9b38@mail.gmail.com>

Hi Vincent,

> -????????? In what aspects does SciPy excel, compared to say IDL or matlab?
>
> -????????? In what ways allows it a scientist to be more effective?

A while ago, I wrote up a quick article on a few reasons that I think
python is great for research stuff.  It's a bit unpolished, but it
gives 10 points that I think are really advantageous.  It's at
http://www.stat.washington.edu/~hoytak/blog/whypython.html.

Let me know if anyone has feedback or comments on it; it's a work in
progress at this point.

--Hoyt

++++++++++++++++++++++++++++++++++++++++++++++++
+ Hoyt Koepke
+ University of Washington Department of Statistics
+ http://www.stat.washington.edu/~hoytak/
+ hoytak at gmail.com
++++++++++++++++++++++++++++++++++++++++++


From oanjao at yahoo.com  Tue Jul  7 11:42:02 2009
From: oanjao at yahoo.com (Craig Finch)
Date: Tue, 7 Jul 2009 08:42:02 -0700 (PDT)
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;
	) (Vincent van Beveren)
In-Reply-To: <mailman.3641.1246978748.2410.scipy-user@scipy.org>
References: <mailman.3641.1246978748.2410.scipy-user@scipy.org>
Message-ID: <767369.32981.qm@web30401.mail.mud.yahoo.com>


As a PhD candidate working in computational nanoscience, I have found Python to be extremely helpful.  I had extensive experience using Matlab before starting my research.  I will tell you about the incident that prompted me to switch to Python.  I wrote a GUI for a data analysis program using Matlab.  It worked well for about six months, and then I decided to make some changes in the GUI.  I went back to look at my code and I had no idea what was going on.  I spent a few hours trying to figure out the mess and decided it would just be faster to do it in Python, even though I was much less familiar with that language.  Matlab is fundamentally a procedural language--it's fine for writing simple one-page programs, but it quickly becomes unmanageable when writing larger, more complex programs.  I find that my Python code is much easier to read and organize.

There is a Python interface (through Numpy and Scipy) to all of the classic numerical routines found on Netlib.org.  The iPython interactive shell, combined with Matplotlib, provide interactive computing and high-quality plotting.  I have not run into any situation in my research where I regretted using Python because some feature was not available, nor have I hit any critical bugs.

I am using Python to write a Brownian dynamics simulator for interface chemistry.  While the interpreted parts of a Python program run more slowly than C or Fortran, most of the time-consuming numerical operations are performed compiled C or Fortran libraries, so the speed difference is not as much as you might think.  Further, there are numerous ways to accelerate Python programs by integrating compiled code.  Check out scipy.weave.inline, which lets you write C code write in the middle of a Python program to accelerate the slow parts.  Tools like boost.python and f2py make it easy for you to write C and Fortran libraries for the slowest portions of your code.  My development cycle is simple: quickly write everything in Python, profile the code to find what runs slowly, and write those parts in C or Fortran (if necessary).  

Overall, I'd recommend Python to another scientist because it's the fastest way to go from ideas to computational results.

   Craig



Message: 3
Date: Tue, 7 Jul 2009 16:03:07 +0200
From: Vincent van Beveren <V.vanBeveren at rijnhuizen.nl>
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
To: "'scipy-user at scipy.org'" <scipy-user at scipy.org>
Message-ID: <2926F4BC94217A43A2D21792DE8818931C5B245B29 at ex1.rijnh.nl>
Content-Type: text/plain; charset="us-ascii"

Hello everyone,

I'm an engineer at Rijnhuizen, which is a research institute for plasma physics. We use Python to drive one of our main research projects, however, in the scientific circles in this institute Python (and SciPy) are largely unknown. Time to change this, I think :). However, since I am an engineer and not a scientist I my viewpoint on Python and SciPy are more of an engineering perspective, like its Open Source, Free, modern language, functional programming, etc... I'm not entirely sure these are compelling arguments for a scientist to start working with Python (or atleast not on it self). So I was wondering, if I was to promote Python in the scientific community here at Rijnhuizen. So I have a few questions:

-          In what aspects does SciPy excel, compared to say IDL or matlab?
-          In what ways allows it a scientist to be more effective?
-          How usable is SciPy for Plasma physics, molucular dynamics and nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
-          How stable is it compared to other packages. (bugs, computation)?

Any comments and insights are welcome!

Regards,
Vincent

PS. Posted in scipy-dev by accident.
___
Ing. V. van Beveren
Software Engineer, FOM Rijnhuizen
E: V.vanBeveren at rijnhuizen.nl<mailto:V.vanBeveren at rijnhuizen.nl>



      


From dwf at cs.toronto.edu  Tue Jul  7 12:14:03 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Tue, 7 Jul 2009 12:14:03 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>

On 7-Jul-09, at 10:03 AM, Vincent van Beveren wrote:

> -          In what aspects does SciPy excel, compared to say IDL or  
> matlab?

Vectorized operations are (almost) as fast. Depending on what BLAS you  
use, linear algebra can be just as fast. String processing in Python  
is so incredibly better than Matlab that the thought of doing string  
operations in Matlab almost makes me cry. Python has bindings to  
nearly everything -- a dozen different GUI toolkits to low-level  
hardware interfaces.  Likewise there are a handful of different ways  
to interface your code to code written in other languages -- ctypes,  
swig, boost::python, cython for C/C++, PyObjC for Objective C, f2py  
for Fortran, scikits.mlabwrap for Matlab, rpy and rpy2 for R...

> -          In what ways allows it a scientist to be more effective?

As has been echoed here, writing large programs will be easier. Python  
is a real programming language and was designed that way from the  
ground up. Matlab was designed as an interactive environment first and  
programming language features were grafted on as an afterthought.

> -          How stable is it compared to other packages. (bugs,  
> computation)?

I won't lie in that I occasionally run into bugs in SciPy, but one of  
three things usually happens:

- I look at the code myself -- all of which is available to view and  
modify, not the case with Matlab -- and file a bug report and/or patch
- I look at the code and realize it's not a bug but me  
misunderstanding something, at which point I might edit the  
documentation at http://docs.scipy.org/scipy/ to make it clearer if  
necessary.
- If the code is incomprehensible to me, I ask on the mailing lists  
and (either already or shortly thereafter) it's fixed by someone else  
in Subversion.




From bsouthey at gmail.com  Tue Jul  7 12:39:48 2009
From: bsouthey at gmail.com (Bruce Southey)
Date: Tue, 07 Jul 2009 11:39:48 -0500
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
Message-ID: <4A537A54.2010601@gmail.com>

On 07/07/2009 11:14 AM, David Warde-Farley wrote:
> On 7-Jul-09, at 10:03 AM, Vincent van Beveren wrote:
>
>    
>> -          In what aspects does SciPy excel, compared to say IDL or
>> matlab?
>>      
>
> Vectorized operations are (almost) as fast. Depending on what BLAS you
> use, linear algebra can be just as fast. String processing in Python
> is so incredibly better than Matlab that the thought of doing string
> operations in Matlab almost makes me cry. Python has bindings to
> nearly everything -- a dozen different GUI toolkits to low-level
> hardware interfaces.  Likewise there are a handful of different ways
> to interface your code to code written in other languages -- ctypes,
> swig, boost::python, cython for C/C++, PyObjC for Objective C, f2py
> for Fortran, scikits.mlabwrap for Matlab, rpy and rpy2 for R...
>    

I would add because it is not always obvious and overlooked is that 
Python really integrates very different things into a single unified yet 
simple interface. As an example, you can very easily 'bolt on' web stuff 
to an existing peice of Python code to create an almost instant web 
application. (I have mixed linear algebra with html processing including 
parsing cgi and creating html tags and images to provide a couple of web 
apps.) So suddenly instead of saying I can do solve this problem (and 
perhaps here is the code), you can just say 'go to this web site and do 
it'. Plus, I can easily fix bugs and add new code without the user doing 
anything. This is great publicly to grant agencies and such like because 
you can actually show what you have done and also provide some outreach 
to the community.

As a shameless plug, Frontiers in Neuroinformatics had a special issue 
on Python:
http://www.frontiersin.org/neuroinformatics/
http://www.frontiersin.org/neuroinformatics/specialtopics/8/

Bruce

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090707/61913513/attachment.html>

From fperez.net at gmail.com  Tue Jul  7 12:53:29 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Jul 2009 09:53:29 -0700
Subject: [SciPy-user] assign to diagonal values?
In-Reply-To: <9457e7c80907040523g734aa3c0nab686af1476a731d@mail.gmail.com>
References: <47A128A5.7010406@sci.utah.edu>
	<Mahogany-0.67.0-1804-20080130-220801.00@american.edu> 
	<DD23E9F4-9327-4371-8767-6DFB77FADAD7@cs.toronto.edu>
	<2bc7a5a50801302128g79da7ea6jc95ef9abb00b3785@mail.gmail.com> 
	<db6b5ecc0907011238r5e611f51y2c01052bbc0ae83b@mail.gmail.com> 
	<9457e7c80907040523g734aa3c0nab686af1476a731d@mail.gmail.com>
Message-ID: <db6b5ecc0907070953n6595c72dsd8ad2f7e54647920@mail.gmail.com>

Hey Stefan,

2009/7/4 St?fan van der Walt <stefan at sun.ac.za>:
> 2009/7/1 Fernando Perez <fperez.net at gmail.com>:
>> If someone feels like reviewing this ticket:
>>
>> http://projects.scipy.org/numpy/attachment/ticket/1132/numpy-index-funcs.diff
>>
>> it has the above and a few more utilities, with docs and tests.
>
> Thanks! Reviewed and applied in r7101 through r7103.

thanks for working on that ticket!

BTW, looking at

numpy/lib/index_tricks.py

line 740:

step = cumprod((1,)+a.shape[:-1]).sum()

might be better written as suggested by Anand:

step = 1+(cumprod(a.shape[:-1])).sum()

which replaces a tuple concatenation by a simple numerical addition.

Order-epsilon nit though.  Thanks for the extra tests too!

Best,

f


From robince at gmail.com  Tue Jul  7 12:55:48 2009
From: robince at gmail.com (Robin)
Date: Tue, 7 Jul 2009 17:55:48 +0100
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <b14b627d0907070955l38bd88e2rdae17bdba39779fe@mail.gmail.com>

On Tue, Jul 7, 2009 at 3:03 PM, Vincent van
Beveren<V.vanBeveren at rijnhuizen.nl> wrote:
> -????????? In what aspects does SciPy excel, compared to say IDL or matlab?
>
> -????????? In what ways allows it a scientist to be more effective?
>
> -????????? How usable is SciPy for Plasma physics, molucular dynamics and
> nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
>
> -????????? How stable is it compared to other packages. (bugs, computation)?

I think it's probably been covered by others but some of the key
things for me are:

- how easy it is to extend performance critical portions in C or
FORTRAN (cython, ctypes, f2py, weave etc.)
http://www.scipy.org/PerformancePython may be slightly out of date but
I think it's a great illustration.

- how easy it is (relatively speaking) to employ parralel processing.
http://www.scipy.org/ParallelProgramming . Using the multiprocessing
module (available as a backport for 2.5) it is literally 2 lines to
have a for loop parallellised over 8 cores (or 16 if I had them). In
MATLAB anything more than 4 cores gets into really silly money (not
usually included in site license). This is probably the major factor
driving interest in Python from colleagues in my lab. Also of course
there are parralel python, ipython etc. and python bindings to MPI,
cuda, opencl etc. which I think would usually allow much quicker
progress than working in C.

- just how nice it is to use; broadcasting... even just how nice it is
to handle default arguments in functions and update code with new
parameters in a backwards compatible was was a  bit of a revelation
for me after matlab.

- slices as views - in my work I usually have one long data set which
I slice and view in different ways. In matlab I think slices make
copies (at least they used to - possibly its changed now, but another
thing is that you can never really know what matlabs going to do,
whereas in Python you really can control how memory is laid out and
manipulated at a low level) but in Python all my different views point
to the same data.

cheers

Robin


From dwf at cs.toronto.edu  Tue Jul  7 13:22:35 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Tue, 7 Jul 2009 13:22:35 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <4A537A54.2010601@gmail.com>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
	<4A537A54.2010601@gmail.com>
Message-ID: <1BB30B2C-4CE8-44C7-82FF-62E9EFF0FB7C@cs.toronto.edu>


On 7-Jul-09, at 12:39 PM, Bruce Southey wrote:

> As an example, you can very easily 'bolt on' web stuff to an  
> existing peice of Python code to create an almost instant web  
> application. (I have mixed linear algebra with html processing  
> including parsing cgi and creating html tags and images to provide a  
> couple of web apps.) So suddenly instead of saying I can do solve  
> this problem (and perhaps here is the code), you can just say 'go to  
> this web site and do it'. Plus, I can easily fix bugs and add new  
> code without the user doing anything.

Indeed, I've done this too.  Getting matplotlib to render PNGs for  
TurboGears/Django/your-favourite-web-framework is dead simple, too so  
you can provide really quite fancy visualization in your web-based  
interface without having to do anything extra.

David


From cimrman3 at ntc.zcu.cz  Tue Jul  7 13:42:57 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Tue, 07 Jul 2009 19:42:57 +0200
Subject: [SciPy-user] circumference in raster image
Message-ID: <4A538921.9060800@ntc.zcu.cz>

Hi!

Hoping that there are some raster image experts around: I need to 
compute an approximation of circumference of an object (with smooth 
boundary) represented as a filled area in a 2d array. Can scipy.ndimage 
deal with that?

thanks,
r.


From zachary.pincus at yale.edu  Tue Jul  7 14:34:04 2009
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Tue, 7 Jul 2009 14:34:04 -0400
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <4A538921.9060800@ntc.zcu.cz>
References: <4A538921.9060800@ntc.zcu.cz>
Message-ID: <AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>

Hi Robert,

Basically, assuming the object is in a binarized array, you could use  
ndimage to do one iteration of erosion, giving you the same object but  
one pixel smaller. Then xor the eroded and original binary images to  
give an image where the single-pixel border around the object is 1 and  
the rest is zero; from here you can just sum the pixels to give a  
(very rough) perimeter value. (Note of course that this doesn't  
account for the spacing between pixels being different on the diagonal  
than horizontal or vertical... for that you'd need some chain code  
things, which I think ndimage doesn't provide.)

Personally, in situations like these -- especially when the original  
image is not binary and I'd need to threshold to get a binary image --  
I usually run a marching-squares algorithm over the data to extract  
interpolated iso-intensity contours for a particular threshold; these  
contours are polygons with which it is easy to calculate fairly  
accurate perimeter, area, etc. values. I can send a C extension that  
does this very quickly, if desired.

Zach



On Jul 7, 2009, at 1:42 PM, Robert Cimrman wrote:

> Hi!
>
> Hoping that there are some raster image experts around: I need to
> compute an approximation of circumference of an object (with smooth
> boundary) represented as a filled area in a 2d array. Can  
> scipy.ndimage
> deal with that?
>
> thanks,
> r.
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From seb.haase at gmail.com  Tue Jul  7 16:22:47 2009
From: seb.haase at gmail.com (Sebastian Haase)
Date: Tue, 7 Jul 2009 22:22:47 +0200
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>
References: <4A538921.9060800@ntc.zcu.cz>
	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>
Message-ID: <bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>

On Tue, Jul 7, 2009 at 8:34 PM, Zachary Pincus<zachary.pincus at yale.edu> wrote:
> Hi Robert,
>
> Basically, assuming the object is in a binarized array, you could use
> ndimage to do one iteration of erosion, giving you the same object but
> one pixel smaller. Then xor the eroded and original binary images to
> give an image where the single-pixel border around the object is 1 and
> the rest is zero; from here you can just sum the pixels to give a
> (very rough) perimeter value. (Note of course that this doesn't
> account for the spacing between pixels being different on the diagonal
> than horizontal or vertical... for that you'd need some chain code
> things, which I think ndimage doesn't provide.)
>
> Personally, in situations like these -- especially when the original
> image is not binary and I'd need to threshold to get a binary image --
> I usually run a marching-squares algorithm over the data to extract
> interpolated iso-intensity contours for a particular threshold; these
> contours are polygons with which it is easy to calculate fairly
> accurate perimeter, area, etc. values. I can send a C extension that
> does this very quickly, if desired.
>
> Zach
>
Hi Zach,
I would also be interested in that code - if you don't mind sending me
a copy ....

Thanks,
Sebastian



>
>
> On Jul 7, 2009, at 1:42 PM, Robert Cimrman wrote:
>
>> Hi!
>>
>> Hoping that there are some raster image experts around: I need to
>> compute an approximation of circumference of an object (with smooth
>> boundary) represented as a filled area in a 2d array. Can
>> scipy.ndimage
>> deal with that?
>>
>> thanks,
>> r.


From zachary.pincus at yale.edu  Tue Jul  7 16:56:32 2009
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Tue, 7 Jul 2009 16:56:32 -0400
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>
References: <4A538921.9060800@ntc.zcu.cz>
	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>
	<bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>
Message-ID: <DC866963-E5B1-45AF-98D8-E75ADAA75098@yale.edu>

Hi all,

Attached is find_contours, an extension module that finds all contours  
in a 2D array at a specified iso-value. The contour position is  
linearly interpolated between pixels. Let me know if you have any  
questions, etc. (Looking over the code, I'm surprised how well I  
documented and commented it... lucky break for anyone who wants to use  
or modify it, I guess.) It's GPL because it comes from a larger GPL'd  
project of mine, but just ask me and I'll send it under a different  
license.

Zach


On Jul 7, 2009, at 4:22 PM, Sebastian Haase wrote:

> On Tue, Jul 7, 2009 at 8:34 PM, Zachary  
> Pincus<zachary.pincus at yale.edu> wrote:
>> Hi Robert,
>>
>> Basically, assuming the object is in a binarized array, you could use
>> ndimage to do one iteration of erosion, giving you the same object  
>> but
>> one pixel smaller. Then xor the eroded and original binary images to
>> give an image where the single-pixel border around the object is 1  
>> and
>> the rest is zero; from here you can just sum the pixels to give a
>> (very rough) perimeter value. (Note of course that this doesn't
>> account for the spacing between pixels being different on the  
>> diagonal
>> than horizontal or vertical... for that you'd need some chain code
>> things, which I think ndimage doesn't provide.)
>>
>> Personally, in situations like these -- especially when the original
>> image is not binary and I'd need to threshold to get a binary image  
>> --
>> I usually run a marching-squares algorithm over the data to extract
>> interpolated iso-intensity contours for a particular threshold; these
>> contours are polygons with which it is easy to calculate fairly
>> accurate perimeter, area, etc. values. I can send a C extension that
>> does this very quickly, if desired.
>>
>> Zach
>>
> Hi Zach,
> I would also be interested in that code - if you don't mind sending me
> a copy ....
>
> Thanks,
> Sebastian
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: find_contours.zip
Type: application/zip
Size: 6411 bytes
Desc: not available
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090707/7f86c182/attachment.zip>

From gruben at bigpond.net.au  Tue Jul  7 19:03:54 2009
From: gruben at bigpond.net.au (Gary Ruben)
Date: Wed, 08 Jul 2009 09:03:54 +1000
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <4A538921.9060800@ntc.zcu.cz>
References: <4A538921.9060800@ntc.zcu.cz>
Message-ID: <4A53D45A.1050908@bigpond.net.au>

I recently sat in on a lecture on how to do this. In summary, there is 
no best way to do it - it's an open research problem.
Search for "Best Line Unbiased Estimators" to find out more.
If you just walk around the discretised 8-connected circumference and 
try to compute pi from it, it will converge to 8(sqrt(2)-1), which is 
close to pi and comes from the sum of sides of an octagon enclosing the 
circle (this makes sense because all the traversals can be ordered to 
lie on this octagon), so you need to be smarter, unless that's an 
acceptable uncertainty for your problem.

Zach's suggestion to use find_contours is probably a good one, depending 
on how the contouring is done - certainly sounds like a promising approach,

Gary R.

Robert Cimrman wrote:
> Hi!
> 
> Hoping that there are some raster image experts around: I need to 
> compute an approximation of circumference of an object (with smooth 
> boundary) represented as a filled area in a 2d array. Can scipy.ndimage 
> deal with that?
> 
> thanks,
> r.



From robert.kern at gmail.com  Tue Jul  7 21:20:47 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 7 Jul 2009 20:20:47 -0500
Subject: [SciPy-user] Additional Kern-ification of zunzun.com
In-Reply-To: <268756d30907040922k76cfdde1gc73cfc7cdbab4fcd@mail.gmail.com>
References: <268756d30907040922k76cfdde1gc73cfc7cdbab4fcd@mail.gmail.com>
Message-ID: <3d375d730907071820w16ba7aaeq518275d966569c4a@mail.gmail.com>

On Sat, Jul 4, 2009 at 11:22, James Phillips<zunzun at zunzun.com> wrote:
> ODR is now one of the available fitting targets at http://zunzun.com, and
> again my thanks to Robert Kern.

Look on my works, ye mighty, and despair!

Good work James! Thanks for the heads up.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From jh at physics.ucf.edu  Tue Jul  7 21:36:29 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Tue, 07 Jul 2009 21:36:29 -0400
Subject: [SciPy-user] Bootstrap?
In-Reply-To: <mailman.3641.1246978748.2410.scipy-user@scipy.org>
	(scipy-user-request@scipy.org)
References: <mailman.3641.1246978748.2410.scipy-user@scipy.org>
Message-ID: <wl8iqi40xcy.fsf@glup.physics.ucf.edu>

On Tue, Jul 7, 2009 at 6:28 AM, Joshua Stults<joshua.stults at gmail.com> wrote:

> I was wondering if scipy had something similar to Octave/Matlab's
> empricial_rnd(). ?Here's the blurb from Octave's help describing the
> function:
>
> ?-- Function File: ?empirical_rnd (N, DATA)
> ?-- Function File: ?empirical_rnd (DATA, R, C)
> ?-- Function File: ?empirical_rnd (DATA, SZ)
> ? ? Generate a bootstrap sample of size N from the empirical
> ? ? distribution obtained from the univariate sample DATA.
>
> ? ? If R and C are given create a matrix with R rows and C columns. Or
> ? ? if SZ is a vector, create a matrix of size SZ.
>
> So basically you pass it an array of data, and it returns bootstrap
> samples (resampling from the array with replacement).
>

Be very careful and be certain you can derive the statistical
justification for what you are doing when you use bootstrap.  There
are numerous cases in which bootstrapping will not give you the right
answer, such as when fitting a function that has a parameter that is
set in just a small subset of the data, because in some samples the
subset may be omitted completely or in large part, admitting wildly
wrong parameter values.  While you didn't specify exactly what you are
trying to do, for many problems Markov-Chain Monte Carlo is both
better and faster, and is often easier to code.  Plus, there is Python
for it (pymc, I think).

--jh--


From joshua.stults at gmail.com  Tue Jul  7 22:07:15 2009
From: joshua.stults at gmail.com (Joshua Stults)
Date: Tue, 7 Jul 2009 22:07:15 -0400
Subject: [SciPy-user] Bootstrap?
In-Reply-To: <wl8iqi40xcy.fsf@glup.physics.ucf.edu>
References: <mailman.3641.1246978748.2410.scipy-user@scipy.org>
	<wl8iqi40xcy.fsf@glup.physics.ucf.edu>
Message-ID: <c5ae9cf00907071907u3dde2546i500f7a718adf8784@mail.gmail.com>

Joe,

Thanks for the tip.

On Tue, Jul 7, 2009 at 9:36 PM, Joe Harrington<jh at physics.ucf.edu> wrote:
> On Tue, Jul 7, 2009 at 6:28 AM, Joshua Stults<joshua.stults at gmail.com> wrote:
>
>> I was wondering if scipy had something similar to Octave/Matlab's
>> empricial_rnd(). ?Here's the blurb from Octave's help describing the
>> function:
>>
>> ?-- Function File: ?empirical_rnd (N, DATA)
>> ?-- Function File: ?empirical_rnd (DATA, R, C)
>> ?-- Function File: ?empirical_rnd (DATA, SZ)
>> ? ? Generate a bootstrap sample of size N from the empirical
>> ? ? distribution obtained from the univariate sample DATA.
>>
>> ? ? If R and C are given create a matrix with R rows and C columns. Or
>> ? ? if SZ is a vector, create a matrix of size SZ.
>>
>> So basically you pass it an array of data, and it returns bootstrap
>> samples (resampling from the array with replacement).
>>
>
> Be very careful and be certain you can derive the statistical
> justification for what you are doing when you use bootstrap. ?There
> are numerous cases in which bootstrapping will not give you the right
> answer, such as when fitting a function that has a parameter that is
> set in just a small subset of the data, because in some samples the
> subset may be omitted completely or in large part, admitting wildly
> wrong parameter values.

I was doing a toy problem with 0-1 data (1=success, 0=failure),
estimating a reliability.  So my statistic was just:
sum(bootstrap_sample) / n.

Does your criticism apply to bootstrapping the residuals too?  I'd
appreciate if you could point me towards any accessible (I'm not a
statistician) references.

> While you didn't specify exactly what you are
> trying to do, for many problems Markov-Chain Monte Carlo is both
> better and faster, and is often easier to code. ?Plus, there is Python
> for it (pymc, I think).

Could you give an example where it's easier to code an MCMC method?
Doing a bootstrap is one or two lines of code in most high level
languages (eg Matlab/Octave), and turns out Python too using the
random indexing method that Josef and Ernest posted (of course you
have to put it in an interpreted loop, which is not very scalable).

>
> --jh--
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>

Thanks again, I've been consistently impressed by the quality of
responses on this list.

-- 
Joshua Stults
Website: http://j-stults.blogspot.com


From jongisli at gmail.com  Wed Jul  8 06:09:31 2009
From: jongisli at gmail.com (=?ISO-8859-1?Q?J=F3n_Egilsson?=)
Date: Wed, 8 Jul 2009 10:09:31 +0000
Subject: [SciPy-user] Problems with the interpolation package
Message-ID: <44cd5a960907080309t481ba009l36fa53d2dee3806e@mail.gmail.com>

Hi there.

I've now tried various functions in the scipy.interpolate package without
getting far.

*What I want to do:*

I want to interpolate using spline interpolation, cubic spline interpolation
to be more precise.

1. I want to be able to choose which points to interpolate (to choose
between which points on the x-axis the cubic polynomials are). Then I'll
have some polynomials on the form ax^3 + bx^2 + cx + d

2. Then I want to be able to extract the coefficients a,b,c,d from each
polynomial because that's the only thing I want to keep. I don't want to
keep an interpolation object of some sort.

*Input:*

Two lists, one representing the x-axis: [0..n] and another one with the
function values: [y0, y1, y2, ... , yn] and the desired points I wish to
interpolate between.

*Output (preferably):*

The coefficients (a,b,c,d) for every polynomial beetween the points I chose
to interpolate between.

Any help at all will be greatly appreciated. Cheers.

J?n G?sli Egilsson.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090708/a614bd3b/attachment.html>

From timmichelsen at gmx-topmail.de  Wed Jul  8 07:48:22 2009
From: timmichelsen at gmx-topmail.de (Tim Michelsen)
Date: Wed, 8 Jul 2009 11:48:22 +0000 (UTC)
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
	<4A537A54.2010601@gmail.com>
	<1BB30B2C-4CE8-44C7-82FF-62E9EFF0FB7C@cs.toronto.edu>
Message-ID: <loom.20090708T114047-554@post.gmane.org>

Hello.

David Warde-Farley <dwf <at> cs.toronto.edu> writes:
> On 7-Jul-09, at 12:39 PM, Bruce Southey wrote:
> > As an example, you can very easily 'bolt on' web stuff to an  
> > existing peice of Python code to create an almost instant web  
> > application.
[...]
> Indeed, I've done this too.  Getting matplotlib to render PNGs for  
> TurboGears/Django/your-favourite-web-framework is dead simple, too so  
> you can provide really quite fancy visualization in your web-based  
> interface without having to do anything extra.

@Bruce, David
Do you have  a particular site that is one-line and 
available to the gereral public?
I am intending something similar.

Although it is not specially topic of this list, I am particulary interested in
how you generate the matplotlib graphics on the fly.
To you store them in a temporary directory or do create 
the files somehow on the fly?

Would be nice to see your implementation.

Thanks,
Timmie



From timmichelsen at gmx-topmail.de  Wed Jul  8 07:52:39 2009
From: timmichelsen at gmx-topmail.de (Tim Michelsen)
Date: Wed, 8 Jul 2009 11:52:39 +0000 (UTC)
Subject: [SciPy-user]
	=?utf-8?q?Why_is_SciPy_better_than_Matlab_or_IDL=3F_?=
	=?utf-8?q?=3B=09=29_=28Vincent_van_Beveren=29?=
References: <mailman.3641.1246978748.2410.scipy-user@scipy.org>
	<767369.32981.qm@web30401.mail.mud.yahoo.com>
Message-ID: <loom.20090708T115057-383@post.gmane.org>

> do it in Python, even though I was much less familiar with that language. 
Matlab is fundamentally a
> procedural language--it's fine for writing simple one-page programs, but it
quickly becomes
> unmanageable when writing larger, more complex programs
>  .  I find that my Python code is much easier to read and organize.
I have the same perception when I see R-project scripts.

I think R is good for full-fledged and proven statistics. The better the Python
interface to R is the better...



From cimrman3 at ntc.zcu.cz  Wed Jul  8 08:38:51 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 08 Jul 2009 14:38:51 +0200
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <DC866963-E5B1-45AF-98D8-E75ADAA75098@yale.edu>
References: <4A538921.9060800@ntc.zcu.cz>	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>	<bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>
	<DC866963-E5B1-45AF-98D8-E75ADAA75098@yale.edu>
Message-ID: <4A54935B.4060209@ntc.zcu.cz>

Hi Zach!

Zachary Pincus wrote:
> Hi all,
> 
> Attached is find_contours, an extension module that finds all contours 
> in a 2D array at a specified iso-value. The contour position is linearly 
> interpolated between pixels. Let me know if you have any questions, etc. 
> (Looking over the code, I'm surprised how well I documented and 
> commented it... lucky break for anyone who wants to use or modify it, I 
> guess.) It's GPL because it comes from a larger GPL'd project of mine, 
> but just ask me and I'll send it under a different license.

Thank you very much, I will try your code. This is the way I thought I 
might try (interpolating by a smooth function/polynomial), so it's cool 
I do not have to dive into it :-)

I would like to use it within my BSD code [1], so it would be awesome if 
you relicensed the code to BSD.

Thanks again,
r.

[1] http://github.com/rc/gensei/tree/master

> On Jul 7, 2009, at 4:22 PM, Sebastian Haase wrote:
> 
>> On Tue, Jul 7, 2009 at 8:34 PM, Zachary 
>> Pincus<zachary.pincus at yale.edu> wrote:
>>> Hi Robert,
>>>
>>> Basically, assuming the object is in a binarized array, you could use
>>> ndimage to do one iteration of erosion, giving you the same object but
>>> one pixel smaller. Then xor the eroded and original binary images to
>>> give an image where the single-pixel border around the object is 1 and
>>> the rest is zero; from here you can just sum the pixels to give a
>>> (very rough) perimeter value. (Note of course that this doesn't
>>> account for the spacing between pixels being different on the diagonal
>>> than horizontal or vertical... for that you'd need some chain code
>>> things, which I think ndimage doesn't provide.)
>>>
>>> Personally, in situations like these -- especially when the original
>>> image is not binary and I'd need to threshold to get a binary image --
>>> I usually run a marching-squares algorithm over the data to extract
>>> interpolated iso-intensity contours for a particular threshold; these
>>> contours are polygons with which it is easy to calculate fairly
>>> accurate perimeter, area, etc. values. I can send a C extension that
>>> does this very quickly, if desired.
>>>
>>> Zach
>>>
>> Hi Zach,
>> I would also be interested in that code - if you don't mind sending me
>> a copy ....
>>
>> Thanks,
>> Sebastian
>>
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From cimrman3 at ntc.zcu.cz  Wed Jul  8 08:43:14 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 08 Jul 2009 14:43:14 +0200
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>
References: <4A538921.9060800@ntc.zcu.cz>
	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>
Message-ID: <4A549462.5060502@ntc.zcu.cz>

Hi Zach,

Zachary Pincus wrote:
> Hi Robert,
> 
> Basically, assuming the object is in a binarized array, you could use  
> ndimage to do one iteration of erosion, giving you the same object but  
> one pixel smaller. Then xor the eroded and original binary images to  
> give an image where the single-pixel border around the object is 1 and  
> the rest is zero; from here you can just sum the pixels to give a  
> (very rough) perimeter value. (Note of course that this doesn't  
> account for the spacing between pixels being different on the diagonal  
> than horizontal or vertical... for that you'd need some chain code  
> things, which I think ndimage doesn't provide.)

Just for the record how I did it (before knowing about your code): I 
have used np.gradient to obtain the edges (yes, the image is binary) and 
counted the circumference like this:

# pixel_sizes = (width, height) of a pixel in some lenght units
grad0, grad1 = np.gradient(mask)
val0 = len(grad0[np.where(grad0)]) * pixel_sizes[0]
val1 = len(grad1[np.where(grad1)]) * pixel_sizes[1]
circumference = 0.5 * (val0 + val1)

A poor man's approach, but worked too.

cheers,
r.


From cimrman3 at ntc.zcu.cz  Wed Jul  8 08:45:23 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 08 Jul 2009 14:45:23 +0200
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <4A53D45A.1050908@bigpond.net.au>
References: <4A538921.9060800@ntc.zcu.cz> <4A53D45A.1050908@bigpond.net.au>
Message-ID: <4A5494E3.1010209@ntc.zcu.cz>

Thank you Gary,

it's good to know the right keyword - my google searches were not 
successful but now I get lots of results :-)

cheers,
r.

Gary Ruben wrote:
> I recently sat in on a lecture on how to do this. In summary, there is 
> no best way to do it - it's an open research problem.
> Search for "Best Line Unbiased Estimators" to find out more.
> If you just walk around the discretised 8-connected circumference and 
> try to compute pi from it, it will converge to 8(sqrt(2)-1), which is 
> close to pi and comes from the sum of sides of an octagon enclosing the 
> circle (this makes sense because all the traversals can be ordered to 
> lie on this octagon), so you need to be smarter, unless that's an 
> acceptable uncertainty for your problem.
> 
> Zach's suggestion to use find_contours is probably a good one, depending 
> on how the contouring is done - certainly sounds like a promising approach,
> 
> Gary R.
> 
> Robert Cimrman wrote:
>> Hi!
>>
>> Hoping that there are some raster image experts around: I need to 
>> compute an approximation of circumference of an object (with smooth 
>> boundary) represented as a filled area in a 2d array. Can scipy.ndimage 
>> deal with that?
>>
>> thanks,
>> r.
> 
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 
> 



From cimrman3 at ntc.zcu.cz  Wed Jul  8 11:35:51 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 08 Jul 2009 17:35:51 +0200
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <4A54935B.4060209@ntc.zcu.cz>
References: <4A538921.9060800@ntc.zcu.cz>	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>	<bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>	<DC866963-E5B1-45AF-98D8-E75ADAA75098@yale.edu>
	<4A54935B.4060209@ntc.zcu.cz>
Message-ID: <4A54BCD7.6020506@ntc.zcu.cz>

Robert Cimrman wrote:
> Hi Zach!
> 
> Zachary Pincus wrote:
>> Hi all,
>>
>> Attached is find_contours, an extension module that finds all contours 
>> in a 2D array at a specified iso-value. The contour position is linearly 
>> interpolated between pixels. Let me know if you have any questions, etc. 
>> (Looking over the code, I'm surprised how well I documented and 
>> commented it... lucky break for anyone who wants to use or modify it, I 
>> guess.) It's GPL because it comes from a larger GPL'd project of mine, 
>> but just ask me and I'll send it under a different license.
> 
> Thank you very much, I will try your code. This is the way I thought I 
> might try (interpolating by a smooth function/polynomial), so it's cool 
> I do not have to dive into it :-)

ok, I have tried find_contours() - it seems very suitable for my purposes!

r.


From zachary.pincus at yale.edu  Wed Jul  8 11:47:32 2009
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Wed, 8 Jul 2009 11:47:32 -0400
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <4A54BCD7.6020506@ntc.zcu.cz>
References: <4A538921.9060800@ntc.zcu.cz>	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>	<bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>	<DC866963-E5B1-45AF-98D8-E75ADAA75098@yale.edu>
	<4A54935B.4060209@ntc.zcu.cz> <4A54BCD7.6020506@ntc.zcu.cz>
Message-ID: <11FF520A-D2FA-4198-97ED-650AEAE570FF@yale.edu>

>>> Attached is find_contours, an extension module that finds all  
>>> contours
>>> in a 2D array at a specified iso-value. The contour position is  
>>> linearly
>>> interpolated between pixels. Let me know if you have any  
>>> questions, etc.
>>> (Looking over the code, I'm surprised how well I documented and
>>> commented it... lucky break for anyone who wants to use or modify  
>>> it, I
>>> guess.) It's GPL because it comes from a larger GPL'd project of  
>>> mine,
>>> but just ask me and I'll send it under a different license.
>>
>> Thank you very much, I will try your code. This is the way I  
>> thought I
>> might try (interpolating by a smooth function/polynomial), so it's  
>> cool
>> I do not have to dive into it :-)
>
> ok, I have tried find_contours() - it seems very suitable for my  
> purposes!

Note of course that if the image starts out as binary, the exact same  
problems that Gary described will apply: the contour segments will all  
point in one of eight directions...

If the image originates as non-binary on the other hand, and instead  
of thresholding at value X, you find the iso-contours at value X, you  
might be able to get better values.

Probably worth doing some ground-truth testing with mockup images. I'd  
be interested in the results.

Zach


From cimrman3 at ntc.zcu.cz  Wed Jul  8 11:53:40 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 08 Jul 2009 17:53:40 +0200
Subject: [SciPy-user] circumference in raster image
In-Reply-To: <11FF520A-D2FA-4198-97ED-650AEAE570FF@yale.edu>
References: <4A538921.9060800@ntc.zcu.cz>	<AB6B17A1-D714-48D5-ACAF-B10D15010989@yale.edu>	<bc657ead0907071322h47ce3b68ucf9c606a312b8350@mail.gmail.com>	<DC866963-E5B1-45AF-98D8-E75ADAA75098@yale.edu>	<4A54935B.4060209@ntc.zcu.cz>
	<4A54BCD7.6020506@ntc.zcu.cz>
	<11FF520A-D2FA-4198-97ED-650AEAE570FF@yale.edu>
Message-ID: <4A54C104.6080109@ntc.zcu.cz>

Zachary Pincus wrote:
>>>> Attached is find_contours, an extension module that finds all  
>>>> contours
>>>> in a 2D array at a specified iso-value. The contour position is  
>>>> linearly
>>>> interpolated between pixels. Let me know if you have any  
>>>> questions, etc.
>>>> (Looking over the code, I'm surprised how well I documented and
>>>> commented it... lucky break for anyone who wants to use or modify  
>>>> it, I
>>>> guess.) It's GPL because it comes from a larger GPL'd project of  
>>>> mine,
>>>> but just ask me and I'll send it under a different license.
>>> Thank you very much, I will try your code. This is the way I  
>>> thought I
>>> might try (interpolating by a smooth function/polynomial), so it's  
>>> cool
>>> I do not have to dive into it :-)
>> ok, I have tried find_contours() - it seems very suitable for my  
>> purposes!
> 
> Note of course that if the image starts out as binary, the exact same  
> problems that Gary described will apply: the contour segments will all  
> point in one of eight directions...

It does starts out as binary, so its the best I can get, IMHO. Anyway, 
it is going to be better than my gradient based approach. I will post 
here the results soon (soon in rc's time = next week).

> If the image originates as non-binary on the other hand, and instead  
> of thresholding at value X, you find the iso-contours at value X, you  
> might be able to get better values.

Maybe some decent blurring by ndimage filters would help, but I guess 
that the accuracy is going to be ok without that.

> Probably worth doing some ground-truth testing with mockup images. I'd  
> be interested in the results.

Basically, all my images are mockup (and binary), I can, however, 
increase their resolution at will, as they correspond to slices of 
objects with an analytical description. I will do some accuracy tests.

r.


From 3ukip0s02 at sneakemail.com  Wed Jul  8 11:36:47 2009
From: 3ukip0s02 at sneakemail.com (-)
Date: Wed, 8 Jul 2009 15:36:47 +0000 (UTC)
Subject: [SciPy-user] FFT-based convolution and correlation
Message-ID: <loom.20090708T151352-930@post.gmane.org>

Is there a FFT-based 2D correlation function built into scipy?  There are
functions like:

scipy.signal.correlate2d - "the direct method implemented by convolveND will be
slow for large data"

scipy.ndimage.correlate - "The array is correlated with the given kernel using
exact calculation (i.e. not FFT)."

scipy.fftpack.convolve.convolve, which I don't really understand

Numarray had a correlate2d() function, with a 'fft=True' switch
(http://structure.usc.edu/numarray/node61.html), but I guess numarray was folded
into numpy, and I can't find if this function was included.





From Dharhas.Pothina at twdb.state.tx.us  Wed Jul  8 14:28:05 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Wed, 08 Jul 2009 13:28:05 -0500
Subject: [SciPy-user] Sort by first n columns in an array
Message-ID: <4A549EE5.63BA.009B.0@twdb.state.tx.us>

Hi,

I am reading in an array with np.genfromtxt and I need to sort it by the first 6 columns (ie year,month etc). I do not know the number of column before reading the array in. 

>From the documentation for sorting it looks like I can use the 'order' keyword to do what I need if I have a structured array with names for each column. I haven't been able to work out how to convert the array I have to an ordered array though. Also as far as I can tell I can't set the dtype with field names in np.genfromtxt without previously knowing the number of columns in the data file.

I don't want to use the timeseries toolkit for this since it is on a machine that hasn't got that package installed.

I know I must be missing something simple. Any help is appreciated.

- dharhas



From oanjao at yahoo.com  Wed Jul  8 14:51:36 2009
From: oanjao at yahoo.com (Craig Finch)
Date: Wed, 8 Jul 2009 11:51:36 -0700 (PDT)
Subject: [SciPy-user] Scipy weave errors: undefined symbols when
	importing compiled extension
Message-ID: <245284.53920.qm@web30405.mail.mud.yahoo.com>


Progress!  I asked on the Intel forums, and they pointed out that I can unmangle the symbol name using "c++filt _ZNSt8ios_base4InitC1Ev".  This corresponds to "std::ios_base::Init::Init()" which is in the C++ standard library.  It appears that the code was not being linked with C++ standard library, and I was able to prevent this error by adding "-lstdc++" to the linker command.  At this point, I working copies of Python, Numpy, and Scipy built entirely with Intel tools.


However, I had to "hard code" this flag into the linker command in the file numpy/distutils/intelccompiler.py.  It just didn't work when I added it to the environment variable LDFLAGS (and yes, I double-checked the environment variable to make sure it was set correctly).  Because of this, I'd say I found a hackish workaround more than a solution.  Any suggestions about the "right" way to do this?

Second problem: Scipy now crashes when I try to run the unit tests.  Below is the partial output when I attempt to run the unit tests.  I'm also going to ask about this on the Intel MKL forums, but I'd appreciate any suggestions from you about how to go about diagnosing this problem.

   Craig

Python 2.5.4 (r254:67916, Jul  6 2009, 19:27:14)
[GCC Intel(R) C++ gcc 4.1 mode] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy; scipy.test()
Running unit tests for scipy
NumPy version 1.3.0
NumPy is installed in /home/cfinch/lib/python2.5/site-packages/numpy
SciPy version 0.7.0
SciPy is installed in /home/cfinch/lib/python2.5/site-packages/scipy
Python version 2.5.4 (r254:67916, Jul  6 2009, 19:27:14) [GCC Intel(R) C++ gcc 4.1 mode]
nose version 0.11.1
...
...........................*** glibc detected *** python: free(): invalid next s
ize (fast): 0x0000000010781a50 ***
======= Backtrace: =========
/lib64/libc.so.6[0x315326e8a0]
/lib64/libc.so.6(cfree+0x8c)[0x3153271fbc]
/home/cfinch/lib/python2.5/site-packages/numpy/core/multiarray.so[0x2aaaaebc6827
] 
/home/cfinch/lib/python2.5/site-packages/scipy/linalg/flapack.so[0x2aaab702b927]
/home/cfinch/lib/python2.5/site-packages/scipy/linalg/flapack.so[0x2aaab703937b]
python(PyObject_Call+0x15)[0x417c65]
python[0x4c9c07]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python[0x4ca369]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python(PyEval_EvalFrameEx+0x8ba)[0x4c233c]
python[0x4caa0f]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python(PyEval_EvalFrameEx+0x8ba)[0x4c233c]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python[0x421bcd]
python(PyObject_Call+0x15)[0x417c65]
python[0x48c28f]
python(PyObject_Call+0x15)[0x417c65]
python[0x4c9c07]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python[0x4caa0f]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python(PyEval_EvalFrameEx+0x8ba)[0x4c233c]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python[0x421bcd]
python(PyObject_Call+0x15)[0x417c65]
python[0x48c28f]
python(PyObject_Call+0x15)[0x417c65]
python[0x4c9c07]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python(PyEval_EvalFrameEx+0x8ba)[0x4c233c]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python[0x421bcd]
python(PyObject_Call+0x15)[0x417c65]
python[0x48c28f]
python(PyObject_Call+0x15)[0x417c65]
python[0x4c9c07]
python(PyEval_EvalFrameEx+0xe1e)[0x4c28a0]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]
python(PyObject_Call+0x15)[0x417c65]
python(PyEval_EvalFrameEx+0x8ba)[0x4c233c]
python(PyEval_EvalCodeEx+0x5a2)[0x4cb7ee]
python[0x532ef5]

======= Memory map: ========
00400000-005e2000 r-xp 00000000 00:12 97412                              /gpfs/fs0/work/cfinch/bin/python
007e2000-0081a000 rwxp 001e2000 00:12 97412                              /gpfs/fs0/work/cfinch/bin/python
0081a000-00822000 rwxp 0081a000 00:00 0
0ede6000-10c04000 rwxp 0ede6000 00:00 0
40000000-40001000 ---p 40000000 00:00 0
40001000-40011000 rwxp 40001000 00:00 0
40011000-40012000 ---p 40011000 00:00 0
40012000-4040a000 rwxp 40012000 00:00 0
4040a000-4040b000 ---p 4040a000 00:00 0
4040b000-40803000 rwxp 4040b000 00:00 0
40803000-40804000 ---p 40803000 00:00 0
40804000-40bfc000 rwxp 40804000 00:00 0
40bfc000-40bfd000 ---p 40bfc000 00:00 0
40bfd000-40ff6000 rwxp 40bfd000 00:00 0
40ff6000-40ff7000 ---p 40ff6000 00:00 0
40ff7000-413f0000 rwxp 40ff7000 00:00 0
413f0000-413f1000 ---p 413f0000 00:00 0
413f1000-417ea000 rwxp 413f1000 00:00 0
417ea000-417eb000 ---p 417ea000 00:00 0
417eb000-41be4000 rwxp 417eb000 00:00 0
3152200000-315221a000 r-xp 00000000 08:03 131368                         /lib64/ld-2.5.so
3152419000-315241a000 r-xp 00019000 08:03 131368                         /lib64/ld-2.5.so
315241a000-315241b000 rwxp 0001a000 08:03 131368                         /lib64/ld-2.5.so
3152600000-3152614000 r-xp 00000000 08:03 24085282                       /usr/lib64/libz.so.1.2
.3
3152614000-3152813000 ---p 00014000 08:03 24085282                       /usr/lib64/libz.so.1.2
.3
3152813000-3152814000 rwxp 00013000 08:03 24085282                       /usr/lib64/libz.so.1.2
.3
3152a00000-3152a02000 r-xp 00000000 08:03 131373                         /lib64/libdl-2.5.so
3152a02000-3152c02000 ---p 00002000 08:03 131373                         /lib64/libdl-2.5.so
3152c02000-3152c03000 r-xp 00002000 08:03 131373                         /lib64/libdl-2.5.so
3152c03000-3152c04000 rwxp 00003000 08:03 131373                         /lib64/libdl-2.5.so
3152e00000-3152e35000 r-xp 00000000 08:03 24084785                       /usr/lib64/libreadline
.so.5.1
3152e35000-3153034000 ---p 00035000 08:03 24084785                       /usr/lib64/libreadline
.so.5.1
3153034000-315303c000 rwxp 00034000 08:03 24084785                       /usr/lib64/libreadline
.so.5.1
315303c000-315303d000 rwxp 315303c000 00:00 0
3153200000-3153344000 r-xp 00000000 08:03 131369                         /lib64/libc-2.5.so
3153344000-3153544000 ---p 00144000 08:03 131369                         /lib64/libc-2.5.so
3153544000-3153548000 r-xp 00144000 08:03 131369                         /lib64/libc-2.5.so

3153548000-3153549000 rwxp 00148000 08:03 131369                         /lib64/libc-2.5.so
3153549000-315354e000 rwxp 3153549000 00:00 0
3153600000-3153682000 r-xp 00000000 08:03 131374                         /lib64/libm-2.5.so
3153682000-3153881000 ---p 00082000 08:03 131374                         /lib64/libm-2.5.so
3153881000-3153882000 r-xp 00081000 08:03 131374                         /lib64/libm-2.5.so
3153882000-3153883000 rwxp 00082000 08:03 131374                         /lib64/libm-2.5.so
3153a00000-3153a15000 r-xp 00000000 08:03 131372                         /lib64/libpthread-2.5.so
3153a15000-3153c14000 ---p 00015000 08:03 131372                         /lib64/libpthread-2.5.so
3153c14000-3153c15000 r-xp 00014000 08:03 131372                         /lib64/libpthread-2.5.so
3153c15000-3153c16000 rwxp 00015000 08:03 131372                         /lib64/libpthread-2.5.so
3153c16000-3153c1a000 rwxp 3153c16000 00:00 0
3156e00000-3156e0d000 r-xp 00000000 08:03 131375                         /lib64/libgcc_s-4.1.1-20070105.so.1
3156e0d000-315700c000 ---p 0000d000 08:03 131375                         /lib64/libgcc_s-4.1.1-20070105.so.1
315700c000-315700d000 rwxp 0000c000 08:03 131375                         /lib64/libgcc_s-4.1.1-20070105.so.1
3157200000-31572e6000 r-xp 00000000 08:03 24085283                       /usr/lib64/libstdc++.so.6.0.8
31572e6000-31574e5000 ---p 000e6000 08:03 24085283                       /usr/lib64/libstdc++.so.6.0.8
31574e5000-31574eb000 r-xp 000e5000 08:03 24085283                       /usr/lib64/libstdc++.so.6.0.8
31574eb000-31574ee000 rwxp 000eb000 08:03 24085283                       /usr/lib64/libstdc++.so.6.0.8
31574ee000-3157500000 rwxp 31574ee000 00:00 0
315c000000-315c125000 r-xp 00000000 08:03 131380                         /lib64/libcrypto.so.0.9.8b
315c125000-315c324000 ---p 00125000 08:03 131380                         /lib64/libcrypto.so.0.9.8b
315c324000-315c343000 rwxp 00124000 08:03 131380                         /lib64/libcrypto.so.0.9.8b
315c343000-315c347000 rwxp 315c343000 00:00 0
315c400000-315c411000 r-xp 00000000 08:03 131378                         /lib64/libresolv-2.5.so
315c411000-315c611000 ---p 00011000 08:03 131378                         /lib64/libresolv-2.5.so
315c611000-315c612000 r-xp 00011000 08:03 131378                         /lib64/libresolv-2.5.so
315c612000-315c613000 rwxp 00012000 08:03 131378                         /lib64/libresolv-2.5.so
315c613000-315c615000 rwxp 315c613000 00:00 0
315d000000-315d002000 r-xp 00000000 08:03 Aborted



----- Original Message ----
From: Craig Finch <oanjao at yahoo.com>
To: David Cournapeau <cournape at gmail.com>; SciPy Users List <scipy-user at scipy.org>
Sent: Monday, July 6, 2009 7:16:42 PM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing  compiled extension

I've been dealing with issues like this, on this one system, for a year now.  At one point, I had a working combination of python, numpy, and scipy.  I tried so many things that I can't really remember how I got to that point.  I think I used gcc to build scipy 0.6 because I couldn't get it to work with icc, and I linked it with the Intel MKL successfully.  However, when I upgraded to scipy-0.7.0 I am again facing this problem.

The latest thing I tried was to build both Numpy and Scipy with the Intel C and Fortran compilers (icc and ifort).  Numpy builds fine and passes all its unit tests.  Scipy builds without errors or warnings, but I still have problems when trying to import any Scipy module that depends on "sparse."  

Build command:

python setup.py config --compiler=intel --fcompiler=intelem install --prefix=/home/cfinch 2>&1 > scipy_build_log.txt

Results:
>>> from scipy import sparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/__init__.py", line 6, in <module>
    from csr import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/csr.py", line 12, in <module>
    from sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/__init__.py", line 4, in <module>
    from csr import *
  File "/home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/csr.py", line 7, in <module>
    import _csr
ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: _ZNSt8ios_base4InitD1Ev

When I use nm to look at the symbols in _csr.so, I see that these are undefined, along with several dozen other Python-related symbols:
...
00000000000d2810 W _ZNSt6vectorIxSaIxEE6resizeEmx
00000000000d2fdc W _ZNSt6vectorIySaIyEE6resizeEmy
                 U _ZNSt8ios_base4InitC1Ev
                 U _ZNSt8ios_base4InitD1Ev

I have no idea what this symbol is, or where it's supposed to come from.  Has anyone seen an error like this before?  I am going to see what kind of support I can get from Intel.

   Craig


----- Original Message ----
From: Craig Finch <oanjao at yahoo.com>
To: David Cournapeau <cournape at gmail.com>; SciPy Users List <scipy-user at scipy.org>
Sent: Sunday, July 5, 2009 2:30:18 PM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing  compiled extension

By "build log" I assume you mean the output to stderr and stdout when when building Scipy.  It's a huge amount of output, and I'm not sure exactly what to look for.  Do you want me to post a particular section?  Below is the output from sections where g++ is called.  It seems to be linked with g++ everywhere.

One more potential problem area: I'm linking with the Intel Math Kernel Library (MKL) which was almost certainly built with Intel compilers.  Could that be causing this problem?

   Craig
  
-----------------------
building 'scipy.interpolate._interpolate' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-Iscipy/interpolate/src -I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/interpolate/src/_interpolate.cpp
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/interpolate/src/_interpolate.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/interpolate/_interpolate.so
-----------------------
building 'scipy.sparse.sparsetools._csr' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

creating build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools
compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/csr_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/csr_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_csr.so
building 'scipy.sparse.sparsetools._csc' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/csc_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/csc_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_csc.so
building 'scipy.sparse.sparsetools._coo' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/coo_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/coo_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_coo.so
building 'scipy.sparse.sparsetools._bsr' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/bsr_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/bsr_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_bsr.so
building 'scipy.sparse.sparsetools._dia' extension
compiling C++ sources
C compiler: g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

compile options: '-I/home/cfinch/lib/python2.5/site-packages/numpy/core/include -I/home/cfinch/include/python2.5 -c'
g++: scipy/sparse/sparsetools/dia_wrap.cxx
g++ -pthread -shared build/temp.linux-x86_64-2.5/scipy/sparse/sparsetools/dia_wrap.o -Lbuild/temp.linux-x86_64-2.5 -o build/lib.linux-x86_64-2.5/scipy/sparse/sparsetools/_dia.so

-----------------------




----- Original Message ----
From: David Cournapeau <cournape at gmail.com>
To: Craig Finch <cfinch at ieee.org>; SciPy Users List <scipy-user at scipy.org>
Sent: Friday, July 3, 2009 10:21:28 PM
Subject: Re: [SciPy-user] Scipy weave errors: undefined symbols when importing  compiled extension

On Sat, Jul 4, 2009 at 3:38 AM, Craig Finch<oanjao at yahoo.com> wrote:

> ImportError: /home/cfinch/lib/python2.5/site-packages/scipy/sparse/sparsetools/_csr.so: undefined symbol: __gxx_personality_v0
>
> As far as I know, everything Python-related on this machine has now been built with GNU tools, but something is still broken.  One more thing--Python, Numpy, and Scipy are all locally installed in my /home/cfinch directory because the system Python is frozen at 2.4.

This is still a C++ problem, I have sometimes noticed this problem on
some machines. Normally this is caused by a C++ object file being
linked with gcc (instead of g++). Can you post the build log of scipy
?

David


      


From dwf at cs.toronto.edu  Wed Jul  8 15:35:36 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Wed, 8 Jul 2009 15:35:36 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <loom.20090708T114047-554@post.gmane.org>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
	<4A537A54.2010601@gmail.com>
	<1BB30B2C-4CE8-44C7-82FF-62E9EFF0FB7C@cs.toronto.edu>
	<loom.20090708T114047-554@post.gmane.org>
Message-ID: <F860412A-61D0-4622-98C4-DC321FB59439@cs.toronto.edu>

On 8-Jul-09, at 7:48 AM, Tim Michelsen wrote:

> Would be nice to see your implementation.

The way I did it was to do the plotting onto a FigureCanvasAgg and  
then printing it to a fake file using cStringIO.

from matplotlib.backends.backend_agg import FigureCanvasAgg
import matplotlib.pyplot as plt
import cStringIO

def plot(...):
	fig = plt.figure(...) # figure out size and such here
	canvas = FigureCanvasAgg(fig)
	fig.set_canvas(canvas) # I forget whether this is needed
	# Do some plotting commands here
	...
	...
	canvas.draw()
	imgdata = cStringIO.StringIO()
	canvas.print_png(imgdata)
	return imgdata.get_value()


There's something very similar in the Matplotlib cookbook or the  
TurboGears cookbook, but I forget which.

David


From stefan at sun.ac.za  Wed Jul  8 16:22:33 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 8 Jul 2009 22:22:33 +0200
Subject: [SciPy-user] FFT-based convolution and correlation
In-Reply-To: <loom.20090708T151352-930@post.gmane.org>
References: <loom.20090708T151352-930@post.gmane.org>
Message-ID: <9457e7c80907081322q7d3ec762m568796948087e9d6@mail.gmail.com>

2009/7/8 - <3ukip0s02 at sneakemail.com>:
> Is there a FFT-based 2D correlation function built into scipy? ?There are
> functions like:

Yes, try scipy.signal.fftconvolve.

St?fan


From trottier+pylist at gmail.com  Wed Jul  8 20:31:42 2009
From: trottier+pylist at gmail.com (Leo Trottier)
Date: Wed, 8 Jul 2009 17:31:42 -0700
Subject: [SciPy-user] Automatically making a dtype
In-Reply-To: <bfcc38f60907081708v29434955y40a74f0edd179967@mail.gmail.com>
References: <bfcc38f60907081708v29434955y40a74f0edd179967@mail.gmail.com>
Message-ID: <bfcc38f60907081731j17213b30h89232c4256f79916@mail.gmail.com>

Hi,

So, perhaps I'm the only one, but I find using numpy dtypes can be a bit
more troublesome than I typically expect from Python libraries. ?So I've gone
ahead and written a little function that, given an "exemplar" (e.g., a
row from your data set) will create a dtype based on it.

Anyone think that something like this should make it into the
numpy/scipy distribution? ?Also, anyone want to improve the function
so it can handle tuples, sub-arrays, etc?

Anyway, here it is:


def makeDType(exemplar):
? ?'''Return a dtype object based on the given list or dict *exemplar*

? ?This is a convenience function -- if you want to do anything sophisticated
? ?it's best to compose the dtype "by hand".

? ?If given a list, this will return a dtype with fields ordered in the same
? ?sequence as in exemplar.

? ?If given a dict, the field ordering will be alphabetical, based on the
? ?names of the fields.

? ?NB: any str example you give it should be the longest you can imagine,
? ?as the function will return a field based on that length.

? ?Usage:
? ?>>> makeDType(['a string', 4, 3.0, 3j, True, None, eval, u'asdf'])
? ?dtype([('f0', '|S8'), ('f1', '<i4'), ('f2', '<f8'), ('f3', '<c16'),
? ?('f4', '|b1'), ('f5', '|O4'), ('f6', '|O4'), ('f7', '<U4')]))

? ?>>> makeDType(dict(a='0123',b=3.,c=4,d=True,e=3j,f=eval,g=None,h=u'asdf'))
? ?dtype([('a', '|S4'), ('b', '<f8'), ('c', '<i4'), ('d', '|b1'), ('e',
? ?'<c16'), ('f', '|O4'), ('g', '|O4'), ('h', '<U4')])

? ?Note:
? ? ? This can't yet handle tuples or sub-arrays, and it's not
? ? ? smart enough to figure out how big to make each float, int, etc.

? ?This code is put into the public domain.
? ?'''
? ?import numpy as np
? ?if type(exemplar) is dict:
? ? ? ?names = exemplar.keys(); names.sort()
? ? ? ?formats = [np.array(exemplar[key]).dtype for key in names]
? ? ? ?return np.dtype({'names':names, 'formats':formats})
? ?else:
? ? ? ?formats = ','.join([np.array(val).dtype.str for val in exemplar])
? ? ? ?return np.dtype(formats)


From pgmdevlist at gmail.com  Wed Jul  8 21:41:20 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 8 Jul 2009 21:41:20 -0400
Subject: [SciPy-user] Automatically making a dtype
In-Reply-To: <bfcc38f60907081731j17213b30h89232c4256f79916@mail.gmail.com>
References: <bfcc38f60907081708v29434955y40a74f0edd179967@mail.gmail.com>
	<bfcc38f60907081731j17213b30h89232c4256f79916@mail.gmail.com>
Message-ID: <6BB8CFC8-17E2-42CD-B6AA-6036AE017CA7@gmail.com>


On Jul 8, 2009, at 8:31 PM, Leo Trottier wrote:

> Hi,
>
> So, perhaps I'm the only one, but I find using numpy dtypes can be a  
> bit
> more troublesome than I typically expect from Python libraries.  So  
> I've gone
> ahead and written a little function that, given an "exemplar" (e.g., a
> row from your data set) will create a dtype based on it.

Check np.genfromtxt, the portion of the code dealing w/ dtype is None  
(where the dtype is guessed from the data...)


> Anyone think that something like this should make it into the
> numpy/scipy distribution?  Also, anyone want to improve the function
> so it can handle tuples, sub-arrays, etc?

Well, what are you up to these days ;) ?


From dwf at cs.toronto.edu  Wed Jul  8 22:37:13 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Wed, 8 Jul 2009 22:37:13 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <loom.20090708T114047-554@post.gmane.org>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
	<4A537A54.2010601@gmail.com>
	<1BB30B2C-4CE8-44C7-82FF-62E9EFF0FB7C@cs.toronto.edu>
	<loom.20090708T114047-554@post.gmane.org>
Message-ID: <AF8BF724-5167-4EC6-B2FD-492C8E66CCC7@cs.toronto.edu>

On 8-Jul-09, at 7:48 AM, Tim Michelsen wrote:

> @Bruce, David
> Do you have  a particular site that is one-line and
> available to the gereral public?
> I am intending something similar.

http://www.zunzun.com uses matplotlib extensively.

My own frankenstein creation has been superseded by a J2EE version  
designed by a whole team of programmers, unfortunately. :)

David


From neilcrighton at gmail.com  Wed Jul  8 19:14:36 2009
From: neilcrighton at gmail.com (Neil Crighton)
Date: Wed, 8 Jul 2009 23:14:36 +0000 (UTC)
Subject: [SciPy-user] Sort by first n columns in an array
References: <4A549EE5.63BA.009B.0@twdb.state.tx.us>
Message-ID: <loom.20090708T230307-547@post.gmane.org>

Dharhas Pothina <Dharhas.Pothina <at> twdb.state.tx.us> writes:

> I am reading in an array with np.genfromtxt and I need to sort it by the first
6 columns (ie year,month etc). I
> do not know the number of column before reading the array in. 
> 
> >From the documentation for sorting it looks like I can use the 'order'
keyword to do what I need if I have a
> structured array with names for each column. I haven't been able to work out
how to convert the array I have
> to an ordered array though. Also as far as I can tell I can't set the dtype
with field names in np.genfromtxt
> without previously knowing the number of columns in the data file.
> 
> I don't want to use the timeseries toolkit for this since it is on a machine
that hasn't got that package installed.
> 
> I know I must be missing something simple. Any help is appreciated.
> 
> - dharhas
> 

Something like this should work:

data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2),
                     names='year,month,day')
data.sort(order=['year', 'month', 'day'])


Neil



From Dharhas.Pothina at twdb.state.tx.us  Thu Jul  9 11:21:06 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Thu, 09 Jul 2009 10:21:06 -0500
Subject: [SciPy-user] Sort by first n columns in an array
In-Reply-To: <loom.20090708T230307-547@post.gmane.org>
References: <4A549EE5.63BA.009B.0@twdb.state.tx.us>
	<loom.20090708T230307-547@post.gmane.org>
Message-ID: <4A55C491.63BA.009B.0@twdb.state.tx.us>


>>> Neil Crighton <neilcrighton at gmail.com> 7/8/2009 6:14 PM >>>
>Something like this should work:
>
>data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2),
>                    names='year,month,day')
>data.sort(order=['year', 'month', 'day'])

That works if I know the number of columns prior to reading the file. In this case I'm reading a file with an unknown number of columns. I know that the first six columns are the date fields.

thanks

- dharhas



From hnry2k at hotmail.com  Thu Jul  9 12:51:53 2009
From: hnry2k at hotmail.com (=?iso-8859-1?Q?Jorge_E._=B4Sanchez_Sanchez?=)
Date: Thu, 9 Jul 2009 11:51:53 -0500
Subject: [SciPy-user] Is there any easy way to calculate derivatives from
 ndimage.spline_filter coeffs?
Message-ID: <COL104-W12CD28CEC77D49BC909A339B260@phx.gbl>



Hi all,

 I have interpolated a surface function with x, y, fvals using:

coeffs = ndimage.spline_filter(fvals)
newf = ndimage.map_coordinates(coeffs, coords, prefilter=False)

and I also need the x and y partial derivatives of fvals at the new interpolation points in coords. I suppose that there should be
some command similar to bisplev (as in the next code lines) which allows to calculate them, and I have been looking for it 
unsuccsesfully, so I would appreciate it very much if somebody could help me with this.

#########################################################################
tck =  interpolate.bisplrep(xn,yn,fvalsn)

newff= interpolate.bisplev(newx[:,0],newy[0,:],tck)
#THE PARTIAL DERIVATIVE RESPECT x
newfdx = interpolate.bisplev(newx[:,0],newy[0,:],tck,dx=1,dy=0)

#THE PARTIAL DERIVATIVE RESPECT y
newfdy = interpolate.bisplev(newx[:,0],newy[0,:],tck,dx=0,dy=1)

#########################################################################

I thank you in advance for your kind help
jorge



_________________________________________________________________
Actualiza tu Perfil y gana!
http://www.actualizatuperfil.com.mx/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090709/fe375c70/attachment.html>

From kpdere at verizon.net  Thu Jul  9 18:51:45 2009
From: kpdere at verizon.net (Ken Dere)
Date: Thu, 09 Jul 2009 18:51:45 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <h35sa1$93c$1@ger.gmane.org>

Vincent van Beveren wrote:

> Hello everyone,
> 
> I'm an engineer at Rijnhuizen, which is a research institute for plasma
> physics. We use Python to drive one of our main research projects,
> however, in the scientific circles in this institute Python (and SciPy)
> are largely unknown. Time to change this, I think :). However, since I am
> an engineer and not a scientist I my viewpoint on Python and SciPy are
> more of an engineering perspective, like its Open Source, Free, modern
> language, functional programming, etc... I'm not entirely sure these are
> compelling arguments for a scientist to start working with Python (or
> atleast not on it self). So I was wondering, if I was to promote Python in
> the scientific community here at Rijnhuizen. So I have a few questions:
> 
> -          In what aspects does SciPy excel, compared to say IDL or
> matlab?
> -          In what ways allows it a scientist to be more effective?
> -          How usable is SciPy for Plasma physics, molucular dynamics and
> nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
> -          How stable is it compared to other packages. (bugs,
> computation)?
> 
> Any comments and insights are welcome!
> 
> Regards,
> Vincent
> 
> PS. Posted in scipy-dev by accident.
> ___
> Ing. V. van Beveren
> Software Engineer, FOM Rijnhuizen
> E: V.vanBeveren at rijnhuizen.nl<mailto:V.vanBeveren at rijnhuizen.nl>

The real question is "why is Python better than Matlab or IDL?"

I don't have an opinion on Matlab since I never used it.  However, I used
IDL professionally for about 20 years or so (just for reference, I am a
solar physicist).

A few years ago, having gotten some proficiency in Python, I pretty much
dropped my usage of IDL and now use Python.  This means python, ipython,
numpy, scipy, matplotlib and whatever other modules I need.  One basic
difference is that I can use it at home, on the road and at work without
need of an expensive license.  Also, I enjoy using it.

I have seem some object-oriented implementations is IDL and they are
pathetic.  It may just be the way it was done but I don't know.  The use of
objects in Python is very straightforward.

Ken Dere

-- 
K. Dere



From erik.tollerud at gmail.com  Thu Jul  9 21:44:09 2009
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Thu, 9 Jul 2009 18:44:09 -0700
Subject: [SciPy-user] Problems with the interpolation package
In-Reply-To: <44cd5a960907080309t481ba009l36fa53d2dee3806e@mail.gmail.com>
References: <44cd5a960907080309t481ba009l36fa53d2dee3806e@mail.gmail.com>
Message-ID: <fbab3e0d0907091844t117f9f7ah9b146405e9673b52@mail.gmail.com>

I'm not sure I fully understand your question, but I think
scipy.interpolate.InterpolatedUnivariateSpline is what you want - fit
your spline with that, then if you want to get the properties of the
spline at some subset of points, just use those values.

Specifically, suppose you have your data as x and y vectors.  Do
s=scipy.interpolate.InterpolatedUnivariateSpline(x,y)

This will subdivide the curve into some number of knots (n-k+1, to be
exact), so get the knots with
ks = s.get_knots()

now, if you want the polynomial, bewteen, say the 1st and 2nd knot, do
v = (ks[1]-ks[0])/2
derivs = s.derivatives(v)

and derivs will be an array of derivatives at that point, and that
gives you the coefficients you want (if I am interpreting your
question correctly).

On Wed, Jul 8, 2009 at 3:09 AM, J?n Egilsson<jongisli at gmail.com> wrote:
> Hi there.
>
> I've now tried various functions in the scipy.interpolate package without
> getting far.
>
> What I want to do:
>
> I want to interpolate using spline interpolation, cubic spline interpolation
> to be more precise.
>
> 1. I want to be able to choose which points to interpolate (to choose
> between which points on the x-axis the cubic polynomials are). Then I'll
> have some polynomials on the form ax^3 + bx^2 + cx + d
>
> 2. Then I want to be able to extract the coefficients a,b,c,d from each
> polynomial because that's the only thing I want to keep. I don't want to
> keep an interpolation object of some sort.
>
> Input:
>
> Two lists, one representing the x-axis: [0..n] and another one with the
> function values: [y0, y1, y2, ... , yn] and the desired points I wish to
> interpolate between.
>
> Output (preferably):
>
> The coefficients (a,b,c,d) for every polynomial beetween the points I chose
> to interpolate between.
>
> Any help at all will be greatly appreciated. Cheers.
>
> J?n G?sli Egilsson.
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


From V.vanBeveren at rijnhuizen.nl  Fri Jul 10 03:20:44 2009
From: V.vanBeveren at rijnhuizen.nl (Vincent van Beveren)
Date: Fri, 10 Jul 2009 09:20:44 +0200
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
Message-ID: <2926F4BC94217A43A2D21792DE8818931C5B245B2A@ex1.rijnh.nl>

Thanks everyone for your comments. Because of the many responses I can't reply to everyone personally, but thank you for your input. I have summarised your comments into a list, if you have any corrections, or additions, I would like to hear:

-          Completely Free: No ridiculous licenses or restrictions
-          Open source:
o        If there is a bug you can fix it yourself, or ask the community
o        You can always go back to the source
-          Stable:
o        Originally written by commercial company, before released opensource,
o        The base of EPD (Enthought Python Distribution)
-          Large user community:
o        well supported
o        actively developed
-          Easy to build arbitrary complex applications (IDL and Matlab make this more difficult)
-          As a language it:
o        has many modules out of the box
o        is very readable, and modular
o        easy parallel programming
o        well featured:
?         has support for many data structures (Lists, Tuples, Sets, Dicts)
?         Object Orientated (OO in IDL is a pain, Matlab: what objects?)
?         has compact array manipulation notation, functional programming and support for complex numbers
-          Cross-platform (So is IDL? I don't know about Matlab)
-          Integration with many different things into a unified interface
o        has many additional modules which can be installed
?         HDF5, Numpy, SciPy, Matplotlib (2D plotting), VTK (3D graphics)
o        Easy to interface with other languages (Fortran, C, R)
-          Complete applications for numerical computation build on SciPy freely available:
o        SAGE
o        Python XY

Thanks so far, you all have been really helpful.

Regards,
Vincent

________________________________
From: scipy-user-bounces at scipy.org [mailto:scipy-user-bounces at scipy.org] On Behalf Of Vincent van Beveren
Sent: dinsdag 7 juli 2009 16:03
To: 'scipy-user at scipy.org'
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)

Hello everyone,

I'm an engineer at Rijnhuizen, which is a research institute for plasma physics. We use Python to drive one of our main research projects, however, in the scientific circles in this institute Python (and SciPy) are largely unknown. Time to change this, I think :). However, since I am an engineer and not a scientist I my viewpoint on Python and SciPy are more of an engineering perspective, like its Open Source, Free, modern language, functional programming, etc... I'm not entirely sure these are compelling arguments for a scientist to start working with Python (or atleast not on it self). So I was wondering, if I was to promote Python in the scientific community here at Rijnhuizen. So I have a few questions:

-       In what aspects does SciPy excel, compared to say IDL or matlab?
-       In what ways allows it a scientist to be more effective?
-       How usable is SciPy for Plasma physics, molucular dynamics and nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
-       How stable is it compared to other packages. (bugs, computation)?

Any comments and insights are welcome!

Regards,
Vincent

PS. Posted in scipy-dev by accident.
___
Ing. V. van Beveren
Software Engineer, FOM Rijnhuizen
E: V.vanBeveren at rijnhuizen.nl<mailto:V.vanBeveren at rijnhuizen.nl>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090710/d1d7b974/attachment.html>

From david at ar.media.kyoto-u.ac.jp  Fri Jul 10 03:11:46 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 10 Jul 2009 16:11:46 +0900
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B2A@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<2926F4BC94217A43A2D21792DE8818931C5B245B2A@ex1.rijnh.nl>
Message-ID: <4A56E9B2.1060505@ar.media.kyoto-u.ac.jp>

Vincent van Beveren wrote:
>
> Thanks everyone for your comments. Because of the many responses I
> can?t reply to everyone personally, but thank you for your input. I
> have summarised your comments into a list, if you have any
> corrections, or additions, I would like to hear:
>
>  
>
> -          Completely Free: No ridiculous licenses or restrictions
>
> -          Open source:
>
> o        If there is a bug you can fix it yourself, or ask the community
>
> o        You can always go back to the source
>
> -          Stable:
>
> o        Originally written by commercial company, before released
> opensource,
>

It depends on which parts of the stack you are talking about. AFAIK, the
original design for numpy was done ~15 years by Jim Hugunin, and scipy
was born as an open source project from the start. The "official"
history can be found here for scipy:

http://www.scipy.org/History_of_SciPy

David


From neilcrighton at gmail.com  Fri Jul 10 07:26:20 2009
From: neilcrighton at gmail.com (Neil Crighton)
Date: Fri, 10 Jul 2009 11:26:20 +0000 (UTC)
Subject: [SciPy-user] Sort by first n columns in an array
References: <4A549EE5.63BA.009B.0@twdb.state.tx.us>
	<loom.20090708T230307-547@post.gmane.org>
	<4A55C491.63BA.009B.0@twdb.state.tx.us>
Message-ID: <loom.20090710T112055-235@post.gmane.org>

Dharhas Pothina <Dharhas.Pothina <at> twdb.state.tx.us> writes:

> >>> Neil Crighton <neilcrighton <at> gmail.com> 7/8/2009 6:14 PM >>>
> >Something like this should work:
> >
> >data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2),
> >                    names='year,month,day')
> >data.sort(order=['year', 'month', 'day'])
> 
> That works if I know the number of columns prior to reading the file. In this
case I'm reading a file with an
> unknown number of columns. I know that the first six columns are the date 
fields.
> 

If you know the first six columns are the date fields, can't you use

data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2,3,4,5),
                     names='year,month,day,hour,min,sec')

?

That doesn't make any assumption about the number of columns (as long as 
there's at least six of them).


Neil





From Dharhas.Pothina at twdb.state.tx.us  Fri Jul 10 08:26:11 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Fri, 10 Jul 2009 07:26:11 -0500
Subject: [SciPy-user] Sort by first n columns in an array
In-Reply-To: <loom.20090710T112055-235@post.gmane.org>
References: <4A549EE5.63BA.009B.0@twdb.state.tx.us>
	<loom.20090708T230307-547@post.gmane.org>
	<4A55C491.63BA.009B.0@twdb.state.tx.us>
	<loom.20090710T112055-235@post.gmane.org>
Message-ID: <4A56ED12.63BA.009B.0@twdb.state.tx.us>


Yes, but I don't need just the first six columns sorted, I need the *entire* dataset (ie all columns) sorted by date.

- dharhas

>>> Neil Crighton <neilcrighton at gmail.com> 7/10/2009 6:26 AM >>>
Dharhas Pothina <Dharhas.Pothina <at> twdb.state.tx.us> writes:

> >>> Neil Crighton <neilcrighton <at> gmail.com> 7/8/2009 6:14 PM >>>
> >Something like this should work:
> >
> >data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2),
> >                    names='year,month,day')
> >data.sort(order=['year', 'month', 'day'])
> 
> That works if I know the number of columns prior to reading the file. In this
case I'm reading a file with an
> unknown number of columns. I know that the first six columns are the date 
fields.
> 

If you know the first six columns are the date fields, can't you use

data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2,3,4,5),
                     names='year,month,day,hour,min,sec')

?

That doesn't make any assumption about the number of columns (as long as 
there's at least six of them).


Neil



_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org 
http://mail.scipy.org/mailman/listinfo/scipy-user



From neilcrighton at gmail.com  Fri Jul 10 09:45:58 2009
From: neilcrighton at gmail.com (Neil Crighton)
Date: Fri, 10 Jul 2009 13:45:58 +0000 (UTC)
Subject: [SciPy-user] Sort by first n columns in an array
References: <4A549EE5.63BA.009B.0@twdb.state.tx.us>
	<loom.20090708T230307-547@post.gmane.org>
	<4A55C491.63BA.009B.0@twdb.state.tx.us>
	<loom.20090710T112055-235@post.gmane.org>
	<4A56ED12.63BA.009B.0@twdb.state.tx.us>
Message-ID: <loom.20090710T131833-711@post.gmane.org>

Dharhas Pothina <Dharhas.Pothina <at> twdb.state.tx.us> writes:

> Yes, but I don't need just the first six columns sorted, I need the *entire* d
ataset (ie all columns) sorted
> by date.

I see. You could use argsort instead of sort to get the indices that sort the
array.

data = np.genfromtxt(filename, dtype=None, usecols=(0,1,2,3,4,5),
                     names='year,month,day,hour,min,sec')
ind = data.argsort(order='year month day hour min sec'.split())
alldata = np.genfromtxt(filename, dtype=None)
sorted_alldata = alldata[ind]

But it's probably better to let genfromtxt choose its own field names and then
sort the whole thing using the first 6 fields.

alldata = np.genfromtxt(filename, dtype=None)
alldata.sort(order='f0 f1 f2 f3 f4 f5'.split())


Neil




From lou_boog2000 at yahoo.com  Fri Jul 10 09:47:03 2009
From: lou_boog2000 at yahoo.com (Lou Pecora)
Date: Fri, 10 Jul 2009 06:47:03 -0700 (PDT)
Subject: [SciPy-user] How good, stable,
	and usable is the Enthought Distribution?
Message-ID: <729229.11467.qm@web34402.mail.mud.yahoo.com>

I am thinking of buying an Enthought Package through my laboratory. ?I am familiar with some of the modules available in the package since I've used Python for 5 years in computational physics. ?But I'm wondering what others think of the Enthought package. ?I would be very interested in hearing from those with some experience with the package and the company.
Thanks for any information.

-- Lou Pecora,   my views are my own.


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090710/e6b59a1b/attachment.html>

From bblais at bryant.edu  Fri Jul 10 16:23:36 2009
From: bblais at bryant.edu (Brian Blais)
Date: Fri, 10 Jul 2009 16:23:36 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <2926F4BC94217A43A2D21792DE8818931C5B245B2A@ex1.rijnh.nl>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<2926F4BC94217A43A2D21792DE8818931C5B245B2A@ex1.rijnh.nl>
Message-ID: <D6931C3F-797A-4149-A897-F742223C63B7@bryant.edu>

On Jul 10, 2009, at 3:20 , Vincent van Beveren wrote:

> Thanks everyone for your comments. Because of the many responses I  
> can?t reply to everyone personally, but thank you for your input. I  
> have summarised your comments into a list, if you have any  
> corrections, or additions, I would like to hear:
>
> -          Completely Free: No ridiculous licenses or restrictions
> -          Open source:
> o        If there is a bug you can fix it yourself, or ask the  
> community
> o        You can always go back to the source
> -          Stable:
> o        Originally written by commercial company, before released  
> opensource,
> o        The base of EPD (Enthought Python Distribution)
> -          Large user community:
> o        well supported
> o        actively developed
> -          Easy to build arbitrary complex applications (IDL and  
> Matlab make this more difficult)
> -          As a language it:
> o        has many modules out of the box
> o        is very readable, and modular
> o        easy parallel programming
> o        well featured:
> ?         has support for many data structures (Lists, Tuples,  
> Sets, Dicts)
> ?         Object Orientated (OO in IDL is a pain, Matlab: what  
> objects?)

actually, Matlab has had objects for some time, but it feels very  
tacked-on.

> ?         has compact array manipulation notation, functional  
> programming and support for complex numbers

well, so do both IDL and Matlab.

> -          Cross-platform (So is IDL? I don?t know about Matlab)

Matlab is cross platform.

> -          Integration with many different things into a unified  
> interface
> o        has many additional modules which can be installed
> ?         HDF5, Numpy, SciPy, Matplotlib (2D plotting), VTK (3D  
> graphics)

much of the functionality of this is already in Matlab (can't speak  
for IDL).  That's actually one of the pains of Python, that you have  
to install a whole lot of 3rd party packages where Matlab, it is all  
there.  The Enthought Python Distribution, and Python XY go a long  
way to solve that, although 3d plotting seems easier in Matlab.


> o        Easy to interface with other languages (Fortran, C, R)
> -          Complete applications for numerical computation build on  
> SciPy freely available:
> o        SAGE
> o        Python XY

I converted from Matlab to Python a few years ago.  Much of it had to  
do with price and convenience (one example of Matlab craziness is the  
license manager, which always went berserk on me and kicked me out of  
my legitimate copy of Matlab).  However, once I switched, there was  
another advantage that hadn't occurred to me.  I could replace all of  
my shell scripts, perl scripts, etc... with python too.  That way, I  
got a lot more fluid with the language with non-scientific apps.  I  
could make a web-based gradebook, and easily manipulate directories  
of images files.  I could parse html files (BeautifulSoup), and many  
other things.  It opened up many more productive applications of the  
language than I had anticipated.

For scientific work, cython is an unbeatable tool for optimizing slow  
pieces of your code.  It's like Matlab cmex files, but infinitely  
more convenient and powerful.  I haven't run Matlab for over a year  
now, and I haven't really missed it.  Off my website, I have a  
presentation for Matlab users wanting to switch to python.  It might  
be useful for you.


				bb

-- 
Brian Blais
bblais at bryant.edu
http://web.bryant.edu/~bblais



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090710/91028701/attachment.html>

From xavier.gnata at gmail.com  Fri Jul 10 18:51:47 2009
From: xavier.gnata at gmail.com (Xavier Gnata)
Date: Sat, 11 Jul 2009 00:51:47 +0200
Subject: [SciPy-user] How good, stable,
	and usable is the Enthought Distribution?
In-Reply-To: <729229.11467.qm@web34402.mail.mud.yahoo.com>
References: <729229.11467.qm@web34402.mail.mud.yahoo.com>
Message-ID: <4A57C603.30103@gmail.com>

Have a look at pythonxy.

http://www.pythonxy.com/foreword.php

Xavier

> I am thinking of buying an Enthought Package through my laboratory.  I 
> am familiar with some of the modules available in the package since 
> I've used Python for 5 years in computational physics.  But I'm 
> wondering what others think of the Enthought package.  I would be very 
> interested in hearing from those with some experience with the package 
> and the company.
>
> Thanks for any information.
>
> -- Lou Pecora, my views are my own.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   



From gael.varoquaux at normalesup.org  Fri Jul 10 18:54:42 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 11 Jul 2009 00:54:42 +0200
Subject: [SciPy-user] How good, stable,
	and usable is the Enthought	Distribution?
In-Reply-To: <729229.11467.qm@web34402.mail.mud.yahoo.com>
References: <729229.11467.qm@web34402.mail.mud.yahoo.com>
Message-ID: <20090710225442.GF31151@phare.normalesup.org>

On Fri, Jul 10, 2009 at 06:47:03AM -0700, Lou Pecora wrote:
>    I am thinking of buying an Enthought Package through my laboratory.  I am
>    familiar with some of the modules available in the package since I've used
>    Python for 5 years in computational physics.  But I'm wondering what
>    others think of the Enthought package.  I would be very interested in
>    hearing from those with some experience with the package and the company.
>    Thanks for any information.

I think you should simply try it out. You are allowed by the license to
do this for one month.

I will not comment on the rest, as I am biased (positively), having
worked at Enthought.

Ga?l


From millman at berkeley.edu  Sat Jul 11 04:32:21 2009
From: millman at berkeley.edu (Jarrod Millman)
Date: Sat, 11 Jul 2009 01:32:21 -0700
Subject: [SciPy-user] ANN: SciPy 2009 early registration extended to July
	17th
Message-ID: <c7009a550907110132n4f6de349q1c75d4068390a74f@mail.gmail.com>

The early registration deadline for SciPy 2009 has been extended
for one week to July 17, 2009.  Please register (
http://conference.scipy.org/to_register )
by this date to take advantage of the reduced early registration rate.

About the conference
--------------------

SciPy 2009, the 8th Python in Science conference, will be held from
August 18-23, 2009 at Caltech in Pasadena, CA, USA.  The conference
starts with two days of tutorials to the scientific Python tools.
There will be two tracks, one for introduction of the basic tools to
beginners, and one for more advanced tools.  The tutorials will be
followed by two days of talks.  Both days of talks will begin with a
keynote address.  The first day?s keynote will be given by Peter
Norvig, the Director of Research at Google; while, the second keynote
will be delivered by Jon Guyer, a Materials Scientist in the
Thermodynamics and Kinetics Group at NIST.  The program committee will
select the remaining talks from submissions to our call for papers.
All selected talks will be included in our conference proceedings
edited by the program committee.  After the talks each day we will
provide several rooms for impromptu birds of a feather discussions.
Finally, the last two days of the conference will be used for a number
of coding sprints on the major software projects in our community.

For the 8th consecutive year, the conference will bring together the
developers and users of the open source software stack for scientific
computing with Python.  Attendees have the opportunity to review the
available tools and how they apply to specific problems.  By providing
a forum for developers to share their Python expertise with the wider
commercial, academic, and research communities, this conference
fosters collaboration and facilitates the sharing of software
components, techniques, and a vision for high level language use in
scientific computing.

For further information, please visit the conference homepage:
http://conference.scipy.org.

Important Dates
---------------

* Friday, July 3: Abstracts Due
* Wednesday, July 15: Announce accepted talks, post schedule
* Friday, July 17: Early Registration ends
* Tuesday-Wednesday, August 18-19: Tutorials
* Thursday-Friday, August 20-21: Conference
* Saturday-Sunday, August 22-23: Sprints
* Friday, September 4: Papers for proceedings due

Executive Committee
-------------------

* Jarrod Millman, UC Berkeley, USA (Conference Chair)
* Ga?l Varoquaux, INRIA Saclay, France (Program Co-Chair)
* St?fan van der Walt, University of Stellenbosch, South Africa
(Program Co-Chair)
* Fernando P?rez, UC Berkeley, USA (Tutorial Chair)


From jdh2358 at gmail.com  Sat Jul 11 07:41:56 2009
From: jdh2358 at gmail.com (John Hunter)
Date: Sat, 11 Jul 2009 06:41:56 -0500
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <F860412A-61D0-4622-98C4-DC321FB59439@cs.toronto.edu>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
	<4A537A54.2010601@gmail.com>
	<1BB30B2C-4CE8-44C7-82FF-62E9EFF0FB7C@cs.toronto.edu>
	<loom.20090708T114047-554@post.gmane.org>
	<F860412A-61D0-4622-98C4-DC321FB59439@cs.toronto.edu>
Message-ID: <88e473830907110441i5adcfefbr736dec9e20629b0b@mail.gmail.com>

On Wed, Jul 8, 2009 at 2:35 PM, David Warde-Farley<dwf at cs.toronto.edu> wrote:
> On 8-Jul-09, at 7:48 AM, Tim Michelsen wrote:
>
>> Would be nice to see your implementation.
>
> The way I did it was to do the plotting onto a FigureCanvasAgg and
> then printing it to a fake file using cStringIO.
>
> from matplotlib.backends.backend_agg import FigureCanvasAgg
> import matplotlib.pyplot as plt
> import

Apologies since this is OT for scipy, so please followup to
matplolib-users if you have further questions or comments, but there
is a slight problem with the example above.  If you are using mpl in a
web server, and want explicit control of the canvas, you can either
use pyplot and the canvas like so::

  import matplotlib.use('Agg')
  import matplotlib.pyplot as plt
  fig = plt.figure()
  fig.canvas  # use thie canvas (it is FIgureCanvasAgg) rather than
creating your own

Or you can use the API and manually create your own figure and canvas
(this is recommended since you have maximum control over object
management)::

  from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
  from matplotlib.figure import Figure
  fig = Figure()
  canvas = FigureCanvas(fig)
  ax = fig.add_subplot(111)

The code you posted is a hybrid of using pyplot and managing your own
Canvas creation, which can work under certain configurations but is
brittle and not supported.  This is mainly because pyplot creates and
manages its own canvas instances.

JDH


From lou_boog2000 at yahoo.com  Sat Jul 11 08:21:37 2009
From: lou_boog2000 at yahoo.com (Lou Pecora)
Date: Sat, 11 Jul 2009 05:21:37 -0700 (PDT)
Subject: [SciPy-user] How good, stable,
	and usable is the Enthought Distribution?
Message-ID: <914921.10145.qm@web34408.mail.mud.yahoo.com>

This appears to be a Window only package. ?I didn't make it clear that I have Mac OS X. ?
And I really want information on the Enthought distribution.
Thank you anyway.
-- Lou Pecora,   my views are my own.

--- On Fri, 7/10/09, Xavier Gnata <xavier.gnata at gmail.com> wrote:

From: Xavier Gnata <xavier.gnata at gmail.com>
Subject: Re: [SciPy-user] How good, stable, and usable is the Enthought Distribution?
To: "SciPy Users List" <scipy-user at scipy.org>
Date: Friday, July 10, 2009, 6:51 PM

Have a look at pythonxy.

http://www.pythonxy.com/foreword.php

Xavier





      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090711/c3bc7cdf/attachment.html>

From lou_boog2000 at yahoo.com  Sat Jul 11 08:24:09 2009
From: lou_boog2000 at yahoo.com (Lou Pecora)
Date: Sat, 11 Jul 2009 05:24:09 -0700 (PDT)
Subject: [SciPy-user] How good, stable,
	and usable is the Enthought Distribution?
Message-ID: <785491.66970.qm@web34405.mail.mud.yahoo.com>

Good idea. ?I'll do that. ?
One question. ?If I install it, does it stay separate from my present python installation? ?I don't want to mess with that.
Thanks.
-- Lou P.
--- On Fri, 7/10/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

From: Gael Varoquaux <gael.varoquaux at normalesup.org>
Subject: Re: [SciPy-user] How good, stable, and usable is the Enthought Distribution?
To: "SciPy Users List" <scipy-user at scipy.org>
Date: Friday, July 10, 2009, 6:54 PM

On Fri, Jul 10, 2009 at 06:47:03AM -0700, Lou Pecora wrote:
>? ? I am thinking of buying an Enthought Package through my laboratory.? I am
[cut]
I think you should simply try it out. You are allowed by the license to
do this for one month.



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090711/14fd647f/attachment.html>

From cohen at lpta.in2p3.fr  Sat Jul 11 09:13:20 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Sat, 11 Jul 2009 15:13:20 +0200
Subject: [SciPy-user] How good, stable,
	and usable is the Enthought Distribution?
In-Reply-To: <785491.66970.qm@web34405.mail.mud.yahoo.com>
References: <785491.66970.qm@web34405.mail.mud.yahoo.com>
Message-ID: <4A588FF0.9050601@lpta.in2p3.fr>

yes, it does, if you wish so
Johann

Lou Pecora wrote:
> Good idea.  I'll do that.  
>
> One question.  If I install it, does it stay separate from my present 
> python installation?  I don't want to mess with that.
>
> Thanks.
>
> -- Lou P.
>
> --- On *Fri, 7/10/09, Gael Varoquaux 
> /<gael.varoquaux at normalesup.org>/* wrote:
>
>
>     From: Gael Varoquaux <gael.varoquaux at normalesup.org>
>     Subject: Re: [SciPy-user] How good, stable, and usable is the
>     Enthought Distribution?
>     To: "SciPy Users List" <scipy-user at scipy.org>
>     Date: Friday, July 10, 2009, 6:54 PM
>
>     On Fri, Jul 10, 2009 at 06:47:03AM -0700, Lou Pecora wrote:
>     >    I am thinking of buying an Enthought Package through my
>     laboratory.  I am
>     [cut]
>
>     I think you should simply try it out. You are allowed by the
>     license to
>     do this for one month.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From xyz.account at arcor.de  Sat Jul 11 09:55:53 2009
From: xyz.account at arcor.de (Martin)
Date: Sat, 11 Jul 2009 13:55:53 +0000 (UTC)
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
Message-ID: <loom.20090711T134245-615@post.gmane.org>

In an "ipython -pylab -p scipy" console I get
In [1]: sqrt(-1)
Out[1]: nan
but I would prefer 1j as the result. This is achieved by
In [2]: from scipy import *
After that I get
In [3]: sqrt(-1)
Out[3]: 1j

I found out that the ipython option -p scipy executes the startup script
Python25\Lib\site-packages\IPython\Extensions\ipy_profile_scipy.py
which contains

    ip = IPython.ipapi.get()

    try:
        ip.ex("import math,cmath")
        ip.ex("import numpy")
        ip.ex("import scipy")

        ip.ex("import numpy as np")
        ip.ex("import scipy as sp")
    
        ip.ex("from numpy import *")
        ip.ex("from scipy import *")
        print('... import * done')

Here the command "from scipy import *" is the last executed command as in my 
interactive session above. Why is this startup script not sufficient to have 
sqrt(-1)=1j right from the console start??
What I should do additionally to achieve this?



From carlos.grohmann at gmail.com  Sat Jul 11 17:01:31 2009
From: carlos.grohmann at gmail.com (=?ISO-8859-1?Q?Carlos_=22Gu=E2no=22_Grohmann?=)
Date: Sat, 11 Jul 2009 18:01:31 -0300
Subject: [SciPy-user] finding values in a list?
Message-ID: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>

Sorry if this is too simple or even OT, but say I have a list like

[1,2,3,4,5,6,7,8,9]

and I want to get the values just below and above a given value.

So, if I enter 6.2, I get 6 and 7.

but how?

TIA

Carlos

-- 
Carlos Henrique Grohmann - Geologist D.Sc.
a.k.a. Guano - Linux User #89721
ResearcherID: A-9030-2008

http://digitalelevation.blogspot.com

http://www.igc.usp.br/pessoais/guano
_________________
Can?t stop the signal.


From alan.mcintyre at gmail.com  Sat Jul 11 17:21:02 2009
From: alan.mcintyre at gmail.com (Alan McIntyre)
Date: Sat, 11 Jul 2009 14:21:02 -0700
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
Message-ID: <1d36917a0907111421h1fd79d51l292d3a288332ebd@mail.gmail.com>

2009/7/11 Carlos "Gu?no" Grohmann <carlos.grohmann at gmail.com>:
> Sorry if this is too simple or even OT, but say I have a list like
>
> [1,2,3,4,5,6,7,8,9]
>
> and I want to get the values just below and above a given value.
>
> So, if I enter 6.2, I get 6 and 7.
>
> but how?

There's probably more efficient ways to do it, but this works:

def get_nearest(a, v):
     '''Get the values in 'a' that bracket 'v'.  Assumes a is a list
already sorted in ascending order.'''
     A = np.array(a)
     return A[A < v][-1], A[A > v][0]

get_nearest([1,2,3,4,5,6,7,8,9], 6.2)
[6, 7]


From alan.mcintyre at gmail.com  Sat Jul 11 17:22:02 2009
From: alan.mcintyre at gmail.com (Alan McIntyre)
Date: Sat, 11 Jul 2009 14:22:02 -0700
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <1d36917a0907111421h1fd79d51l292d3a288332ebd@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
	<1d36917a0907111421h1fd79d51l292d3a288332ebd@mail.gmail.com>
Message-ID: <1d36917a0907111422v495633f3vbd38d814442c2eba@mail.gmail.com>

On Sat, Jul 11, 2009 at 2:21 PM, Alan McIntyre<alan.mcintyre at gmail.com> wrote:
> There's probably more efficient ways to do it, but this works:
>
> def get_nearest(a, v):
> ? ? '''Get the values in 'a' that bracket 'v'. ?Assumes a is a list
> already sorted in ascending order.'''
> ? ? A = np.array(a)
> ? ? return A[A < v][-1], A[A > v][0]
>
> get_nearest([1,2,3,4,5,6,7,8,9], 6.2)
> [6, 7]
>

Sorry, I forgot to include an "import numpy as np" in there. :)


From josef.pktd at gmail.com  Sat Jul 11 17:22:44 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 11 Jul 2009 17:22:44 -0400
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
Message-ID: <1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com>

2009/7/11 Carlos "Gu?no" Grohmann <carlos.grohmann at gmail.com>:
> Sorry if this is too simple or even OT, but say I have a list like
>
> [1,2,3,4,5,6,7,8,9]
>
> and I want to get the values just below and above a given value.
>
> So, if I enter 6.2, I get 6 and 7.
>
> but how?

maybe like this
>>> a = (np.array([1,2,3,4,5,6,7,8,9])-6.1)
>>> a[a>0].min() + 6.2
7.0
>>> a[a<0].max() + 6.2
6.0

Josef

>
> TIA
>
> Carlos
>
> --
> Carlos Henrique Grohmann - Geologist D.Sc.
> a.k.a. Guano - Linux User #89721
> ResearcherID: A-9030-2008
>
> http://digitalelevation.blogspot.com
>
> http://www.igc.usp.br/pessoais/guano
> _________________
> Can?t stop the signal.
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From josef.pktd at gmail.com  Sat Jul 11 17:25:27 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 11 Jul 2009 17:25:27 -0400
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
	<1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com>
Message-ID: <1cd32cbb0907111425v3eb581e3scfcd25473e5309e5@mail.gmail.com>

On Sat, Jul 11, 2009 at 5:22 PM, <josef.pktd at gmail.com> wrote:
> 2009/7/11 Carlos "Gu?no" Grohmann <carlos.grohmann at gmail.com>:
>> Sorry if this is too simple or even OT, but say I have a list like
>>
>> [1,2,3,4,5,6,7,8,9]
>>
>> and I want to get the values just below and above a given value.
>>
>> So, if I enter 6.2, I get 6 and 7.
>>
>> but how?
>
> maybe like this
>>>> a = (np.array([1,2,3,4,5,6,7,8,9])-6.1)
>>>> a[a>0].min() + 6.2
> 7.0
>>>> a[a<0].max() + 6.2
> 6.0

except for the cut and paste error

>>> d = 6.2
>>> a = (np.array([1,7,8,9,2,3,4,5,6]) - d)
>>> a[a>0].min() + d
7.0
>>> a[a<0].max() + d
6.0

>
> Josef
>
>>
>> TIA
>>
>> Carlos
>>
>> --
>> Carlos Henrique Grohmann - Geologist D.Sc.
>> a.k.a. Guano - Linux User #89721
>> ResearcherID: A-9030-2008
>>
>> http://digitalelevation.blogspot.com
>>
>> http://www.igc.usp.br/pessoais/guano
>> _________________
>> Can?t stop the signal.
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>


From jsseabold at gmail.com  Sat Jul 11 17:29:29 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 11 Jul 2009 17:29:29 -0400
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com> 
	<1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com>
Message-ID: <c048da1c0907111429l3ba26c77uf760de3640600a70@mail.gmail.com>

On Sat, Jul 11, 2009 at 5:22 PM, <josef.pktd at gmail.com> wrote:
> 2009/7/11 Carlos "Gu?no" Grohmann <carlos.grohmann at gmail.com>:
>> Sorry if this is too simple or even OT, but say I have a list like
>>
>> [1,2,3,4,5,6,7,8,9]
>>
>> and I want to get the values just below and above a given value.
>>
>> So, if I enter 6.2, I get 6 and 7.
>>
>> but how?
>
> maybe like this
>>>> a = (np.array([1,2,3,4,5,6,7,8,9])-6.1)
>>>> a[a>0].min() + 6.2
> 7.0
>>>> a[a<0].max() + 6.2
> 6.0
>
> Josef
>

Yet another solution (though I too don't know if it's optimal).  This
will work if you know that your values are always going to be within 1
as in your example.

>>> import numpy as np
>>> L = [1,2,3,4,5,6,7,8]
>>> A = np.array(L)
>>> index = np.where(np.abs((A-6.2))<1)
>>> A[index]
array([6, 7])

Skipper


From josef.pktd at gmail.com  Sat Jul 11 17:32:10 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 11 Jul 2009 17:32:10 -0400
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <c048da1c0907111429l3ba26c77uf760de3640600a70@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com>
	<1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com>
	<c048da1c0907111429l3ba26c77uf760de3640600a70@mail.gmail.com>
Message-ID: <1cd32cbb0907111432y57b1fecel63115dbbb1c2e200@mail.gmail.com>

On Sat, Jul 11, 2009 at 5:29 PM, Skipper Seabold<jsseabold at gmail.com> wrote:
> On Sat, Jul 11, 2009 at 5:22 PM, <josef.pktd at gmail.com> wrote:
>> 2009/7/11 Carlos "Gu?no" Grohmann <carlos.grohmann at gmail.com>:
>>> Sorry if this is too simple or even OT, but say I have a list like
>>>
>>> [1,2,3,4,5,6,7,8,9]
>>>
>>> and I want to get the values just below and above a given value.
>>>
>>> So, if I enter 6.2, I get 6 and 7.
>>>
>>> but how?
>>
>> maybe like this
>>>>> a = (np.array([1,2,3,4,5,6,7,8,9])-6.1)
>>>>> a[a>0].min() + 6.2
>> 7.0
>>>>> a[a<0].max() + 6.2
>> 6.0
>>
>> Josef
>>
>
> Yet another solution (though I too don't know if it's optimal). ?This
> will work if you know that your values are always going to be within 1
> as in your example.
>
>>>> import numpy as np
>>>> L = [1,2,3,4,5,6,7,8]
>>>> A = np.array(L)
>>>> index = np.where(np.abs((A-6.2))<1)
>>>> A[index]
> array([6, 7])
>
> Skipper
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>

for sorted lists, this might be fastest

>>> ind = np.searchsorted([1,2,3,4,5,6,7,8,9], 6.2)
>>> ind
6
>>> [1,2,3,4,5,6,7,8,9][ind-1]
6
>>> [1,2,3,4,5,6,7,8,9][ind]

It is unclear what should happen if the values is an element of the list.

Josef


From xavier.gnata at gmail.com  Sat Jul 11 17:56:00 2009
From: xavier.gnata at gmail.com (Xavier Gnata)
Date: Sat, 11 Jul 2009 23:56:00 +0200
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
In-Reply-To: <loom.20090711T134245-615@post.gmane.org>
References: <loom.20090711T134245-615@post.gmane.org>
Message-ID: <4A590A70.1070400@gmail.com>

outch!

numpy.sqrt(-1)
nan

scipy.sqrt(-1)
1j

IMHO, it should behave the same way.
Python provides us with math and cmath.
It is a bit strange from a mathematical point of view but it is 
perfectly valid from a computer science point of view.

Why numpy.sqrt(-1)!=scipy.sqrt(-1) ??
I know it would be hard to change that now...but still.

Xavier


> In an "ipython -pylab -p scipy" console I get
> In [1]: sqrt(-1)
> Out[1]: nan
> but I would prefer 1j as the result. This is achieved by
> In [2]: from scipy import *
> After that I get
> In [3]: sqrt(-1)
> Out[3]: 1j
>
> I found out that the ipython option -p scipy executes the startup script
> Python25\Lib\site-packages\IPython\Extensions\ipy_profile_scipy.py
> which contains
>
>     ip = IPython.ipapi.get()
>
>     try:
>         ip.ex("import math,cmath")
>         ip.ex("import numpy")
>         ip.ex("import scipy")
>
>         ip.ex("import numpy as np")
>         ip.ex("import scipy as sp")
>     
>         ip.ex("from numpy import *")
>         ip.ex("from scipy import *")
>         print('... import * done')
>
> Here the command "from scipy import *" is the last executed command as in my 
> interactive session above. Why is this startup script not sufficient to have 
> sqrt(-1)=1j right from the console start??
> What I should do additionally to achieve this?
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   



From robert.kern at gmail.com  Sun Jul 12 02:01:25 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Jul 2009 01:01:25 -0500
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
In-Reply-To: <4A590A70.1070400@gmail.com>
References: <loom.20090711T134245-615@post.gmane.org>
	<4A590A70.1070400@gmail.com>
Message-ID: <3d375d730907112301x4c6eb1eeife76ae9b62465cd5@mail.gmail.com>

On Sat, Jul 11, 2009 at 16:56, Xavier Gnata<xavier.gnata at gmail.com> wrote:
> outch!
>
> numpy.sqrt(-1)
> nan
>
> scipy.sqrt(-1)
> 1j
>
> IMHO, it should behave the same way.
> Python provides us with math and cmath.
> It is a bit strange from a mathematical point of view but it is
> perfectly valid from a computer science point of view.
>
> Why numpy.sqrt(-1)!=scipy.sqrt(-1) ??
> I know it would be hard to change that now...but still.

As with most such things, the answer is "history". Old Numeric had
only the NaN behavior. scipy added functions with the 1j behavior.
When numpy was formed, it kept the NaN behavior for the functions in
the main numpy namespace and moved the 1j implementations from scipy
into numpy.lib.scimath. scipy kept the 1j behaviors for the functions
it exposes.

Personally, I suggest simply forgetting that the scipy namespace
aliases the numpy functions and only use the subpackages in scipy. Use
the functions from numpy or numpy.lib.scimath directly, as needed.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From xyz.account at arcor.de  Sun Jul 12 12:22:37 2009
From: xyz.account at arcor.de (Martin Kirsch)
Date: Sun, 12 Jul 2009 16:22:37 +0000 (UTC)
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
References: <loom.20090711T134245-615@post.gmane.org>
	<4A590A70.1070400@gmail.com>
	<3d375d730907112301x4c6eb1eeife76ae9b62465cd5@mail.gmail.com>
Message-ID: <loom.20090712T155202-273@post.gmane.org>

Robert Kern <robert.kern <at> gmail.com> writes:

> ... forgetting that the scipy namespace
> aliases the numpy functions and only use the subpackages in scipy. Use
> the functions from numpy or numpy.lib.scimath directly, as needed.
> 

Ok, using only the scipy subpackages helps to decouple the interdependencies
between numpy and scipy (many pitfalls for newcomers).
But why it makes a difference doing "from scipy import *"
(1) per ipy_profile_scipy.py (ipython -pylab -p scipy) => sqrt(-1)= NaN
(2) per console input => sqrt(-1)= 1j
?
Originally I guessed that in both cases sqrt comes from scipy because of the
preceding "from scipy import *" in both cases, but case (1) references
numpy.sqrt despite "from scipy import *" in ipy_profile_scipy.py and I don't
know the reason.





From pav+sp at iki.fi  Sun Jul 12 13:12:25 2009
From: pav+sp at iki.fi (Pauli Virtanen)
Date: Sun, 12 Jul 2009 17:12:25 +0000 (UTC)
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
References: <loom.20090711T134245-615@post.gmane.org>
	<4A590A70.1070400@gmail.com>
	<3d375d730907112301x4c6eb1eeife76ae9b62465cd5@mail.gmail.com>
	<loom.20090712T155202-273@post.gmane.org>
Message-ID: <h3d5hp$r17$1@ger.gmane.org>

On 2009-07-12, Martin Kirsch <xyz.account at arcor.de> wrote:
[clip?]
> But why it makes a difference doing "from scipy import *"
> (1) per ipy_profile_scipy.py (ipython -pylab -p scipy) => sqrt(-1)= NaN
> (2) per console input => sqrt(-1)= 1j
>
> Originally I guessed that in both cases sqrt comes from scipy because of the
> preceding "from scipy import *" in both cases, but case (1) references
> numpy.sqrt despite "from scipy import *" in ipy_profile_scipy.py and I don't
> know the reason.

Perhaps the -pylab switch makes Ipython to run "from numpy import *",
or something similar, *after* ipy_profile_scipy.py? I'd suppose that
if you do "from scipy import *" in the Ipython console, you'd get
Scipy's version of sqrt.

Also, does the order of the -pylab and -p switches on the Ipython 
command line make a difference?

-- 
Pauli Virtanen



From robert.kern at gmail.com  Sun Jul 12 14:10:37 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Jul 2009 13:10:37 -0500
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
In-Reply-To: <h3d5hp$r17$1@ger.gmane.org>
References: <loom.20090711T134245-615@post.gmane.org>
	<4A590A70.1070400@gmail.com> 
	<3d375d730907112301x4c6eb1eeife76ae9b62465cd5@mail.gmail.com> 
	<loom.20090712T155202-273@post.gmane.org> <h3d5hp$r17$1@ger.gmane.org>
Message-ID: <3d375d730907121110h2f9f7382se20e5b4c6672fcc8@mail.gmail.com>

On Sun, Jul 12, 2009 at 12:12, Pauli Virtanen<pav+sp at iki.fi> wrote:
> On 2009-07-12, Martin Kirsch <xyz.account at arcor.de> wrote:
> [clip?]
>> But why it makes a difference doing "from scipy import *"
>> (1) per ipy_profile_scipy.py (ipython -pylab -p scipy) => sqrt(-1)= NaN
>> (2) per console input => sqrt(-1)= 1j

Note that "from scipy import *" is precisely the thing that I am
suggesting that you should never do.

>> Originally I guessed that in both cases sqrt comes from scipy because of the
>> preceding "from scipy import *" in both cases, but case (1) references
>> numpy.sqrt despite "from scipy import *" in ipy_profile_scipy.py and I don't
>> know the reason.
>
> Perhaps the -pylab switch makes Ipython to run "from numpy import *",
> or something similar, *after* ipy_profile_scipy.py? I'd suppose that
> if you do "from scipy import *" in the Ipython console, you'd get
> Scipy's version of sqrt.

Correct.

> Also, does the order of the -pylab and -p switches on the Ipython
> command line make a difference?

I don't think so.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From humufr at yahoo.fr  Sun Jul 12 15:42:11 2009
From: humufr at yahoo.fr (Nicolas Gruel)
Date: Sun, 12 Jul 2009 12:42:11 -0700 (PDT)
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
In-Reply-To: <3d375d730907121110h2f9f7382se20e5b4c6672fcc8@mail.gmail.com>
Message-ID: <936733.65958.qm@web111411.mail.gq1.yahoo.com>


Why keeping the numpy comportment (NaN)? If it's only for historical reason and it's mathematicaly better to have 1j, this behaviour should be corrected. It was a limitation of numeric but numpy has been written to extend and to remove the limitation.

H. 

--- En date de?: Dim 12.7.09, Robert Kern <robert.kern at gmail.com> a ?crit?:

> De: Robert Kern <robert.kern at gmail.com>
> Objet: Re: [SciPy-user] How to get sqrt(-1) = 1j per default?
> ?: "SciPy Users List" <scipy-user at scipy.org>
> Date: Dimanche 12 Juillet 2009, 14h10
> On Sun, Jul 12, 2009 at 12:12, Pauli
> Virtanen<pav+sp at iki.fi> wrote:
> > On 2009-07-12, Martin Kirsch <xyz.account at arcor.de>
> wrote:
> > [clip?]
> >> But why it makes a difference doing "from scipy
> import *"
> >> (1) per ipy_profile_scipy.py (ipython -pylab -p
> scipy) => sqrt(-1)= NaN
> >> (2) per console input => sqrt(-1)= 1j
> 
> Note that "from scipy import *" is precisely the thing that
> I am
> suggesting that you should never do.
> 
> >> Originally I guessed that in both cases sqrt comes
> from scipy because of the
> >> preceding "from scipy import *" in both cases, but
> case (1) references
> >> numpy.sqrt despite "from scipy import *" in
> ipy_profile_scipy.py and I don't
> >> know the reason.
> >
> > Perhaps the -pylab switch makes Ipython to run "from
> numpy import *",
> > or something similar, *after* ipy_profile_scipy.py?
> I'd suppose that
> > if you do "from scipy import *" in the Ipython
> console, you'd get
> > Scipy's version of sqrt.
> 
> Correct.
> 
> > Also, does the order of the -pylab and -p switches on
> the Ipython
> > command line make a difference?
> 
> I don't think so.
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 


      



From robert.kern at gmail.com  Sun Jul 12 15:56:55 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Jul 2009 14:56:55 -0500
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
In-Reply-To: <936733.65958.qm@web111411.mail.gq1.yahoo.com>
References: <3d375d730907121110h2f9f7382se20e5b4c6672fcc8@mail.gmail.com> 
	<936733.65958.qm@web111411.mail.gq1.yahoo.com>
Message-ID: <3d375d730907121256i7e913512se9731f577552301b@mail.gmail.com>

On Sun, Jul 12, 2009 at 14:42, Nicolas Gruel<humufr at yahoo.fr> wrote:
>
> Why keeping the numpy comportment (NaN)? If it's only for historical reason and it's mathematicaly better to have 1j, this behaviour should be corrected. It was a limitation of numeric but numpy has been written to extend and to remove the limitation.

Both behaviors are useful in different situations. It was not a
limitation of Numeric but rather a specific design decision. Quite
often, passing a negative number to sqrt() is an error that needs to
be detected. That's why the standard library has math.sqrt(-1) which
raises and exception while cmath.sqrt(-1) returns 1j.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From dwf at cs.toronto.edu  Sun Jul 12 17:03:32 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sun, 12 Jul 2009 17:03:32 -0400
Subject: [SciPy-user] Why is SciPy better than Matlab or IDL? ;)
In-Reply-To: <88e473830907110441i5adcfefbr736dec9e20629b0b@mail.gmail.com>
References: <2926F4BC94217A43A2D21792DE8818931C5B245B29@ex1.rijnh.nl>
	<652A0D7E-168B-4355-ABDE-F40F7165664B@cs.toronto.edu>
	<4A537A54.2010601@gmail.com>
	<1BB30B2C-4CE8-44C7-82FF-62E9EFF0FB7C@cs.toronto.edu>
	<loom.20090708T114047-554@post.gmane.org>
	<F860412A-61D0-4622-98C4-DC321FB59439@cs.toronto.edu>
	<88e473830907110441i5adcfefbr736dec9e20629b0b@mail.gmail.com>
Message-ID: <4F392897-95CA-4ED8-B10D-951F8EAE90D4@cs.toronto.edu>


On 11-Jul-09, at 7:41 AM, John Hunter wrote:

> The code you posted is a hybrid of using pyplot and managing your own
> Canvas creation, which can work under certain configurations but is
> brittle and not supported.  This is mainly because pyplot creates and
> manages its own canvas instances.

Oops. I suppose it was an instance where I fiddled until it worked and  
never gave it a second look.  Thanks for the clarification John.

David


From xavier.gnata at gmail.com  Sun Jul 12 17:32:11 2009
From: xavier.gnata at gmail.com (Xavier Gnata)
Date: Sun, 12 Jul 2009 23:32:11 +0200
Subject: [SciPy-user] How to get sqrt(-1) = 1j per default?
In-Reply-To: <3d375d730907121256i7e913512se9731f577552301b@mail.gmail.com>
References: <3d375d730907121110h2f9f7382se20e5b4c6672fcc8@mail.gmail.com>
	<936733.65958.qm@web111411.mail.gq1.yahoo.com>
	<3d375d730907121256i7e913512se9731f577552301b@mail.gmail.com>
Message-ID: <4A5A565B.1070806@gmail.com>

Robert Kern wrote:
> On Sun, Jul 12, 2009 at 14:42, Nicolas Gruel<humufr at yahoo.fr> wrote:
>   
>> Why keeping the numpy comportment (NaN)? If it's only for historical reason and it's mathematicaly better to have 1j, this behaviour should be corrected. It was a limitation of numeric but numpy has been written to extend and to remove the limitation.
>>     
>
> Both behaviors are useful in different situations. It was not a
> limitation of Numeric but rather a specific design decision. Quite
> often, passing a negative number to sqrt() is an error that needs to
> be detected. That's why the standard library has math.sqrt(-1) which
> raises and exception while cmath.sqrt(-1) returns 1j.
>
>   
"it's mathematicaly better to have 1j," It is wrong if it is written in 
the doc that sqrt is a function from R to R.
Maybe what we would need here is a nice one page doc for the newcomers.
The page listing the basic functions in numpy/IDL/matlab is very useful 
when you start with numpy/scipy.
A clear written warning saying that "from scipy import *" is evil could 
also help a lot (iff it contains examples like "sqrt" showing *why* it 
is evil).
Ok, "import *" is not a good practice in python. Not onlyin numpy/scipy 
BUT many users discover python as the same time as they discover 
numpy/scipy.

"it a behaviour is clearly documented with a nice red warning, it is not 
quite a bug ;)"

Xavier



From perfreem at gmail.com  Sun Jul 12 17:51:04 2009
From: perfreem at gmail.com (per freem)
Date: Sun, 12 Jul 2009 17:51:04 -0400
Subject: [SciPy-user] nonzero and the matlab equivalent of find
Message-ID: <e95b09750907121451s36323f58ie6a312d52828f10f@mail.gmail.com>

hi all,

i am trying to find an equivalent of matlab's find in scipy. the
documentation says to use nonzero but i am not sure how to interpret the
results.

suppose i have an array:

a = [[0, 1], [1, 1], [1, 0]]

and i want to return the indices (0, 1, or 2 in this case) of the elements
that are [1, 1]. the natural notation seems to be:

nonzero(a == array([1, 1]))

but this returns: (array([0, 1, 1, 2]), array([1, 0, 1, 0])) -- not sure
what this output means.

can someone please explain how to find the elements that are [1,1], like in
matlab's "find"? thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090712/a63da737/attachment.html>

From robert.kern at gmail.com  Sun Jul 12 18:03:47 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Jul 2009 17:03:47 -0500
Subject: [SciPy-user] nonzero and the matlab equivalent of find
In-Reply-To: <e95b09750907121451s36323f58ie6a312d52828f10f@mail.gmail.com>
References: <e95b09750907121451s36323f58ie6a312d52828f10f@mail.gmail.com>
Message-ID: <3d375d730907121503p30b4a011gfd02bec7a58a1dac@mail.gmail.com>

On Sun, Jul 12, 2009 at 16:51, per freem<perfreem at gmail.com> wrote:
> hi all,
>
> i am trying to find an equivalent of matlab's find in scipy. the
> documentation says to use nonzero but i am not sure how to interpret the
> results.
>
> suppose i have an array:
>
> a = [[0, 1], [1, 1], [1, 0]]
>
> and i want to return the indices (0, 1, or 2 in this case) of the elements
> that are [1, 1]. the natural notation seems to be:
>
> nonzero(a == array([1, 1]))
>
> but this returns: (array([0, 1, 1, 2]), array([1, 0, 1, 0])) -- not sure
> what this output means.

Well, let's take a look at the intermediate value of a==array([1,1]):

In [24]: a = array([[0,1], [1,1], [1,0]])

In [25]: a == array([1, 1])
Out[25]:
array([[False,  True],
       [ True,  True],
       [ True, False]], dtype=bool)


The result of nonzero(), when given a multidimensional array is to
return a tuple with the indices for each dimension such that:

In [26]: a[nonzero(a == array([1,1]))]
Out[26]: array([1, 1, 1, 1])


> can someone please explain how to find the elements that are [1,1], like in
> matlab's "find"? thank you.

There isn't really a function that does this out-of-box, but here is
how to do it:

In [30]: nonzero((a[:,0] == 1) | (a[:,1] == 1))[0]
Out[30]: array([0, 1, 2])

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Sun Jul 12 18:05:59 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Jul 2009 17:05:59 -0500
Subject: [SciPy-user] nonzero and the matlab equivalent of find
In-Reply-To: <3d375d730907121503p30b4a011gfd02bec7a58a1dac@mail.gmail.com>
References: <e95b09750907121451s36323f58ie6a312d52828f10f@mail.gmail.com> 
	<3d375d730907121503p30b4a011gfd02bec7a58a1dac@mail.gmail.com>
Message-ID: <3d375d730907121505u2156a23em1b5fe45bcb47c9b1@mail.gmail.com>

On Sun, Jul 12, 2009 at 17:03, Robert Kern<robert.kern at gmail.com> wrote:

> There isn't really a function that does this out-of-box, but here is
> how to do it:
>
> In [30]: nonzero((a[:,0] == 1) | (a[:,1] == 1))[0]
> Out[30]: array([0, 1, 2])

Or more generally:

In [34]: nonzero(logical_or.reduce(a == array([1,1]), axis=1))[0]
Out[34]: array([0, 1, 2])

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robertrobert93 at yahoo.com  Mon Jul 13 02:56:34 2009
From: robertrobert93 at yahoo.com (Robert Robert)
Date: Sun, 12 Jul 2009 23:56:34 -0700 (PDT)
Subject: [SciPy-user] retrieve indexes of 2D array element
Message-ID: <941629.97046.qm@web59305.mail.re1.yahoo.com>

Hi all,
I have a 2D scipy array of integer. I can not figure out how
to retrieve the indexes i, j of a specific element in the array. I can
loop through the array in python and search for it, but I can not
believe that scipy does not have a highly optimized algorithm. 
Thanks,
robert


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090712/27f97e0d/attachment.html>

From robertrobert93 at yahoo.com  Mon Jul 13 02:58:07 2009
From: robertrobert93 at yahoo.com (Robert Robert)
Date: Sun, 12 Jul 2009 23:58:07 -0700 (PDT)
Subject: [SciPy-user] retrieve indexes of element in 2D scipy array
Message-ID: <131065.71917.qm@web59314.mail.re1.yahoo.com>

Hi all,
I have a 2D scipy array of integer. I can not figure out how
to retrieve the indexes i, j of a specific element in the array. I can
loop through the array in python and search for it, but I can not
believe that scipy does not have a highly optimized algorithm. 
Thanks,
robert


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090712/3a6ff135/attachment.html>

From dwf at cs.toronto.edu  Mon Jul 13 03:11:05 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Mon, 13 Jul 2009 03:11:05 -0400
Subject: [SciPy-user] retrieve indexes of 2D array element
In-Reply-To: <941629.97046.qm@web59305.mail.re1.yahoo.com>
References: <941629.97046.qm@web59305.mail.re1.yahoo.com>
Message-ID: <695927F2-F03D-4F13-A990-703EA47FE5C0@cs.toronto.edu>

On 13-Jul-09, at 2:56 AM, Robert Robert wrote:

> Hi all,
> I have a 2D scipy array of integer. I can not figure out how to  
> retrieve the indexes i, j of a specific element in the array. I can  
> loop through the array in python and search for it, but I can not  
> believe that scipy does not have a highly optimized algorithm.
> Thanks,
> robert


By a specific element, do you mean a certain value? If so use a  
boolean condition and where().

http://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html

David


From robertrobert93 at yahoo.com  Mon Jul 13 04:03:05 2009
From: robertrobert93 at yahoo.com (Robert Robert)
Date: Mon, 13 Jul 2009 01:03:05 -0700 (PDT)
Subject: [SciPy-user] retrieve indexes of 2D array element
Message-ID: <218899.69139.qm@web59308.mail.re1.yahoo.com>

Hi,
I've seen this function where, but it requires three inputs and I have no clue how to use it. Is it possible to give me an example. E.g I have an array as

a = [ [1,2,3], 
??????? [4,5,6],
??????? [7,8,9] ]

and I would like to retrieve the indexes of values 9 which are 2, 2.
How do I do that with scipy.where ?
Thanks,
robert

--- On Mon, 7/13/09, David Warde-Farley <dwf at cs.toronto.edu> wrote:

From: David Warde-Farley <dwf at cs.toronto.edu>
Subject: Re: [SciPy-user] retrieve indexes of 2D array element
To: "SciPy Users List" <scipy-user at scipy.org>
Date: Monday, July 13, 2009, 7:11 AM

On 13-Jul-09, at 2:56 AM, Robert Robert wrote:

> Hi all,
> I have a 2D scipy array of integer. I can not figure out how to? 
> retrieve the indexes i, j of a specific element in the array. I can? 
> loop through the array in python and search for it, but I can not? 
> believe that scipy does not have a highly optimized algorithm.
> Thanks,
> robert


By a specific element, do you mean a certain value? If so use a? 
boolean condition and where().

http://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html

David
_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090713/5953bd7c/attachment.html>

From d_l_goldsmith at yahoo.com  Mon Jul 13 04:31:17 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Mon, 13 Jul 2009 01:31:17 -0700 (PDT)
Subject: [SciPy-user] sqrt(-1)
Message-ID: <368305.16353.qm@web52103.mail.re2.yahoo.com>


Ah, memories... ;-)

http://mail.scipy.org/pipermail/numpy-discussion/2006-October/011318.html

DG

--- On Mon, 7/13/09, scipy-user-request at scipy.org <scipy-user-request at scipy.org> wrote:

> Today's Topics:
> 
> ???1. Re: How to get sqrt(-1) = 1j per
> default? (Pauli Virtanen)
> ???2. Re: How to get sqrt(-1) = 1j per
> default? (Robert Kern)
> ???3. Re: How to get sqrt(-1) = 1j per
> default? (Nicolas Gruel)
> ???4. Re: How to get sqrt(-1) = 1j per
> default? (Robert Kern)
> ???6. Re: How to get sqrt(-1) = 1j per
> default? (Xavier Gnata)



      


From dwf at cs.toronto.edu  Mon Jul 13 07:26:35 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Mon, 13 Jul 2009 07:26:35 -0400
Subject: [SciPy-user] retrieve indexes of 2D array element
In-Reply-To: <218899.69139.qm@web59308.mail.re1.yahoo.com>
References: <218899.69139.qm@web59308.mail.re1.yahoo.com>
Message-ID: <68583489-BD7F-495D-AAE4-F85E886B8FEC@cs.toronto.edu>

It doesn't require 3 inputs. Two of them are optional. See the  
examples, or this:

In [20]: A = array([[1,4,9],[3,5,6],[9,9,3]])

In [21]: rows, cols = where(A == 9)

In [22]: rows
Out[22]: array([0, 2, 2])

In [23]: cols
Out[23]: array([2, 0, 1])

A == 9 will produce a boolean array equal to True where the element is  
9 and False otherwise. Use this as the first argument to where.

David

On 13-Jul-09, at 4:03 AM, Robert Robert wrote:

> Hi,
> I've seen this function where, but it requires three inputs and I  
> have no clue how to use it. Is it possible to give me an example.  
> E.g I have an array as
>
> a = [ [1,2,3],
>         [4,5,6],
>         [7,8,9] ]
>
> and I would like to retrieve the indexes of values 9 which are 2, 2.
> How do I do that with scipy.where ?
> Thanks,
> robert
>
> --- On Mon, 7/13/09, David Warde-Farley <dwf at cs.toronto.edu> wrote:
>
> From: David Warde-Farley <dwf at cs.toronto.edu>
> Subject: Re: [SciPy-user] retrieve indexes of 2D array element
> To: "SciPy Users List" <scipy-user at scipy.org>
> Date: Monday, July 13, 2009, 7:11 AM
>
> On 13-Jul-09, at 2:56 AM, Robert Robert wrote:
>
> > Hi all,
> > I have a 2D scipy array of integer. I can not figure out how to
> > retrieve the indexes i, j of a specific element in the array. I can
> > loop through the array in python and search for it, but I can not
> > believe that scipy does not have a highly optimized algorithm.
> > Thanks,
> > robert
>
>
> By a specific element, do you mean a certain value? If so use a
> boolean condition and where().
>
> http://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html
>
> David
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From stefan at sun.ac.za  Mon Jul 13 07:30:49 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 13 Jul 2009 13:30:49 +0200
Subject: [SciPy-user] retrieve indexes of element in 2D scipy array
In-Reply-To: <131065.71917.qm@web59314.mail.re1.yahoo.com>
References: <AcoDiDPLNOwnZEdzQuKV4j0nbXxPbQ==>
	<131065.71917.qm@web59314.mail.re1.yahoo.com>
Message-ID: <9457e7c80907130430v4ea54ea4pa765f9712fca6679@mail.gmail.com>

2009/7/13 Robert Robert <robertrobert93 at yahoo.com>:
> I have a 2D scipy array of integer. I can not figure out how to retrieve the
> indexes i, j of a specific element in the array. I can loop through the
> array in python and search for it, but I can not believe that scipy does not
> have a highly optimized algorithm.

Try

np.where(x == 3)

St?fan


From welby555 at yahoo.com  Mon Jul 13 08:14:13 2009
From: welby555 at yahoo.com (Afi Welbeck)
Date: Mon, 13 Jul 2009 05:14:13 -0700 (PDT)
Subject: [SciPy-user] plot vertical lines
Message-ID: <72226.27338.qm@web51612.mail.re2.yahoo.com>

Hi,

I'm a newbie. I'm stuck trying to link the following
points with vertical lines in the xy plane 
(1,1) (1,4)  and (3,2) (3,6)

Could anyone please help me with the code?
Thanks a great deal.

Harriet.


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090713/b5d4ebbb/attachment.html>

From amcmorl at gmail.com  Mon Jul 13 08:31:17 2009
From: amcmorl at gmail.com (Angus McMorland)
Date: Mon, 13 Jul 2009 08:31:17 -0400
Subject: [SciPy-user] plot vertical lines
In-Reply-To: <72226.27338.qm@web51612.mail.re2.yahoo.com>
References: <72226.27338.qm@web51612.mail.re2.yahoo.com>
Message-ID: <e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>

2009/7/13 Afi Welbeck <welby555 at yahoo.com>:
> Hi,
>
> I'm a newbie. I'm stuck trying to link the following
> points with vertical lines in the xy plane
> (1,1) (1,4)? and (3,2) (3,6)
>
> Could anyone please help me with the code?

Here's an verbose way to do it, so you can see what's going on. You'll
need matplotlib installed for this, and it's the generally recommended
2-d graphics package to accompany scipy.

import matplotlib.pyplot as plt
x0 = [1,1]
y0 = [1,4]
x1 = [3,3]
y1 = [2,6]
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(x0, y0, x1, y1)
ax.set_xlim([0, 4])
ax.set_ylim([0, 7])
plt.show()

Hoping that helps,

Angus.
-- 
AJC McMorland
Post-doctoral research fellow
Neurobiology, University of Pittsburgh


From silva at lma.cnrs-mrs.fr  Mon Jul 13 08:47:40 2009
From: silva at lma.cnrs-mrs.fr (Fabrice Silva)
Date: Mon, 13 Jul 2009 14:47:40 +0200
Subject: [SciPy-user] plot vertical lines
In-Reply-To: <e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
References: <72226.27338.qm@web51612.mail.re2.yahoo.com>
	<e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
Message-ID: <1247489260.4358.7.camel@localhost.localdomain>

Le lundi 13 juillet 2009 ? 08:31 -0400, Angus McMorland a ?crit :
> 2009/7/13 Afi Welbeck <welby555 at yahoo.com>:
> > Hi,
> >
> > I'm a newbie. I'm stuck trying to link the following
> > points with vertical lines in the xy plane
> > (1,1) (1,4)  and (3,2) (3,6)
> >
> > Could anyone please help me with the code?

I was suggesting:
plt.axvline(1, ymin=1, ymax=4, trans=plt.gca().transData)
plt.axvline(3, ymin=2, ymax=6, trans=plt.gca().transData)

when I got the following message :
TypeError: type object got multiple values for keyword argument
'transform'

In fact, despite the doc tells axvline accepts the transform keyworg
argument, there is no special handling of this kwarg in axvline code (in
0.98.5.3). There is a problem since axvline code (and axhline and
ax*span too I suppose) add a transform kwarg before adding the line to
the axes...
-- 
Fabrice Silva
Laboratory of Mechanics and Acoustics - CNRS
31 chemin Joseph Aiguier, 13402 Marseille, France.



From welby555 at yahoo.com  Mon Jul 13 09:32:56 2009
From: welby555 at yahoo.com (Afi Welbeck)
Date: Mon, 13 Jul 2009 06:32:56 -0700 (PDT)
Subject: [SciPy-user] plot vertical lines
In-Reply-To: <e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
References: <72226.27338.qm@web51612.mail.re2.yahoo.com>
	<e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
Message-ID: <135317.46656.qm@web51608.mail.re2.yahoo.com>

Hi,
Thanks it worked.
But I observed that for the values I gave you,
you altered them a bit before it plotted it just 
they way I expected. Is there some formula
for this sort of plot?




________________________________
From: Angus McMorland <amcmorl at gmail.com>
To: SciPy Users List <scipy-user at scipy.org>
Sent: Monday, July 13, 2009 2:31:17 PM
Subject: Re: [SciPy-user] plot vertical lines

2009/7/13 Afi Welbeck <welby555 at yahoo.com>:
> Hi,
>
> I'm a newbie. I'm stuck trying to link the following
> points with vertical lines in the xy plane
> (1,1) (1,4)  and (3,2) (3,6)
>
> Could anyone please help me with the code?

Here's an verbose way to do it, so you can see what's going on. You'll
need matplotlib installed for this, and it's the generally recommended
2-d graphics package to accompany scipy.

import matplotlib.pyplot as plt
x0 = [1,1]
y0 = [1,4]
x1 = [3,3]
y1 = [2,6]
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(x0, y0, x1, y1)
ax.set_xlim([0, 4])
ax.set_ylim([0, 7])
plt.show()

Hoping that helps,

Angus.
-- 
AJC McMorland
Post-doctoral research fellow
Neurobiology, University of Pittsburgh
_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090713/5f05c862/attachment.html>

From carlos.grohmann at gmail.com  Mon Jul 13 10:04:45 2009
From: carlos.grohmann at gmail.com (=?ISO-8859-1?Q?Carlos_=22Gu=E2no=22_Grohmann?=)
Date: Mon, 13 Jul 2009 11:04:45 -0300
Subject: [SciPy-user] finding values in a list?
In-Reply-To: <1cd32cbb0907111432y57b1fecel63115dbbb1c2e200@mail.gmail.com>
References: <bd07447b0907111401y2a0204f6i3f1349e6ec906ef7@mail.gmail.com> 
	<1cd32cbb0907111422m42d6a053x965523d7e17579d3@mail.gmail.com> 
	<c048da1c0907111429l3ba26c77uf760de3640600a70@mail.gmail.com> 
	<1cd32cbb0907111432y57b1fecel63115dbbb1c2e200@mail.gmail.com>
Message-ID: <bd07447b0907130704j533e86ealfeda532b1f1036e8@mail.gmail.com>

Thanks all for the help

Carlos



On Sat, Jul 11, 2009 at 18:32, <josef.pktd at gmail.com> wrote:
> On Sat, Jul 11, 2009 at 5:29 PM, Skipper Seabold<jsseabold at gmail.com> wrote:
>> On Sat, Jul 11, 2009 at 5:22 PM, <josef.pktd at gmail.com> wrote:
>>> 2009/7/11 Carlos "Gu?no" Grohmann <carlos.grohmann at gmail.com>:
>>>> Sorry if this is too simple or even OT, but say I have a list like
>>>>
>>>> [1,2,3,4,5,6,7,8,9]
>>>>
>>>> and I want to get the values just below and above a given value.
>>>>
>>>> So, if I enter 6.2, I get 6 and 7.
>>>>
>>>> but how?
>>>
>>> maybe like this
>>>>>> a = (np.array([1,2,3,4,5,6,7,8,9])-6.1)
>>>>>> a[a>0].min() + 6.2
>>> 7.0
>>>>>> a[a<0].max() + 6.2
>>> 6.0
>>>
>>> Josef
>>>
>>
>> Yet another solution (though I too don't know if it's optimal). ?This
>> will work if you know that your values are always going to be within 1
>> as in your example.
>>
>>>>> import numpy as np
>>>>> L = [1,2,3,4,5,6,7,8]
>>>>> A = np.array(L)
>>>>> index = np.where(np.abs((A-6.2))<1)
>>>>> A[index]
>> array([6, 7])
>>
>> Skipper
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>
> for sorted lists, this might be fastest
>
>>>> ind = np.searchsorted([1,2,3,4,5,6,7,8,9], 6.2)
>>>> ind
> 6
>>>> [1,2,3,4,5,6,7,8,9][ind-1]
> 6
>>>> [1,2,3,4,5,6,7,8,9][ind]
>
> It is unclear what should happen if the values is an element of the list.
>
> Josef
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



-- 
Carlos Henrique Grohmann - Geologist D.Sc.
a.k.a. Guano - Linux User #89721
ResearcherID: A-9030-2008

http://digitalelevation.blogspot.com

http://www.igc.usp.br/pessoais/guano
_________________
Can?t stop the signal.


From Dharhas.Pothina at twdb.state.tx.us  Mon Jul 13 10:26:35 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Mon, 13 Jul 2009 09:26:35 -0500
Subject: [SciPy-user] Sort by first n columns in an array
In-Reply-To: <loom.20090710T131833-711@post.gmane.org>
References: <4A549EE5.63BA.009B.0@twdb.state.tx.us>
	<loom.20090708T230307-547@post.gmane.org>
	<4A55C491.63BA.009B.0@twdb.state.tx.us>
	<loom.20090710T112055-235@post.gmane.org>
	<4A56ED12.63BA.009B.0@twdb.state.tx.us>
	<loom.20090710T131833-711@post.gmane.org>
Message-ID: <4A5AFDCB.63BA.009B.0@twdb.state.tx.us>

> But it's probably better to let genfromtxt choose its own field names and then
> sort the whole thing using the first 6 fields.
> 
> alldata = np.genfromtxt(filename, dtype=None)
> alldata.sort(order='f0 f1 f2 f3 f4 f5'.split())

Great, this is exactly what I needed. I had thought of the first approach but didn't want to have to read the file in twice.

thanks,

- dharhas






From amcmorl at gmail.com  Mon Jul 13 10:49:00 2009
From: amcmorl at gmail.com (Angus McMorland)
Date: Mon, 13 Jul 2009 10:49:00 -0400
Subject: [SciPy-user] plot vertical lines
In-Reply-To: <135317.46656.qm@web51608.mail.re2.yahoo.com>
References: <72226.27338.qm@web51612.mail.re2.yahoo.com>
	<e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
	<135317.46656.qm@web51608.mail.re2.yahoo.com>
Message-ID: <e85f13c40907130749p17a8bc90t27837471f1fa0fa4@mail.gmail.com>

2009/7/13 Afi Welbeck <welby555 at yahoo.com>:
> Hi,
> Thanks it worked.
> But I observed that for the values I gave you,
> you altered them a bit before it plotted it just
> they way I expected. Is there some formula
> for this sort of plot?

I take your question as asking what the convention is for the
co-ordinate listing in the plot command (?). As I tried to indicate
with the variable names, the x-values and y-values are listed
separately, and multiple lines can be listed sequentially, as in
line0_x, line0_y, line1_x, line1_y... so the first variable is the x
values of the first line, the second is the corresponding y values of
the first line and then the same for the second line. If you're using
ipython (highly recommended), then you could find this sort of
information by typing:

In [1]: ax.plot?

or

In [1]: plt.plot?

which will bring up the docstring for the associated function.

Angus.
-- 
AJC McMorland
Post-doctoral research fellow
Neurobiology, University of Pittsburgh


From mailanhilli at googlemail.com  Mon Jul 13 11:38:49 2009
From: mailanhilli at googlemail.com (Matthias Hillenbrand)
Date: Mon, 13 Jul 2009 17:38:49 +0200
Subject: [SciPy-user] 2D/3D visualization
Message-ID: <67b3a51f0907130838i57cd782dk70b7950252c56812@mail.gmail.com>

Hello,

In the new future I want to build a small GUI for my optical raytracing
program written in python. For this purpose I intend to use wxpython,
perhaps in combination with ETS. I especially want to be able to create 2D
figures like this one:

http://www.optima-research.com/Software/Optical/Zemax/images/2DLAYOUT.gif

and 3D figures like those:

http://www.zemax.com/kb/content_images/presentation/Default%20Shaded%20Model%20of%20Double%20Gauss.gif
http://www.zemax.com/kb/content_images/presentation/revised%20shaded%20model%20of%20Double%20Gauss.jpg

Do you have any advice which software packages for 2D/3D visualization could
be most appropriate for my purpose? In the near future I also want to work
with NURBS, so it will be even better if the package supports rendering of
NURBS.

As this is my first programming project, I would prefer a high level
package, where I only have to define the surfaces and don't have to care
about how they are rendered. It would also be nice if I could zoom, pan, and
rotate the figures.

I have already looked at Matplotlib, Chaco, Mayavi, and PyOpenGL but don't
know about further, perhaps more appropriate, alternatives.

Thank you very much for your help!

Matthias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090713/34e2c621/attachment.html>

From gokhansever at gmail.com  Mon Jul 13 11:49:12 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Mon, 13 Jul 2009 10:49:12 -0500
Subject: [SciPy-user] 2D/3D visualization
In-Reply-To: <67b3a51f0907130838i57cd782dk70b7950252c56812@mail.gmail.com>
References: <67b3a51f0907130838i57cd782dk70b7950252c56812@mail.gmail.com>
Message-ID: <49d6b3500907130849y59a32319o202202dce0a3fa98@mail.gmail.com>

On Mon, Jul 13, 2009 at 10:38 AM, Matthias Hillenbrand <
mailanhilli at googlemail.com> wrote:

> Hello,
>
> As this is my first programming project, I would prefer a high level
> package, where I only have to define the surfaces and don't have to care
> about how they are rendered. It would also be nice if I could zoom, pan, and
> rotate the figures.
>


Although I don't know exactly about how Mayavi handles surfaces, its VTK
engine handles rendering via OpenGL, and has all the other functionalities
you are seeking.

I am not a super user in Mayavi, but from my more than a couple months of
usage experiences Mayavi would seem to be a great fit for your design needs.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090713/aa135a15/attachment.html>

From welby555 at yahoo.com  Mon Jul 13 12:41:28 2009
From: welby555 at yahoo.com (Afi Welbeck)
Date: Mon, 13 Jul 2009 09:41:28 -0700 (PDT)
Subject: [SciPy-user] plot vertical lines
In-Reply-To: <e85f13c40907130749p17a8bc90t27837471f1fa0fa4@mail.gmail.com>
References: <72226.27338.qm@web51612.mail.re2.yahoo.com>
	<e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
	<135317.46656.qm@web51608.mail.re2.yahoo.com>
	<e85f13c40907130749p17a8bc90t27837471f1fa0fa4@mail.gmail.com>
Message-ID: <459568.89486.qm@web51605.mail.re2.yahoo.com>

Hi,
Thanks it worked. I realised my error.
If I may ask, is there a way to put the
pairs of points that make a vertical line 
into a list and then plot them so you have
a number of vertical lines without having 
to list them separately? 
Thanks.

H. A. Welbeck.


________________________________
From: Angus McMorland <amcmorl at gmail.com>
To: SciPy Users List <scipy-user at scipy.org>
Sent: Monday, July 13, 2009 4:49:00 PM
Subject: Re: [SciPy-user] plot vertical lines

2009/7/13 Afi Welbeck <welby555 at yahoo.com>:
> Hi,
> Thanks it worked.
> But I observed that for the values I gave you,
> you altered them a bit before it plotted it just
> they way I expected. Is there some formula
> for this sort of plot?

I take your question as asking what the convention is for the
co-ordinate listing in the plot command (?). As I tried to indicate
with the variable names, the x-values and y-values are listed
separately, and multiple lines can be listed sequentially, as in
line0_x, line0_y, line1_x, line1_y... so the first variable is the x
values of the first line, the second is the corresponding y values of
the first line and then the same for the second line. If you're using
ipython (highly recommended), then you could find this sort of
information by typing:

In [1]: ax.plot?

or

In [1]: plt.plot?

which will bring up the docstring for the associated function.

Angus.
-- 
AJC McMorland
Post-doctoral research fellow
Neurobiology, University of Pittsburgh
_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090713/1806aa41/attachment.html>

From amcmorl at gmail.com  Mon Jul 13 13:06:33 2009
From: amcmorl at gmail.com (Angus McMorland)
Date: Mon, 13 Jul 2009 13:06:33 -0400
Subject: [SciPy-user] plot vertical lines
In-Reply-To: <459568.89486.qm@web51605.mail.re2.yahoo.com>
References: <72226.27338.qm@web51612.mail.re2.yahoo.com>
	<e85f13c40907130531n5bd342afse765a64cd1ccda9@mail.gmail.com>
	<135317.46656.qm@web51608.mail.re2.yahoo.com>
	<e85f13c40907130749p17a8bc90t27837471f1fa0fa4@mail.gmail.com>
	<459568.89486.qm@web51605.mail.re2.yahoo.com>
Message-ID: <e85f13c40907131006m6377af01x2df3cca960a5a019@mail.gmail.com>

2009/7/13 Afi Welbeck <welby555 at yahoo.com>:
> Hi,
> Thanks it worked. I realised my error.
> If I may ask, is there a way to put the
> pairs of points that make a vertical line
> into a list and then plot them so you have
> a number of vertical lines without having
> to list them separately?
> Thanks.

You can construct a list of lists (or a 2d numpy array) that will do
this. In array terminology (which is easier to describe) the elements
of the first dimension are the points in each line, and the second
dimension gives multiple lines to plot, like so:

import matplotlib.pyplot as plt
xs = [[1,3],[1,3]]
ys = [[1,2],[4,6]]
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(xs, ys)
ax.set_xlim([0, 4])
ax.set_ylim([0, 7])
plt.show()

The corresponding arrays would be created by substituting in:

import numpy as np
xs = np.array([[1,3],[1,3]])
ys = np.array([[1,2],[4,6]])

These are really matplotlib questions, so it would be best to move any
further questions on to the matplotlib list
(matplotlib-users at lists.sourceforge.net), which will make sure they
reach the most relevant audience.

Angus.
-- 
AJC McMorland
Post-doctoral research fellow
Neurobiology, University of Pittsburgh


From s.mientki at ru.nl  Mon Jul 13 14:41:42 2009
From: s.mientki at ru.nl (Stef Mientki)
Date: Mon, 13 Jul 2009 20:41:42 +0200
Subject: [SciPy-user] 2D/3D visualization
In-Reply-To: <67b3a51f0907130838i57cd782dk70b7950252c56812@mail.gmail.com>
References: <67b3a51f0907130838i57cd782dk70b7950252c56812@mail.gmail.com>
Message-ID: <4A5B7FE6.4090000@ru.nl>

take a look at VPython,
here some examples made with VPython
http://www-ee.eng.hawaii.edu/~zqyun/caevp.html
cheers,
Stef

Matthias Hillenbrand wrote:
> Hello,
>
> In the new future I want to build a small GUI for my optical 
> raytracing program written in python. For this purpose I intend to use 
> wxpython, perhaps in combination with ETS. I especially want to be 
> able to create 2D figures like this one:
>
> http://www.optima-research.com/Software/Optical/Zemax/images/2DLAYOUT.gif
>
> and 3D figures like those:
>
> http://www.zemax.com/kb/content_images/presentation/Default%20Shaded%20Model%20of%20Double%20Gauss.gif
> http://www.zemax.com/kb/content_images/presentation/revised%20shaded%20model%20of%20Double%20Gauss.jpg
>
> Do you have any advice which software packages for 2D/3D visualization 
> could be most appropriate for my purpose? In the near future I also 
> want to work with NURBS, so it will be even better if the package 
> supports rendering of NURBS.
>
> As this is my first programming project, I would prefer a high level 
> package, where I only have to define the surfaces and don't have to 
> care about how they are rendered. It would also be nice if I could 
> zoom, pan, and rotate the figures.
>
> I have already looked at Matplotlib, Chaco, Mayavi, and PyOpenGL but 
> don't know about further, perhaps more appropriate, alternatives.
>
> Thank you very much for your help!
>
> Matthias
> ------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From rajanikanth at gmail.com  Mon Jul 13 21:44:27 2009
From: rajanikanth at gmail.com (rajanikanth at gmail.com)
Date: Tue, 14 Jul 2009 01:44:27 +0000
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
	Error)
Message-ID: <0016e649c9263fd92c046ea0943d@google.com>

Hi

Could somebody tell me how to prevent this:

>>> from scipy import signal
Bus error

Thanks,

Raj
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090714/a82bb849/attachment.html>

From cournape at gmail.com  Mon Jul 13 22:11:50 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 14 Jul 2009 11:11:50 +0900
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
	Error)
In-Reply-To: <0016e649c9263fd92c046ea0943d@google.com>
References: <0016e649c9263fd92c046ea0943d@google.com>
Message-ID: <5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>

On Tue, Jul 14, 2009 at 10:44 AM, <rajanikanth at gmail.com> wrote:
> Hi
>
> Could somebody tell me how to prevent this:
>
>>>> from scipy import signal
> Bus error

Which version of numpy and scipy are you using ?

David


From rajanikanth at gmail.com  Mon Jul 13 22:54:08 2009
From: rajanikanth at gmail.com (Rajanikanth Jammalamadaka)
Date: Mon, 13 Jul 2009 19:54:08 -0700
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
	Error)
In-Reply-To: <5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>
References: <0016e649c9263fd92c046ea0943d@google.com>
	<5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>
Message-ID: <84bdef3c0907131954r8658f28x7a3dbd00dfe72614@mail.gmail.com>

On Mon, Jul 13, 2009 at 7:11 PM, David Cournapeau<cournape at gmail.com> wrote:
> On Tue, Jul 14, 2009 at 10:44 AM, <rajanikanth at gmail.com> wrote:
>> Hi
>>
>> Could somebody tell me how to prevent this:
>>
>>>>> from scipy import signal
>> Bus error
>
> Which version of numpy and scipy are you using ?

Hi David:

I am using

numpy version: 1.2.1
scipy version:  0.7.1

I tried it on another system (also Mac OS X) which had numpy version
1.3 and it seems to work fine on that one.

So, is it a version problem?

Thanks,

Raj
>
> David
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



-- 
Rajanikanth


From david at ar.media.kyoto-u.ac.jp  Mon Jul 13 22:38:54 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 14 Jul 2009 11:38:54 +0900
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
 Error)
In-Reply-To: <84bdef3c0907131954r8658f28x7a3dbd00dfe72614@mail.gmail.com>
References: <0016e649c9263fd92c046ea0943d@google.com>	<5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>
	<84bdef3c0907131954r8658f28x7a3dbd00dfe72614@mail.gmail.com>
Message-ID: <4A5BEFBE.7040006@ar.media.kyoto-u.ac.jp>

Rajanikanth Jammalamadaka wrote:
> I tried it on another system (also Mac OS X) which had numpy version
> 1.3 and it seems to work fine on that one.
>
> So, is it a version problem?
>   

Yes - if you use the official scipy binary, you need to use the
corresponding official numpy binary (which is 1.3.0 for python 2.6). As
an alternative, you can build scipy by yourself against numpy - in this
case, you can use numpy 1.2.1.

cheers,

David


From rajanikanth at gmail.com  Mon Jul 13 23:01:27 2009
From: rajanikanth at gmail.com (Rajanikanth Jammalamadaka)
Date: Mon, 13 Jul 2009 20:01:27 -0700
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
	Error)
In-Reply-To: <4A5BEFBE.7040006@ar.media.kyoto-u.ac.jp>
References: <0016e649c9263fd92c046ea0943d@google.com>
	<5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>
	<84bdef3c0907131954r8658f28x7a3dbd00dfe72614@mail.gmail.com>
	<4A5BEFBE.7040006@ar.media.kyoto-u.ac.jp>
Message-ID: <84bdef3c0907132001t18210683q9aea3a70b861d48f@mail.gmail.com>

On Mon, Jul 13, 2009 at 7:38 PM, David
Cournapeau<david at ar.media.kyoto-u.ac.jp> wrote:
> Rajanikanth Jammalamadaka wrote:
>> I tried it on another system (also Mac OS X) which had numpy version
>> 1.3 and it seems to work fine on that one.
>>
>> So, is it a version problem?
>>
>
> Yes - if you use the official scipy binary, you need to use the
> corresponding official numpy binary (which is 1.3.0 for python 2.6). As
> an alternative, you can build scipy by yourself against numpy - in this
> case, you can use numpy 1.2.1.

Thanks. I upgraded to numpy-1.3. It works fine now.

Raj

>
> cheers,
>
> David
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



-- 
Rajanikanth


From adrian.prw at gmail.com  Tue Jul 14 10:56:33 2009
From: adrian.prw at gmail.com (Adrian Price-Whelan)
Date: Tue, 14 Jul 2009 10:56:33 -0400
Subject: [SciPy-user] Quick way to delete all 'values' from array
Message-ID: <31F448E2-6034-46D5-8231-4CDEE2ED836F@gmail.com>

Hey --

I'm just looking for the quickest way to remove all X from an array  
[a,b,c,d,X,e,X,f,gX] or it could be multidimensional, I suppose, but  
thats the idea. I understand delete() will remove a value at a  
specific index, but I was unsuccessful in combining this function with  
'where' to get what I want. Any suggestions?

Thanks,
-Adrian


From gokhansever at gmail.com  Tue Jul 14 11:07:17 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 14 Jul 2009 10:07:17 -0500
Subject: [SciPy-user] Quick way to delete all 'values' from array
In-Reply-To: <31F448E2-6034-46D5-8231-4CDEE2ED836F@gmail.com>
References: <31F448E2-6034-46D5-8231-4CDEE2ED836F@gmail.com>
Message-ID: <49d6b3500907140807m5fdef2d8h2e2941cbf1934bc0@mail.gmail.com>

|


On Tue, Jul 14, 2009 at 9:56 AM, Adrian Price-Whelan
<adrian.prw at gmail.com> wrote:
>
> Hey --
>
> I'm just looking for the quickest way to remove all X from an array
> [a,b,c,d,X,e,X,f,gX] or it could be multidimensional, I suppose, but
> thats the idea. I understand delete() will remove a value at a
> specific index, but I was unsuccessful in combining this function with
> 'where' to get what I want. Any suggestions?
>
> Thanks,
> -Adrian
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user


This is my quickest solution with a list comprehension :

In [2]: a = array([1,2,3,4,6,5,6,7,8,6])

In [3]: a = array([a[i] for i in range(len(a)) if a[i] != 6])

In [4]: a
Out[4]: array([1, 2, 3, 4, 5, 7, 8])


--
G?khan


From scott.sinclair.za at gmail.com  Tue Jul 14 11:14:48 2009
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Tue, 14 Jul 2009 17:14:48 +0200
Subject: [SciPy-user] Quick way to delete all 'values' from array
In-Reply-To: <31F448E2-6034-46D5-8231-4CDEE2ED836F@gmail.com>
References: <31F448E2-6034-46D5-8231-4CDEE2ED836F@gmail.com>
Message-ID: <6a17e9ee0907140814t1c81e208ydd8dd01a4923e758@mail.gmail.com>

>2009/7/14 Adrian Price-Whelan <adrian.prw at gmail.com>:
> I'm just looking for the quickest way to remove all X from an array
> [a,b,c,d,X,e,X,f,gX] or it could be multidimensional, I suppose, but
> thats the idea. I understand delete() will remove a value at a
> specific index, but I was unsuccessful in combining this function with
> 'where' to get what I want. Any suggestions?

If by 'quickest' you mean 'easiest'. Here's how to do it using fancy indexing:

>>> import numpy as np
>>> a = np.array([0, -1, 2, 3, -1, 4])
>>> a
array([ 0, -1,  2,  3, -1,  4])
>>> a = a[a != -1]
>>> a
array([0, 2, 3, 4])

This works because a != 1 returns a boolean array that can be used as
indices into the original array.

>>> a = np.array([0, -1, 2, 3, -1, 4])
>>> a != -1
array([ True, False,  True,  True, False,  True], dtype=bool)

See also:

http://docs.scipy.org/doc/numpy/user/basics.indexing.html

Cheers,
Scott


From Dharhas.Pothina at twdb.state.tx.us  Tue Jul 14 13:23:39 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Tue, 14 Jul 2009 12:23:39 -0500
Subject: [SciPy-user] scikits.timeseries : Calculate previous 30 day and
	30-60 day	cumulative flows.
Message-ID: <4A5C78CB.63BA.009B.0@twdb.state.tx.us>

Hi,

I have a daily timeseries called 'inflow_ts' and for each date in the series I need to calculate the cumulative inflow for the previous 30 days and the previous 30-60 days. From the documentation I think scikits.timeseries.lib.moving_funcs.mov_sum should do what I need for the first case using

inflow_30 = mov_sum(inflow_ts,30)

but this throws an 'AttributeError : logical_not'

Also, how would I calculate the 30-60 day flow. I guess, I could calculate the previous 60 day flow and subtract the 30 day flow. Is there a better way?

thanks.

- dharhas



From pgmdevlist at gmail.com  Tue Jul 14 13:48:06 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 14 Jul 2009 13:48:06 -0400
Subject: [SciPy-user] scikits.timeseries : Calculate previous 30 day and
	30-60 day	cumulative flows.
In-Reply-To: <4A5C78CB.63BA.009B.0@twdb.state.tx.us>
References: <4A5C78CB.63BA.009B.0@twdb.state.tx.us>
Message-ID: <0BD05B3E-E2C0-49C3-8AC3-3FA7BBEB2932@gmail.com>


On Jul 14, 2009, at 1:23 PM, Dharhas Pothina wrote:

> Hi,
>
> I have a daily timeseries called 'inflow_ts' and for each date in  
> the series I need to calculate the cumulative inflow for the  
> previous 30 days and the previous 30-60 days. From the documentation  
> I think scikits.timeseries.lib.moving_funcs.mov_sum should do what I  
> need for the first case using
>
> inflow_30 = mov_sum(inflow_ts,30)
>
> but this throws an 'AttributeError : logical_not'

Now that's a strange one. Send me more info (inflows_ts, version, ...)


> Also, how would I calculate the 30-60 day flow. I guess, I could  
> calculate the previous 60 day flow and subtract the 30 day flow. Is  
> there a better way?

Not that I see right now.



From asreeve at maine.edu  Tue Jul 14 14:08:22 2009
From: asreeve at maine.edu (A.Reeve)
Date: Tue, 14 Jul 2009 14:08:22 -0400 (EDT)
Subject: [SciPy-user] passing vector to integrate.ode
Message-ID: <alpine.LNX.2.00.0907141406110.2740@hydro11>

I'm attempting to use the scipy integrators to solve a suite of similar 
ODE's within a finite volume model (one system of equations in each grid 
cell of the model). Currently, I'm looping over each cell in the model and 
applying ode from scipy integrate. Is it possible to speed this up by 
passing vectors of my parameters to the ode method?

Here's what I'm currently doing:

---
from scipy.integrate import ode
import numpy as N

def MDiff1(t,C,k,porI,porM):
     #function for conc exchange between mobile and immobile domains
     #C[0] is mobile and and C[1] is immobile concentration
     dCI=(k/porI)*(C[0]-C[1])
     dCM=(k/porM)*(C[1]-C[0])
     return N.array([dCM,dCI])


porI=N.array([.5,.5,.5])
porM=N.array([.2,.2,.2])
dt=5000.
k=1.e-6

CMob=N.array([10.,10.,10.])
CIm=N.array([10.1,5.,1.])

r = ode(MDiff1)
r.set_integrator('vode', method='adams')

for i in range(1): #loop over time
     for j in range(3): #loop over cells
         r.set_initial_value(N.array([CMob[j],CIm[j]]))
         r.set_f_params(k,porI[j],porM[j])
         r.integrate(r.t+dt)
         print (i+1)*r.t,r.y#, r.y[0]*porM+r.y[1]*porI
         CMob[j],CIm[j]=r.y
---

In my simulation, CMob, CIm, porM, and porI are vectors with lenths in 
the thousands. Are there better (faster) ways of doing this?


Andrew Reeve
Dept. of Earth Sciences
University of Maine


From Dharhas.Pothina at twdb.state.tx.us  Tue Jul 14 14:18:35 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Tue, 14 Jul 2009 13:18:35 -0500
Subject: [SciPy-user] scikits.timeseries : Calculate previous 30	day
	and30-60 day	cumulative flows.
Message-ID: <4A5C85AB0200009B0001FF4F@GWWEB.twdb.state.tx.us>

>>> Pierre GM <pgmdevlist at gmail.com> 07/14/09 12:48 PM >>>
> Now that's a strange one. Send me more info (inflows_ts, version, ...)

I was able to recreate the error with a shorter dataset. I'm using np.__version__ = '1.3.0'
and ts.__version__ =  '0.91.1'

In [24]: inflows_ts
Out[24]: 
timeseries([(798062.75047999993,) (634009.66576,) (753657.40423999995,)
 (783260.9683999999,) (653745.37520000001,) (572335.57375999994,)
 (582203.42848,) (567401.64639999997,) (542732.00959999999,)
 (616740.91999999993,)],
   dtype = [('Inflow', '<f8')],
   dates = [01-Jan-1977 ... 10-Jan-1977],
   freq  = D)

In [25]: mov_sum(inflows_ts,2)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)

/T/SWR/BaysEstuaries/PROJECTS/SAN_BERNARD/SB_FWI_2007/SB_WETLAND_STUDY/SB_FieldData/SB_Final/regress_sbdata.py in <module>()
----> 1 
      2 
      3 
      4 
      5 

/usr/lib64/python2.5/site-packages/scikits.timeseries-0.91.1-py2.5-linux-x86_64.egg/scikits/timeseries/lib/moving_funcs.pyc in mov_sum(data, span, dtype)
    169     """
    170 
--> 171     return _mov_sum(data, span, dtype=dtype)
    172 #...............................................................................
    173 def mov_median(data, span, dtype=None):

/usr/lib64/python2.5/site-packages/scikits.timeseries-0.91.1-py2.5-linux-x86_64.egg/scikits/timeseries/lib/moving_funcs.pyc in _mov_sum(data, span, dtype, type_num_double)
    154     if dtype is not None:
    155         kwargs['dtype'] = dtype
--> 156     return _moving_func(data, MA_mov_sum, kwargs)
    157 #...............................................................................
    158 def mov_sum(data, span, dtype=None):

/usr/lib64/python2.5/site-packages/scikits.timeseries-0.91.1-py2.5-linux-x86_64.egg/scikits/timeseries/lib/moving_funcs.pyc in _moving_func(data, cfunc, kwargs)
    119 def _moving_func(data, cfunc, kwargs):
    120 
--> 121     data = ma.fix_invalid(data)
    122     data = ma.array(data.filled(0), mask=data._mask)
    123 

/usr/lib/python2.5/site-packages/numpy-1.3.0-py2.5-linux-x86_64.egg/numpy/ma/core.pyc in fix_invalid(a, mask, copy, fill_value)
    514     a = masked_array(a, copy=copy, mask=mask, subok=True)
    515     #invalid = (numpy.isnan(a._data) | numpy.isinf(a._data))
--> 516     invalid = np.logical_not(np.isfinite(a._data))
    517     if not invalid.any():
    518         return a

AttributeError: logical_not


From pgmdevlist at gmail.com  Tue Jul 14 14:31:36 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 14 Jul 2009 14:31:36 -0400
Subject: [SciPy-user] scikits.timeseries : Calculate previous 30	day
	and30-60 day	cumulative flows.
In-Reply-To: <4A5C85AB0200009B0001FF4F@GWWEB.twdb.state.tx.us>
References: <4A5C85AB0200009B0001FF4F@GWWEB.twdb.state.tx.us>
Message-ID: <C75EBCCF-03CD-4D82-AE14-13100DBD3743@gmail.com>


On Jul 14, 2009, at 2:18 PM, Dharhas Pothina wrote:

>>>> Pierre GM <pgmdevlist at gmail.com> 07/14/09 12:48 PM >>>
>> Now that's a strange one. Send me more info (inflows_ts,  
>> version, ...)
>
> I was able to recreate the error with a shorter dataset. I'm using  
> np.__version__ = '1.3.0'
> and ts.__version__ =  '0.91.1'

OK, I've been able to reproduce the bug. I gonna investigate further  
why we get an AttributeError, but here's a workaround already:
Your input series is a structured array, and a lot of numerical  
operations choke on it. You can either transform it into a non- 
structured array with .view(dtype=float), or call mov_sum on the  
'inflow' field (as in mov_sum(inflows_ts['inflow'],2))


From Dharhas.Pothina at twdb.state.tx.us  Tue Jul 14 14:34:53 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Tue, 14 Jul 2009 13:34:53 -0500
Subject: [SciPy-user] scikits.timeseries : Calculate
	previous	30	dayand30-60 day	cumulative flows.
In-Reply-To: <C75EBCCF-03CD-4D82-AE14-13100DBD3743@gmail.com>
References: <4A5C85AB0200009B0001FF4F@GWWEB.twdb.state.tx.us>
	<C75EBCCF-03CD-4D82-AE14-13100DBD3743@gmail.com>
Message-ID: <4A5C897D.63BA.009B.0@twdb.state.tx.us>


>>>> Pierre GM <pgmdevlist at gmail.com> 07/14/09 12:48 PM >>>
>> Now that's a strange one. Send me more info (inflows_ts,  
>> version, ...)

> OK, I've been able to reproduce the bug. I gonna investigate further  
> why we get an AttributeError, but here's a workaround already:
> Your input series is a structured array, and a lot of numerical  
> operations choke on it. You can either transform it into a non- 
> structured array with .view(dtype=float), or call mov_sum on the  
> 'inflow' field (as in mov_sum(inflows_ts['inflow'],2))

Ok good to know. I'll use the work around.

thanks,

- dharhas



From rowen at uw.edu  Tue Jul 14 14:33:10 2009
From: rowen at uw.edu (Russell E. Owen)
Date: Tue, 14 Jul 2009 11:33:10 -0700
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
	Error)
References: <0016e649c9263fd92c046ea0943d@google.com>
	<5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>
	<84bdef3c0907131954r8658f28x7a3dbd00dfe72614@mail.gmail.com>
	<4A5BEFBE.7040006@ar.media.kyoto-u.ac.jp>
Message-ID: <rowen-00B964.11331014072009@news.gmane.org>

In article <4A5BEFBE.7040006 at ar.media.kyoto-u.ac.jp>,
 David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:

> Rajanikanth Jammalamadaka wrote:
> > I tried it on another system (also Mac OS X) which had numpy version
> > 1.3 and it seems to work fine on that one.
> >
> > So, is it a version problem?
> >   
> 
> Yes - if you use the official scipy binary, you need to use the
> corresponding official numpy binary (which is 1.3.0 for python 2.6). As
> an alternative, you can build scipy by yourself against numpy - in this
> case, you can use numpy 1.2.1.

Except that numpy 1.3.0 is the minimum version that is compatible with 
Python 2.6.

-- Russell



From pgmdevlist at gmail.com  Tue Jul 14 14:41:46 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 14 Jul 2009 14:41:46 -0400
Subject: [SciPy-user] scikits.timeseries : Calculate previous	30
	dayand30-60 day	cumulative flows.
In-Reply-To: <4A5C897D.63BA.009B.0@twdb.state.tx.us>
References: <4A5C85AB0200009B0001FF4F@GWWEB.twdb.state.tx.us>
	<C75EBCCF-03CD-4D82-AE14-13100DBD3743@gmail.com>
	<4A5C897D.63BA.009B.0@twdb.state.tx.us>
Message-ID: <2982EC6F-A82F-4378-8E78-05171E903EF7@gmail.com>


On Jul 14, 2009, at 2:34 PM, Dharhas Pothina wrote:
>
> Ok good to know. I'll use the work around.

I checked a bit more.
ma.fix_invalid calls np.isfinite on the data part (a structured  
ndarray). Because the input is a structured array, np.isfinite returns  
a NotImplemented (a specific object of type NotImplementedType).  
np.logical_not chokes on that and raises the AttributeError exception.
I'm a bit surprised, I would have expected np.isfinite to raise a  
NotImplemented exception instead of returning something. I'll ask on  
the numpy mailing list.
cheers
P.




From d_l_goldsmith at yahoo.com  Tue Jul 14 16:19:21 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 14 Jul 2009 13:19:21 -0700 (PDT)
Subject: [SciPy-user] Quick way to delete all 'values' > from array
Message-ID: <202517.47117.qm@web52110.mail.re2.yahoo.com>


> From: Scott Sinclair <scott.sinclair.za at gmail.com>
> Subject: Re: [SciPy-user] Quick way to delete all 'values'
> from array

<snip>

> If by 'quickest' you mean 'easiest'. Here's how to do it
> using fancy indexing:
> 
> >>> import numpy as np
> >>> a = np.array([0, -1, 2, 3, -1, 4])
> >>> a
> array([ 0, -1,? 2,? 3, -1,? 4])
> >>> a = a[a != -1]
> >>> a
> array([0, 2, 3, 4])

what if he wants to eliminate more than one value at a time; is that possible?  I tried a bunch of ways, the closest I got (methinks) to success was:

>>> a[[index for index in range(len(a)) a[index] not in (0,2)]]

That gave an invalid syntax error at the third a; using a colon following the len(a)) gave an invalid syntax error at the colon; and using a comma after len(a)) gave a NameError: name 'index' not defined.  Am I just forgetting how to do conditional list comprehension, and/or is this approach doomed to failure anyway?  If the latter, is there an alternative way to do this?

Curious,
DG


      


From pgmdevlist at gmail.com  Tue Jul 14 16:49:50 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 14 Jul 2009 16:49:50 -0400
Subject: [SciPy-user] Quick way to delete all 'values' > from array
In-Reply-To: <202517.47117.qm@web52110.mail.re2.yahoo.com>
References: <202517.47117.qm@web52110.mail.re2.yahoo.com>
Message-ID: <47445786-ED66-403E-BEF9-7A80DB5ABD53@gmail.com>


On Jul 14, 2009, at 4:19 PM, David Goldsmith wrote:
>
> what if he wants to eliminate more than one value at a time; is that  
> possible?

David,
Put it that way: a ndarray occupies a fixed space in memory (set when  
you create), and you cannot add nor delete entries. What you can do is  
to create a *new* array that contains only the values that satisfy a  
given condition. It's what we're doing with the syntax
 >>> a[a!=1].
Now, if you have several entries that you want to discard, you can try  
to create a boolean array that satisfies all the conditions  
simulatenously, like
 >>> cond=np.logical_and.reduce([a!=_ for _ in (1,2)])
and use that to select the proper values
 >>> a[cond]


>  I tried a bunch of ways, the closest I got (methinks) to success was:
>




>>>> a[[index for index in range(len(a)) a[index] not in (0,2)]]
>
> That gave an invalid syntax error at the third a; using a colon  
> following the len(a)) gave an invalid syntax error at the colon; and  
> using a comma after len(a)) gave a NameError: name 'index' not  
> defined.  Am I just forgetting how to do conditional list  
> comprehension, and/or is this approach doomed to failure anyway?  If  
> the latter, is there an alternative way to do this?
>
> Curious,
> DG
>
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From sccolbert at gmail.com  Tue Jul 14 20:00:14 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Tue, 14 Jul 2009 20:00:14 -0400
Subject: [SciPy-user] non-linear multi-variate optimization
Message-ID: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>

The routines for non-linear optimization in scipy.optimize take an
argument for a function that computes the gradient.

What should be the format of return value of this function? I am
assuming that its the gradient of the functions with repect to the
independent variables in row vector format.

for example say we have:

f(x,y,z; a1, a2, a3) where a1, a2, and a3 are the independent variables.

Should the gradient of N x,y,z points then be of the form:

df/da  =   [[df(X0)/da1, df(X0)/da2, df(X0)/da3],
               [df(X1)/da1, df(X1)/da2, df(X1)/da3],
               ....
               [df(Xn)/da1, df(Xn)/da2, df(Xn)/da3]]

where Xn is the set of (xn, yn, zn) ?

cheers!

Chris


From cournape at gmail.com  Tue Jul 14 20:43:01 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 15 Jul 2009 09:43:01 +0900
Subject: [SciPy-user] python 2.6.2 mac os x (import signal causes a Bus
	Error)
In-Reply-To: <rowen-00B964.11331014072009@news.gmane.org>
References: <0016e649c9263fd92c046ea0943d@google.com>
	<5b8d13220907131911h743b445bn2529439e00db5c20@mail.gmail.com>
	<84bdef3c0907131954r8658f28x7a3dbd00dfe72614@mail.gmail.com>
	<4A5BEFBE.7040006@ar.media.kyoto-u.ac.jp>
	<rowen-00B964.11331014072009@news.gmane.org>
Message-ID: <5b8d13220907141743u2d2915dbt3e85ce541bbdfecf@mail.gmail.com>

On Wed, Jul 15, 2009 at 3:33 AM, Russell E. Owen<rowen at uw.edu> wrote:
> In article <4A5BEFBE.7040006 at ar.media.kyoto-u.ac.jp>,
> ?David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>
>> Rajanikanth Jammalamadaka wrote:
>> > I tried it on another system (also Mac OS X) which had numpy version
>> > 1.3 and it seems to work fine on that one.
>> >
>> > So, is it a version problem?
>> >
>>
>> Yes - if you use the official scipy binary, you need to use the
>> corresponding official numpy binary (which is 1.3.0 for python 2.6). As
>> an alternative, you can build scipy by yourself against numpy - in this
>> case, you can use numpy 1.2.1.
>
> Except that numpy 1.3.0 is the minimum version that is compatible with
> Python 2.6.

You can get without on mac os x if you don't care about a few bugs.
Certainly, the OP had numpy 1.2.1 installed for 2.6, after all (numpy
would have never load and pass tests under 2.6 if it was using a
version built against 2.6).

cheers,

David


From sebastian.walter at gmail.com  Wed Jul 15 03:32:55 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Wed, 15 Jul 2009 09:32:55 +0200
Subject: [SciPy-user] non-linear multi-variate optimization
In-Reply-To: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>
References: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>
Message-ID: <ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com>

The gradient g is only defined for functions f: R^N --> R and is
simply an array with shape (N,)

what you sketched in your post is the Jacobian J of a function f: R^N --> R^M
Typically, the Jacobian J is defined to have the shape (M,N), but
there are exceptions.

hope that helps a little
Sebastian



On Wed, Jul 15, 2009 at 2:00 AM, Chris Colbert<sccolbert at gmail.com> wrote:
> The routines for non-linear optimization in scipy.optimize take an
> argument for a function that computes the gradient.
>
> What should be the format of return value of this function? I am
> assuming that its the gradient of the functions with repect to the
> independent variables in row vector format.
>
> for example say we have:
>
> f(x,y,z; a1, a2, a3) where a1, a2, and a3 are the independent variables.
>
> Should the gradient of N x,y,z points then be of the form:
>
> df/da ?= ? [[df(X0)/da1, df(X0)/da2, df(X0)/da3],
> ? ? ? ? ? ? ? [df(X1)/da1, df(X1)/da2, df(X1)/da3],
> ? ? ? ? ? ? ? ....
> ? ? ? ? ? ? ? [df(Xn)/da1, df(Xn)/da2, df(Xn)/da3]]
>
> where Xn is the set of (xn, yn, zn) ?
>
> cheers!
>
> Chris
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From vanforeest at gmail.com  Wed Jul 15 08:29:48 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Wed, 15 Jul 2009 14:29:48 +0200
Subject: [SciPy-user] comparing two lists/arrays
Message-ID: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>

Hi,

Given two vectors x and y, the (perhaps) common mathematical
definition of x < y is that x_i < y_i for all i.  Thus, the
mathematical comparison x <y returns just one boolean, not an array of
booleans for each x_i < y_i.  I implemented this behavior as
prod(less(X,Y)) (I use less to be able to deal with lists X and Y
also). Is there perhaps a more straighforward/elegant/readible way to
achieve the same behavior?

Thanks

Nicky


From cimrman3 at ntc.zcu.cz  Wed Jul 15 08:32:02 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 15 Jul 2009 14:32:02 +0200
Subject: [SciPy-user] comparing two lists/arrays
In-Reply-To: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>
References: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>
Message-ID: <4A5DCC42.4080003@ntc.zcu.cz>

Hi Nicky,

nicky van foreest wrote:
> Hi,
> 
> Given two vectors x and y, the (perhaps) common mathematical
> definition of x < y is that x_i < y_i for all i.  Thus, the
> mathematical comparison x <y returns just one boolean, not an array of
> booleans for each x_i < y_i.  I implemented this behavior as
> prod(less(X,Y)) (I use less to be able to deal with lists X and Y
> also). Is there perhaps a more straighforward/elegant/readible way to
> achieve the same behavior?

assuming x, y are numpy arrays: (x < y).all()

r.


From scott.sinclair.za at gmail.com  Wed Jul 15 08:43:26 2009
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Wed, 15 Jul 2009 14:43:26 +0200
Subject: [SciPy-user] comparing two lists/arrays
In-Reply-To: <4A5DCC42.4080003@ntc.zcu.cz>
References: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>
	<4A5DCC42.4080003@ntc.zcu.cz>
Message-ID: <6a17e9ee0907150543o4fac8471off2f6fb1214791af@mail.gmail.com>

> 2009/7/15 Robert Cimrman <cimrman3 at ntc.zcu.cz>:
> nicky van foreest wrote:
>> Given two vectors x and y, the (perhaps) common mathematical
>> definition of x < y is that x_i < y_i for all i. ?Thus, the
>> mathematical comparison x <y returns just one boolean, not an array of
>> booleans for each x_i < y_i. ?I implemented this behavior as
>> prod(less(X,Y)) (I use less to be able to deal with lists X and Y
>> also). Is there perhaps a more straighforward/elegant/readible way to
>> achieve the same behavior?
>
> assuming x, y are numpy arrays: (x < y).all()

You could do the following to handle the case where they aren't:

>>> import numpy as np
>>> x = range(10)
>>> y = range(1, 11)
>>> np.all(x < y)
True

Cheers,
Scott


From scott.sinclair.za at gmail.com  Wed Jul 15 08:48:18 2009
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Wed, 15 Jul 2009 14:48:18 +0200
Subject: [SciPy-user] comparing two lists/arrays
In-Reply-To: <6a17e9ee0907150543o4fac8471off2f6fb1214791af@mail.gmail.com>
References: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>
	<4A5DCC42.4080003@ntc.zcu.cz>
	<6a17e9ee0907150543o4fac8471off2f6fb1214791af@mail.gmail.com>
Message-ID: <6a17e9ee0907150548n18c9eb95h17ad20c9e2c5e31d@mail.gmail.com>

>2009/7/15 Scott Sinclair <scott.sinclair.za at gmail.com>:
>> 2009/7/15 Robert Cimrman <cimrman3 at ntc.zcu.cz>:
>> nicky van foreest wrote:
>>> Given two vectors x and y, the (perhaps) common mathematical
>>> definition of x < y is that x_i < y_i for all i. ?Thus, the
>>> mathematical comparison x <y returns just one boolean, not an array of
>>> booleans for each x_i < y_i. ?I implemented this behavior as
>>> prod(less(X,Y)) (I use less to be able to deal with lists X and Y
>>> also). Is there perhaps a more straighforward/elegant/readible way to
>>> achieve the same behavior?
>>
>> assuming x, y are numpy arrays: (x < y).all()
>
> You could do the following to handle the case where they aren't:
>
>>>> import numpy as np
>>>> x = range(10)
>>>> y = range(1, 11)
>>>> np.all(x < y)
> True

Scratch that

>>> x < y
True

S


From cimrman3 at ntc.zcu.cz  Wed Jul 15 08:53:57 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Wed, 15 Jul 2009 14:53:57 +0200
Subject: [SciPy-user] comparing two lists/arrays
In-Reply-To: <6a17e9ee0907150548n18c9eb95h17ad20c9e2c5e31d@mail.gmail.com>
References: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>	<4A5DCC42.4080003@ntc.zcu.cz>	<6a17e9ee0907150543o4fac8471off2f6fb1214791af@mail.gmail.com>
	<6a17e9ee0907150548n18c9eb95h17ad20c9e2c5e31d@mail.gmail.com>
Message-ID: <4A5DD165.2000707@ntc.zcu.cz>

Scott Sinclair wrote:
>> 2009/7/15 Scott Sinclair <scott.sinclair.za at gmail.com>:
>>> 2009/7/15 Robert Cimrman <cimrman3 at ntc.zcu.cz>:
>>> nicky van foreest wrote:
>>>> Given two vectors x and y, the (perhaps) common mathematical
>>>> definition of x < y is that x_i < y_i for all i.  Thus, the
>>>> mathematical comparison x <y returns just one boolean, not an array of
>>>> booleans for each x_i < y_i.  I implemented this behavior as
>>>> prod(less(X,Y)) (I use less to be able to deal with lists X and Y
>>>> also). Is there perhaps a more straighforward/elegant/readible way to
>>>> achieve the same behavior?
>>> assuming x, y are numpy arrays: (x < y).all()
>> You could do the following to handle the case where they aren't:
>>
>>>>> import numpy as np
>>>>> x = range(10)
>>>>> y = range(1, 11)
>>>>> np.all(x < y)
>> True
> 
> Scratch that
> 
>>>> x < y
> True

beware!

y[2] = -1

In [21]: y
Out[21]: [1, 2, -1, 4, 5, 6, 7, 8, 9, 10]

In [22]: x < y
Out[22]: True



From michael at ini.phys.ethz.ch  Wed Jul 15 09:00:25 2009
From: michael at ini.phys.ethz.ch (Michael Graber)
Date: Wed, 15 Jul 2009 15:00:25 +0200
Subject: [SciPy-user] reading (labview) binary data
Message-ID: <FCEBCCF4-E856-46D5-B353-C924CFF29BA8@ini.phys.ethz.ch>


hi all,


i'm looking for the fastest way to read (labview) binary files.

i figured out a method which is basically the same as described on:

http://www.shocksolution.com/2008/06/25/reading-labview-binary-files-with-python/

that is:

binaryFile = open("Measurement_4.bin", mode='rb')
(data.offset,) = struct.unpack('>d', binaryFile.read(8))

but this is terribly slow. to read 12 MB takes about 80 seconds.  
matlab is orders of magnitude faster ..

is there a better, i.e. faster way to read in (labview) binary files?


thanks,
michael


From emanuele at relativita.com  Wed Jul 15 09:47:17 2009
From: emanuele at relativita.com (Emanuele Olivetti)
Date: Wed, 15 Jul 2009 15:47:17 +0200
Subject: [SciPy-user] reading (labview) binary data
In-Reply-To: <FCEBCCF4-E856-46D5-B353-C924CFF29BA8@ini.phys.ethz.ch>
References: <FCEBCCF4-E856-46D5-B353-C924CFF29BA8@ini.phys.ethz.ch>
Message-ID: <4A5DDDE5.1050504@relativita.com>

Michael Graber wrote:
> hi all,
>
>
> i'm looking for the fastest way to read (labview) binary files.
>
> i figured out a method which is basically the same as described on:
>
> http://www.shocksolution.com/2008/06/25/reading-labview-binary-files-with-python/
>
> that is:
>
> binaryFile = open("Measurement_4.bin", mode='rb')
> (data.offset,) = struct.unpack('>d', binaryFile.read(8))
>
> but this is terribly slow. to read 12 MB takes about 80 seconds.  
> matlab is orders of magnitude faster ..
>
> is there a better, i.e. faster way to read in (labview) binary files?
>
>
>   

Try numpy.fromfile(). Example:

import numpy as np
f = open('<filename>','rb')
data = np.fromfile(f, dtype='>d') # reads the whole file
f.close()
print data.shape

'data' should now be a vector of big-endian doubles from the whole file.
If you want to parse just a subset of all doubles within the file use the
optional argument 'count='. See np.fromfile() docstring.

Cheers,

Emanuele




From david.huard at gmail.com  Wed Jul 15 09:56:08 2009
From: david.huard at gmail.com (David Huard)
Date: Wed, 15 Jul 2009 09:56:08 -0400
Subject: [SciPy-user] reading (labview) binary data
In-Reply-To: <FCEBCCF4-E856-46D5-B353-C924CFF29BA8@ini.phys.ethz.ch>
References: <FCEBCCF4-E856-46D5-B353-C924CFF29BA8@ini.phys.ethz.ch>
Message-ID: <91cf711d0907150656v417e2aafyf9d28588241b535b@mail.gmail.com>

Michael,

I have no experience with Labview files, but for uniform data structures, I
found numpy.fromfile and numpy.memmap to be more practical than
struct.unpack.

HTH,

David

On Wed, Jul 15, 2009 at 9:00 AM, Michael Graber <michael at ini.phys.ethz.ch>wrote:

>
> hi all,
>
>
> i'm looking for the fastest way to read (labview) binary files.
>
> i figured out a method which is basically the same as described on:
>
>
> http://www.shocksolution.com/2008/06/25/reading-labview-binary-files-with-python/
>
> that is:
>
> binaryFile = open("Measurement_4.bin", mode='rb')
> (data.offset,) = struct.unpack('>d', binaryFile.read(8))
>
> but this is terribly slow. to read 12 MB takes about 80 seconds.
> matlab is orders of magnitude faster ..
>
> is there a better, i.e. faster way to read in (labview) binary files?
>
>
> thanks,
> michael
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090715/14f19942/attachment.html>

From sccolbert at gmail.com  Wed Jul 15 10:13:27 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Wed, 15 Jul 2009 10:13:27 -0400
Subject: [SciPy-user] comparing two lists/arrays
In-Reply-To: <4A5DD165.2000707@ntc.zcu.cz>
References: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>
	<4A5DCC42.4080003@ntc.zcu.cz>
	<6a17e9ee0907150543o4fac8471off2f6fb1214791af@mail.gmail.com>
	<6a17e9ee0907150548n18c9eb95h17ad20c9e2c5e31d@mail.gmail.com>
	<4A5DD165.2000707@ntc.zcu.cz>
Message-ID: <7f014ea60907150713i21259ac0q3a96dcb4bef9ecc5@mail.gmail.com>

this should work for any case:

>>> y = np.random.rand(5)
>>> x = np.random.rand(5)

>>> y
array([ 0.21991179,  0.82874802,  0.65327351,  0.02277029,  0.14618527])

>>> x
array([ 0.46541554,  0.86719123,  0.50618409,  0.13140126,  0.24533278])

>>> (y - x) > 0
array([False, False,  True, False, False], dtype=bool)


On Wed, Jul 15, 2009 at 8:53 AM, Robert Cimrman<cimrman3 at ntc.zcu.cz> wrote:
> Scott Sinclair wrote:
>>> 2009/7/15 Scott Sinclair <scott.sinclair.za at gmail.com>:
>>>> 2009/7/15 Robert Cimrman <cimrman3 at ntc.zcu.cz>:
>>>> nicky van foreest wrote:
>>>>> Given two vectors x and y, the (perhaps) common mathematical
>>>>> definition of x < y is that x_i < y_i for all i. ?Thus, the
>>>>> mathematical comparison x <y returns just one boolean, not an array of
>>>>> booleans for each x_i < y_i. ?I implemented this behavior as
>>>>> prod(less(X,Y)) (I use less to be able to deal with lists X and Y
>>>>> also). Is there perhaps a more straighforward/elegant/readible way to
>>>>> achieve the same behavior?
>>>> assuming x, y are numpy arrays: (x < y).all()
>>> You could do the following to handle the case where they aren't:
>>>
>>>>>> import numpy as np
>>>>>> x = range(10)
>>>>>> y = range(1, 11)
>>>>>> np.all(x < y)
>>> True
>>
>> Scratch that
>>
>>>>> x < y
>> True
>
> beware!
>
> y[2] = -1
>
> In [21]: y
> Out[21]: [1, 2, -1, 4, 5, 6, 7, 8, 9, 10]
>
> In [22]: x < y
> Out[22]: True
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From vanforeest at gmail.com  Wed Jul 15 13:59:51 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Wed, 15 Jul 2009 19:59:51 +0200
Subject: [SciPy-user] comparing two lists/arrays
In-Reply-To: <7f014ea60907150713i21259ac0q3a96dcb4bef9ecc5@mail.gmail.com>
References: <fa510ff80907150529u62e8cb44r6ac0f9564e40b39b@mail.gmail.com>
	<4A5DCC42.4080003@ntc.zcu.cz>
	<6a17e9ee0907150543o4fac8471off2f6fb1214791af@mail.gmail.com>
	<6a17e9ee0907150548n18c9eb95h17ad20c9e2c5e31d@mail.gmail.com>
	<4A5DD165.2000707@ntc.zcu.cz>
	<7f014ea60907150713i21259ac0q3a96dcb4bef9ecc5@mail.gmail.com>
Message-ID: <fa510ff80907151059w3c40daa6nb955ac8d39594de1@mail.gmail.com>

Hi,

Thanks for your advice.

Nicky

2009/7/15 Chris Colbert <sccolbert at gmail.com>:
> this should work for any case:
>
>>>> y = np.random.rand(5)
>>>> x = np.random.rand(5)
>
>>>> y
> array([ 0.21991179, ?0.82874802, ?0.65327351, ?0.02277029, ?0.14618527])
>
>>>> x
> array([ 0.46541554, ?0.86719123, ?0.50618409, ?0.13140126, ?0.24533278])
>
>>>> (y - x) > 0
> array([False, False, ?True, False, False], dtype=bool)
>
>
> On Wed, Jul 15, 2009 at 8:53 AM, Robert Cimrman<cimrman3 at ntc.zcu.cz> wrote:
>> Scott Sinclair wrote:
>>>> 2009/7/15 Scott Sinclair <scott.sinclair.za at gmail.com>:
>>>>> 2009/7/15 Robert Cimrman <cimrman3 at ntc.zcu.cz>:
>>>>> nicky van foreest wrote:
>>>>>> Given two vectors x and y, the (perhaps) common mathematical
>>>>>> definition of x < y is that x_i < y_i for all i. ?Thus, the
>>>>>> mathematical comparison x <y returns just one boolean, not an array of
>>>>>> booleans for each x_i < y_i. ?I implemented this behavior as
>>>>>> prod(less(X,Y)) (I use less to be able to deal with lists X and Y
>>>>>> also). Is there perhaps a more straighforward/elegant/readible way to
>>>>>> achieve the same behavior?
>>>>> assuming x, y are numpy arrays: (x < y).all()
>>>> You could do the following to handle the case where they aren't:
>>>>
>>>>>>> import numpy as np
>>>>>>> x = range(10)
>>>>>>> y = range(1, 11)
>>>>>>> np.all(x < y)
>>>> True
>>>
>>> Scratch that
>>>
>>>>>> x < y
>>> True
>>
>> beware!
>>
>> y[2] = -1
>>
>> In [21]: y
>> Out[21]: [1, 2, -1, 4, 5, 6, 7, 8, 9, 10]
>>
>> In [22]: x < y
>> Out[22]: True
>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From gael.varoquaux at normalesup.org  Wed Jul 15 22:13:11 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 16 Jul 2009 04:13:11 +0200
Subject: [SciPy-user] [ANN] Announcing the SciPy conference schedule
Message-ID: <20090716021311.GA21642@phare.normalesup.org>

The SciPy conference committee is pleased to announce the schedule of the
conference:

http://conference.scipy.org/schedule

This year?s program is very rich. In order to limit the number of
interesting talks that we had to turn down, we decided to reduce the
length of talks. Although this results in many short talks, we hope that
it will foster discussions, and give new ideas. Many subjects are
covered, both varying technical subject in the scientific computing
spectrum, and covering a lot of different research areas.

I would personally like to thank the members of the program committee,
who spent time reviewing the proposed abstracts and giving the chairs
feedback.

Fernando Perez and the tutorial presenters are hard at work finishing
planning all the details of the two-day tutorial session that will
precede the conference. An introduction tutorial track and an advanced
tutorial track, both covering various aspect of scientific computing in
Python, presented by experts in the field, should help many people
getting up to speed on the amazing technology driving this community.

The SciPy 2009 program committee

    * Co-Chair Ga?l Varoquaux, Applied Mathematics and Neuroscience,
    * Neurospin, CEA - INRIA Saclay (France)
    * Co-Chair St?fan van der Walt, Applied Mathematics, University of
    * Stellenbosch (South Africa)
    * Michael Aivazis, Center for Advanced Computing Research, California
    * Institute of Technology (USA)
    * Brian Granger, Physics Department, California Polytechnic State
    * University, San Luis Obispo (USA)
    * Aric Hagberg, Theoretical Division, Los Alamos National Laboratory
    * (USA)
    * Konrad Hinsen, Centre de Biophysique Mol?culaire, CNRS Orl?ans
    * (France)
    * Randall LeVeque, Mathematics, University of Washington, Seattle
    * (USA)
    * Travis Oliphant, Enthought (USA)
    * Prabhu Ramachandran, Department of Aerospace Engineering, IIT
    * Bombay (India)
    * Raphael Ritz, International Neuroinformatics Coordinating Facility
    * (Sweden)
    * William Stein, Mathematics, University of Washington, Seattle (USA)

Conference Chair: Jarrod Millman, Neuroscience Institute, UC Berkeley
(USA)



From Dharhas.Pothina at twdb.state.tx.us  Thu Jul 16 08:53:25 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Thu, 16 Jul 2009 07:53:25 -0500
Subject: [SciPy-user] scikits.timeseries: unable to stack timeseries with
	different	number of columns
Message-ID: <4A5EDC75.63BA.009B.0@twdb.state.tx.us>

Hi,

I'm trying to loop through a set of timeseries objects and add each timeseries to a single timeseries using ts.stack(). I'm getting the following error:

TimeseriesCompatibility Error: Incompatible Sizes! ( 1: (432,) <> 2: (432,4) )

from what I can see I can only stack timeseries with the exact same dimensions. ie if I have a timeseries 
a_ts with dimensions (432,) and b_ts with dimensions (432,4) I can use:

ts.stack(a_ts,a_ts) or ts.stack(b_ts,b_ts)

but I cannot do ts.stack(a_ts,b_ts)

Is this correct and if it is there a workaround?

thanks

- dharhas



From Dharhas.Pothina at twdb.state.tx.us  Thu Jul 16 09:08:07 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Thu, 16 Jul 2009 08:08:07 -0500
Subject: [SciPy-user] scikits.timeseries : Problem using tsfromtxt with
	converter
Message-ID: <4A5EDFE70200009B0002007A@GWWEB.twdb.state.tx.us>

Hi,

  I have data in the following format:

Date, 28.898 -95.334
09/01/2008 00:00, 0mm
09/01/2008 01:00, 0mm
09/01/2008 02:00, 0mm
...

If I read it using 

data = ts.tsfromtxt(file,skiprows=1,datecols=(0),delimiter=',',freq = 'H')

It reads in correctly but the data is read in as strings with the 'mm'. ie
timeseries(
 [[ 0mm]
 [ 0mm]
 ..., 
 [ 0mm]],
    dates =
 [01-Sep-2008 00:00 ... 01-Jun-2009 23:00],
    freq  = H)

I want to remove the 'mm' and read the data as floats. I tried using a converter:

conv = {1: lambda s: s.split('m')[0]}
data = ts.tsfromtxt(file,skiprows=1,datecols=(0),delimiter=',',freq = 'H',converters=conv)

but then I get an error which seems to have something to do with the date converter that was working in the earlier command:

/usr/lib64/python2.5/site-packages/scikits.timeseries-0.91.1-py2.5-linux-x86_64.egg/scikits/timeseries/extras.pyc in tsfromtxt(fname, dtype, freq, comments, delimiter, skiprows, converters, dateconverter, missing, missing_values, usecols, datecols, names, excludelist, deletechars, case_sensitive, unpack, loose, asrecarray)
    433     if len(dateinfo) == 1:
    434         dateinfo = np.array(dateinfo[0], copy=False, ndmin=1)
--> 435         dates = date_array([dateconv(args) for args in dateinfo],
    436                            freq=freq, autosort=False)
    437     else:

UnboundLocalError: local variable 'dateconv' referenced before assignment

What am I doing wrong?

thanks

- dharhas







From eadrogue at gmx.net  Thu Jul 16 10:01:54 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Thu, 16 Jul 2009 16:01:54 +0200
Subject: [SciPy-user] counting pairs of values
Message-ID: <20090716140154.GA25712@doriath.local>

Hi,

I've got a series of observations that consist of pairs of
values (x, y) and I'm interested in counting the number of
occurrences of arbitrary pairs of values.

For example, if "a" is a 2-dimensional array such as

array([[1, 2],
       [3, 4],
       [5, 5],
       [1, 2]])

I want to find out how many (1, 2), or how many (3, 3) there
are.

Currently I'm doing this:

numpy.logical_and(a[:,0] == 1, a[:,1] == 2).sum()

which seems to work well enough, but I wonder if there's a
more elegant/correct way of doing it.


Ernest


From sccolbert at gmail.com  Thu Jul 16 10:31:24 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Thu, 16 Jul 2009 10:31:24 -0400
Subject: [SciPy-user] counting pairs of values
In-Reply-To: <20090716140154.GA25712@doriath.local>
References: <20090716140154.GA25712@doriath.local>
Message-ID: <7f014ea60907160731g7d7a4954j1546fde2c7acac42@mail.gmail.com>

I don't know if its any more elegant than yours though:

>>> a
array([[1, 2],
       [3, 4],
       [5, 5],
       [1, 2]])
>>> len(np.where(a==[1, 2])[0])/2
2
>>> len(np.where(a==[3, 4])[0])/2
1
>>> len(np.where(a==[5, 5])[0])/2
1



2009/7/16 Ernest Adrogu? <eadrogue at gmx.net>:
> Hi,
>
> I've got a series of observations that consist of pairs of
> values (x, y) and I'm interested in counting the number of
> occurrences of arbitrary pairs of values.
>
> For example, if "a" is a 2-dimensional array such as
>
> array([[1, 2],
> ? ? ? [3, 4],
> ? ? ? [5, 5],
> ? ? ? [1, 2]])
>
> I want to find out how many (1, 2), or how many (3, 3) there
> are.
>
> Currently I'm doing this:
>
> numpy.logical_and(a[:,0] == 1, a[:,1] == 2).sum()
>
> which seems to work well enough, but I wonder if there's a
> more elegant/correct way of doing it.
>
>
> Ernest
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From zachary.pincus at yale.edu  Thu Jul 16 10:48:55 2009
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Thu, 16 Jul 2009 10:48:55 -0400
Subject: [SciPy-user] counting pairs of values
In-Reply-To: <20090716140154.GA25712@doriath.local>
References: <20090716140154.GA25712@doriath.local>
Message-ID: <B721573D-0215-4FDB-B9A2-28B94C90B3A5@yale.edu>

Check out the discussion from yesterday on the numpy list "Neighbour- 
frequency matrix" for a generalization of this problem. You could also  
use numpy.histogram2d for this.

Zach


On Jul 16, 2009, at 10:01 AM, Ernest Adrogu? wrote:

> Hi,
>
> I've got a series of observations that consist of pairs of
> values (x, y) and I'm interested in counting the number of
> occurrences of arbitrary pairs of values.
>
> For example, if "a" is a 2-dimensional array such as
>
> array([[1, 2],
>       [3, 4],
>       [5, 5],
>       [1, 2]])
>
> I want to find out how many (1, 2), or how many (3, 3) there
> are.
>
> Currently I'm doing this:
>
> numpy.logical_and(a[:,0] == 1, a[:,1] == 2).sum()
>
> which seems to work well enough, but I wonder if there's a
> more elegant/correct way of doing it.
>
>
> Ernest
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From pgmdevlist at gmail.com  Thu Jul 16 17:21:22 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 16 Jul 2009 17:21:22 -0400
Subject: [SciPy-user] scikits.timeseries: unable to stack timeseries
	with different	number of columns
In-Reply-To: <4A5EDC75.63BA.009B.0@twdb.state.tx.us>
References: <4A5EDC75.63BA.009B.0@twdb.state.tx.us>
Message-ID: <3FE87F21-23D5-43E7-A04E-0983DA249567@gmail.com>


On Jul 16, 2009, at 8:53 AM, Dharhas Pothina wrote:

> Hi,
>
> I'm trying to loop through a set of timeseries objects and add each  
> timeseries to a single timeseries using ts.stack(). I'm getting the  
> following error:

[...]
That's a bug all right. stack was initially intended for 1D series,  
not 2D ones. I'll check what I can do.

Meanwhile, you could:
* make sure that your series are compatible: same frequency, same  
starting and endings dates, same intervals between the dates
* stack the series using ma.column_stack : that gives you a masked array
* view the output as a TimeSeries, and set its dates to the dates of  
the input.




From pgmdevlist at gmail.com  Thu Jul 16 17:27:31 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 16 Jul 2009 17:27:31 -0400
Subject: [SciPy-user] scikits.timeseries : Problem using tsfromtxt with
	converter
In-Reply-To: <4A5EDFE70200009B0002007A@GWWEB.twdb.state.tx.us>
References: <4A5EDFE70200009B0002007A@GWWEB.twdb.state.tx.us>
Message-ID: <A77581A3-9F8D-416A-9A50-E891C23CE62E@gmail.com>


On Jul 16, 2009, at 9:08 AM, Dharhas Pothina wrote:

> Hi,
>
> If I read it using
>
> data = ts.tsfromtxt(file,skiprows=1,datecols=(0),delimiter=',',freq  
> = 'H')
>
> ...
> I want to remove the 'mm' and read the data as floats. I tried using  
> a converter:
>
> conv = {1: lambda s: s.split('m')[0]}
> data = ts.tsfromtxt(file,skiprows=1,datecols=(0),delimiter=',',freq  
> = 'H',converters=conv)
>
> but then I get an error which seems to have something to do with the  
> date converter that was working in the earlier command:
>
> /usr/lib64/python2.5/site-packages/scikits.timeseries-0.91.1-py2.5- 
> linux-x86_64.egg/scikits/timeseries/extras.pyc in tsfromtxt(fname,  
> dtype, freq, comments, delimiter, skiprows, converters,  
> dateconverter, missing, missing_values, usecols, datecols, names,  
> excludelist, deletechars, case_sensitive, unpack, loose, asrecarray)
>    433     if len(dateinfo) == 1:
>    434         dateinfo = np.array(dateinfo[0], copy=False, ndmin=1)
> --> 435         dates = date_array([dateconv(args) for args in  
> dateinfo],
>    436                            freq=freq, autosort=False)
>    437     else:
>
> UnboundLocalError: local variable 'dateconv' referenced before  
> assignment
>
> What am I doing wrong?

Nothing. It's a bug, sorry about that. I'll try to correct that later  
today or tmw at worst.

Thanks for reporting,
Cheers
P.


From william.ratcliff at gmail.com  Fri Jul 17 03:38:43 2009
From: william.ratcliff at gmail.com (william ratcliff)
Date: Fri, 17 Jul 2009 03:38:43 -0400
Subject: [SciPy-user] [Numpy-discussion] [ANN] Announcing the SciPy
	conference schedule
In-Reply-To: <20090716021311.GA21642@phare.normalesup.org>
References: <20090716021311.GA21642@phare.normalesup.org>
Message-ID: <827183970907170038jef07198r867b086a284e8b4a@mail.gmail.com>

A humble suggestion--for the March meeting of the american physical society,
there is a roommate finder for splitting hotel rooms.  This could be useful
in keeping expenses down for some.  There should be a way to do it without
liability....
Cheers,
William

On Wed, Jul 15, 2009 at 10:13 PM, Gael Varoquaux <
gael.varoquaux at normalesup.org> wrote:

> The SciPy conference committee is pleased to announce the schedule of the
> conference:
>
> http://conference.scipy.org/schedule
>
> This year?s program is very rich. In order to limit the number of
> interesting talks that we had to turn down, we decided to reduce the
> length of talks. Although this results in many short talks, we hope that
> it will foster discussions, and give new ideas. Many subjects are
> covered, both varying technical subject in the scientific computing
> spectrum, and covering a lot of different research areas.
>
> I would personally like to thank the members of the program committee,
> who spent time reviewing the proposed abstracts and giving the chairs
> feedback.
>
> Fernando Perez and the tutorial presenters are hard at work finishing
> planning all the details of the two-day tutorial session that will
> precede the conference. An introduction tutorial track and an advanced
> tutorial track, both covering various aspect of scientific computing in
> Python, presented by experts in the field, should help many people
> getting up to speed on the amazing technology driving this community.
>
> The SciPy 2009 program committee
>
>    * Co-Chair Ga?l Varoquaux, Applied Mathematics and Neuroscience,
>    * Neurospin, CEA - INRIA Saclay (France)
>    * Co-Chair St?fan van der Walt, Applied Mathematics, University of
>    * Stellenbosch (South Africa)
>    * Michael Aivazis, Center for Advanced Computing Research, California
>    * Institute of Technology (USA)
>    * Brian Granger, Physics Department, California Polytechnic State
>    * University, San Luis Obispo (USA)
>    * Aric Hagberg, Theoretical Division, Los Alamos National Laboratory
>    * (USA)
>    * Konrad Hinsen, Centre de Biophysique Mol?culaire, CNRS Orl?ans
>    * (France)
>    * Randall LeVeque, Mathematics, University of Washington, Seattle
>    * (USA)
>    * Travis Oliphant, Enthought (USA)
>    * Prabhu Ramachandran, Department of Aerospace Engineering, IIT
>    * Bombay (India)
>    * Raphael Ritz, International Neuroinformatics Coordinating Facility
>    * (Sweden)
>    * William Stein, Mathematics, University of Washington, Seattle (USA)
>
> Conference Chair: Jarrod Millman, Neuroscience Institute, UC Berkeley
> (USA)
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090717/95b2d7d0/attachment.html>

From amenity at enthought.com  Fri Jul 17 10:52:58 2009
From: amenity at enthought.com (Amenity Applewhite)
Date: Fri, 17 Jul 2009 09:52:58 -0500
Subject: [SciPy-user] Today's Scientific Computing Webinar
Message-ID: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>

Registration for today's Scientific Computing with Python webinar will  
be open until the webinar begins at 1:00pm CDT (6pm UTC). Join us!

https://www1.gotomeeting.com/register/158182457
July 17th: Step-by-step Chaco

On Friday, July 17th, we'll be taking a look at Chaco, an component of  
our open-source Enthought Tool Suite.

It seems that this powerful tool impressed participants at last  
month's webinar, as numerous attendees responded with "Cool!! But how  
can I do that?" So Peter Wang, Enthought's Director of Technical  
Architecture, has offered to pick up where he left off last month and  
give a step-by step introduction to Chaco. This will include a closer  
examination of the code and more in-depth guidance on how use it for  
visualizing your data with 2D plotting.

This event will take place on Friday, July 17th at 1:00pm CDT (6pm  
UTC) and will last 60 to 90 minutes, depending on the questions asked.  
If you would like to participate, please register at https://www1.gotomeeting.com/register/158182457 
.

Can't get enough scientific computing with Python?

This Scientific Computing with Python series focuses on the free and  
open-source Enthought Tool Suite and and should not be confused with  
our EPD webinar series. The latter is meant to provide customized  
support for our EPD Basic or above subscribers, and we initially  
intended for it to be closed to the public. Many non-subscribers have  
expressed interest in the series, however, so we have decided to allow  
non-subscribers to add their name to a waiting list for each EPD  
webinar. While EPD subscribers will be guaranteed entry and attendance  
will be capped at 35, we'll draw names from the waiting lists to  
available seats. These webinars are more intimate, with participatory  
demonstrations and VOIP question and answer available to attendees,  
and we think it's a great opportunity for us to support the broader  
SciPy community. For details, seehttp://www.enthought.com/training/webinars.php 
.

Thanks, and see you Friday!

The Enthought Team
--
Amenity Applewhite
Enthought, Inc.
Scientific Computing Solutions
www.enthought.com








-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090717/15c979c6/attachment.html>

From robert.kern at gmail.com  Fri Jul 17 11:21:34 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 17 Jul 2009 10:21:34 -0500
Subject: [SciPy-user] [Numpy-discussion] [ANN] Announcing the SciPy
	conference schedule
In-Reply-To: <827183970907170038jef07198r867b086a284e8b4a@mail.gmail.com>
References: <20090716021311.GA21642@phare.normalesup.org>
	<827183970907170038jef07198r867b086a284e8b4a@mail.gmail.com>
Message-ID: <3d375d730907170821l32139208ob3410bd9c9520c01@mail.gmail.com>

On Fri, Jul 17, 2009 at 02:38, william
ratcliff<william.ratcliff at gmail.com> wrote:
> A humble suggestion--for the March meeting of the american physical society,
> there is a roommate finder for splitting hotel rooms. ?This could be useful
> in keeping expenses down for some. ?There should be a way to do it without
> liability....

A wiki page would probably be the best thing given the short time
frame. I recommend either the Saga or the Vagabond hotels for keeping
costs down and staying close to campus.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From Dharhas.Pothina at twdb.state.tx.us  Fri Jul 17 12:18:24 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Fri, 17 Jul 2009 11:18:24 -0500
Subject: [SciPy-user] scikits.timeseries: unable to stack	timeserieswith
	different	number of columns
In-Reply-To: <3FE87F21-23D5-43E7-A04E-0983DA249567@gmail.com>
References: <4A5EDC75.63BA.009B.0@twdb.state.tx.us>
	<3FE87F21-23D5-43E7-A04E-0983DA249567@gmail.com>
Message-ID: <4A605E00.63BA.009B.0@twdb.state.tx.us>



>>> Pierre GM <pgmdevlist at gmail.com> 7/16/2009 4:21 PM >>>

> Meanwhile, you could:
> * make sure that your series are compatible: same frequency, same  
> starting and endings dates, same intervals between the dates
> * stack the series using ma.column_stack : that gives you a masked array
> * view the output as a TimeSeries, and set its dates to the dates of  
> the input.

not sure if I completely understood this.

so if I have a_ts and b_ts that are timeseries objects I do 

c = ma.stack_column((a_ts,b_ts))

then how do I "view the output as a TimeSeries, and set its dates to the dates of the input"

- dharhas



From Dharhas.Pothina at twdb.state.tx.us  Fri Jul 17 12:19:17 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Fri, 17 Jul 2009 11:19:17 -0500
Subject: [SciPy-user] scikits.timeseries : Problem using
	tsfromtxt	withconverter
In-Reply-To: <A77581A3-9F8D-416A-9A50-E891C23CE62E@gmail.com>
References: <4A5EDFE70200009B0002007A@GWWEB.twdb.state.tx.us>
	<A77581A3-9F8D-416A-9A50-E891C23CE62E@gmail.com>
Message-ID: <4A605E35.63BA.009B.0@twdb.state.tx.us>


> Nothing. It's a bug, sorry about that. I'll try to correct that later  
> today or tmw at worst.
> 
> Thanks for reporting,

glad to be of help.

- dharhas



From pgmdevlist at gmail.com  Fri Jul 17 12:25:23 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Fri, 17 Jul 2009 12:25:23 -0400
Subject: [SciPy-user] scikits.timeseries: unable to stack	timeserieswith
	different	number of columns
In-Reply-To: <4A605E00.63BA.009B.0@twdb.state.tx.us>
References: <4A5EDC75.63BA.009B.0@twdb.state.tx.us>
	<3FE87F21-23D5-43E7-A04E-0983DA249567@gmail.com>
	<4A605E00.63BA.009B.0@twdb.state.tx.us>
Message-ID: <57965819-8A49-4048-82F5-406A3647CB54@gmail.com>


On Jul 17, 2009, at 12:18 PM, Dharhas Pothina wrote:

>
>
>>>> Pierre GM <pgmdevlist at gmail.com> 7/16/2009 4:21 PM >>>
>
>> Meanwhile, you could:
>> * make sure that your series are compatible: same frequency, same
>> starting and endings dates, same intervals between the dates
>> * stack the series using ma.column_stack : that gives you a masked  
>> array
>> * view the output as a TimeSeries, and set its dates to the dates of
>> the input.
>
> not sure if I completely understood this.
>
> so if I have a_ts and b_ts that are timeseries objects I do
>
> c = ma.stack_column((a_ts,b_ts))
>
> then how do I "view the output as a TimeSeries, and set its dates to  
> the dates of the input"


Provided that a_ts and b_ts are compatible (see previous post):
 >>> c_ts = ma.column_stack((a_ts, b_ts))
 >>> c_ts = c_ts..view(ts.TimeSeries)
 >>> c_ts.dates = a_ts.dates




From millman at berkeley.edu  Fri Jul 17 16:02:51 2009
From: millman at berkeley.edu (Jarrod Millman)
Date: Fri, 17 Jul 2009 13:02:51 -0700
Subject: [SciPy-user] ANN: SciPy 2009 early registration extended to July
	22nd
Message-ID: <c7009a550907171302l229d7fa9yb191bb4262cad318@mail.gmail.com>

The early registration deadline for SciPy 2009 has been extended
until Wednesday, July 22, 2009.  Please register (
http://conference.scipy.org/to_register )
by this date to take advantage of the reduced early registration rate.
Since we just announced the conference schedule, I was asked to
provide extra time for people to register.  Fortunately, we were
able to get a few extra days from our vendors.  But we will have to
place orders next Thursday, so this is the last time we will be able
to extend the deadline for registration.

The conference schedule is available here:
  http://conference.scipy.org/schedule

About the conference
--------------------

SciPy 2009, the 8th Python in Science conference, will be held from
August 18-23, 2009 at Caltech in Pasadena, CA, USA.  The conference
starts with two days of tutorials to the scientific Python tools.
There will be two tracks, one for introduction of the basic tools to
beginners, and one for more advanced tools.  The tutorials will be
followed by two days of talks.  Both days of talks will begin with a
keynote address.  The first day?s keynote will be given by Peter
Norvig, the Director of Research at Google; while, the second keynote
will be delivered by Jon Guyer, a Materials Scientist in the
Thermodynamics and Kinetics Group at NIST.  The program committee will
select the remaining talks from submissions to our call for papers.
All selected talks will be included in our conference proceedings
edited by the program committee.  After the talks each day we will
provide several rooms for impromptu birds of a feather discussions.
Finally, the last two days of the conference will be used for a number
of coding sprints on the major software projects in our community.

For the 8th consecutive year, the conference will bring together the
developers and users of the open source software stack for scientific
computing with Python.  Attendees have the opportunity to review the
available tools and how they apply to specific problems.  By providing
a forum for developers to share their Python expertise with the wider
commercial, academic, and research communities, this conference
fosters collaboration and facilitates the sharing of software
components, techniques, and a vision for high level language use in
scientific computing.

For further information, please visit the conference homepage:
http://conference.scipy.org.

Important Dates
---------------

* Friday, July 3: Abstracts Due
* Wednesday, July 15: Announce accepted talks, post schedule
* Wednesday, July 22: Early Registration ends
* Tuesday-Wednesday, August 18-19: Tutorials
* Thursday-Friday, August 20-21: Conference
* Saturday-Sunday, August 22-23: Sprints
* Friday, September 4: Papers for proceedings due

Executive Committee
-------------------

* Jarrod Millman, UC Berkeley, USA (Conference Chair)
* Ga?l Varoquaux, INRIA Saclay, France (Program Co-Chair)
* St?fan van der Walt, University of Stellenbosch, South Africa
(Program Co-Chair)
* Fernando P?rez, UC Berkeley, USA (Tutorial Chair)


From eadrogue at gmx.net  Sat Jul 18 08:35:44 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Sat, 18 Jul 2009 14:35:44 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with gradient information
Message-ID: <20090718123544.GA3122@doriath.local>

Hi,

I'm using optimize.fmin_bfgs to find the minimum of a function.
It works, but I'd like to speed it up by supplying the gradient
of the function to fmin_bfgs.

Without supplying the gradient I get these results:

Warning: Desired error not necessarily achieveddue to precision loss
         Current function value: 638.939214
         Iterations: 73
         Function evaluations: 4387
         Gradient evaluations: 102

The output of my gradient function evaluated at xopt:

[ -1.26071352e-06   2.22057130e-06   9.10389060e-06  -3.47809758e-06
   5.26179023e-06  -5.90267183e-06  -3.19019368e-06  -7.39985613e-06
  -2.84634204e-06   3.84543574e-07   1.33341847e-06  -1.59029471e-06
   5.13325055e-06  -3.53840419e-06   2.23408274e-06   1.05588332e-05
   1.04574907e-05  -2.46512209e-06  -2.54991167e-07   1.24356893e-06
  -9.28475141e-06  -2.76441219e-07  -2.81902992e-06   7.59715257e-08
  -4.61241275e-07  -1.57030283e-06   4.43909204e-06   6.66069772e-08
  -1.64478684e-06   4.03578664e-06   6.81269187e-07   9.74726616e-06
  -5.92372950e-06   7.85634341e-06  -1.48669281e-06   2.67525449e-07
   3.50545615e-07   1.44128199e-06   2.71466860e-06   4.23270815e-06
  -2.20851113e-05]

The output of check_grad at xopt: 0.000106523006716

So, apparently it looks like my gradient function is correct,
doesn't it?  However, when I pass the gradient to fmin_bfgs I get
this:

Warning: Desired error not necessarily achieveddue to precision loss
         Current function value: 653.494345
         Iterations: 5
         Function evaluations: 45
         Gradient evaluations: 44

Notice that the minimum is higher now.

My gradient evaluated at xopt is far from zero this time:

[  5.99210031   7.78372931   2.17685535   8.62438169   6.78737246
   4.59089064   6.28766488   3.74376886   5.21582577   1.20448784
   0.26857912   5.17257475   5.32668068   8.14539521   3.21022361
   5.87014267   5.14406772   6.26400519   4.35807008   5.20230664
   1.32962472   6.05407954   2.85062903   5.29204265  10.4366293
   1.78770855  -2.22449411   5.20648252   4.05410094   6.64206808
   2.19202177   5.33385709   5.30404265   3.73158178   4.44347609
   4.38591199   3.12390498   7.01723668   3.93901794   6.31246349
   3.61374379]

And check_grad at xopt says: 34.3464575331

I can't figure out what's going on. From the output of
check_grad, it seems that my gradient function calculates the
gradient rightly at one point and wrongly at another point.
Is that correct?

Ernest


From eadrogue at gmx.net  Sat Jul 18 08:36:50 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Sat, 18 Jul 2009 14:36:50 +0200
Subject: [SciPy-user] counting pairs of values
In-Reply-To: <B721573D-0215-4FDB-B9A2-28B94C90B3A5@yale.edu>
References: <20090716140154.GA25712@doriath.local>
	<B721573D-0215-4FDB-B9A2-28B94C90B3A5@yale.edu>
Message-ID: <20090718123650.GB3122@doriath.local>

16/07/09 @ 10:48 (-0400), thus spake Zachary Pincus:
> Check out the discussion from yesterday on the numpy list "Neighbour- 
> frequency matrix" for a generalization of this problem. You could also  
> use numpy.histogram2d for this.

Thanks for your ideas, Zachary & Chris.

Ernest


From timmichelsen at gmx-topmail.de  Sat Jul 18 12:03:08 2009
From: timmichelsen at gmx-topmail.de (Tim Michelsen)
Date: Sat, 18 Jul 2009 18:03:08 +0200
Subject: [SciPy-user] Today's Scientific Computing Webinar
In-Reply-To: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>
References: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>
Message-ID: <h3srns$9js$2@ger.gmane.org>

Thanks a lot for providing this! A real big thing.

I have issues with the WMV videos on a Linux (Ubuntu) system:

I can see the video correctly:
http://download.enthought.com/media/2009-07Scientific_Computing_with_Python_Webinar.wmv

The earlier ones do not show a video. I can only hear the audio track:
http://download.enthought.com/media/2009-06-19Scientific_Computing_with_Python_Webinar.wmv
http://download.enthought.com/media/2009-05-22_Scientific_Computing_with_Python_Webinar.wmv

I use VLC.

Thanks and regards,
Timmie



From sebastian.walter at gmail.com  Sat Jul 18 12:36:54 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Sat, 18 Jul 2009 18:36:54 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with gradient
	information
In-Reply-To: <20090718123544.GA3122@doriath.local>
References: <20090718123544.GA3122@doriath.local>
Message-ID: <ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>

I don't find it so hard to believe that you got your gradient function wrong.
Could you post the code of your objective function?

Maybe you've just got the wrong sign. Near the optimum this would be OK,
but trying to do a descent step away from the optimizer is to fail.

Just a wild guess,
Sebastian



2009/7/18 Ernest Adrogu? <eadrogue at gmx.net>:
> Hi,
>
> I'm using optimize.fmin_bfgs to find the minimum of a function.
> It works, but I'd like to speed it up by supplying the gradient
> of the function to fmin_bfgs.
>
> Without supplying the gradient I get these results:
>
> Warning: Desired error not necessarily achieveddue to precision loss
> ? ? ? ? Current function value: 638.939214
> ? ? ? ? Iterations: 73
> ? ? ? ? Function evaluations: 4387
> ? ? ? ? Gradient evaluations: 102
>
> The output of my gradient function evaluated at xopt:
>
> [ -1.26071352e-06 ? 2.22057130e-06 ? 9.10389060e-06 ?-3.47809758e-06
> ? 5.26179023e-06 ?-5.90267183e-06 ?-3.19019368e-06 ?-7.39985613e-06
> ?-2.84634204e-06 ? 3.84543574e-07 ? 1.33341847e-06 ?-1.59029471e-06
> ? 5.13325055e-06 ?-3.53840419e-06 ? 2.23408274e-06 ? 1.05588332e-05
> ? 1.04574907e-05 ?-2.46512209e-06 ?-2.54991167e-07 ? 1.24356893e-06
> ?-9.28475141e-06 ?-2.76441219e-07 ?-2.81902992e-06 ? 7.59715257e-08
> ?-4.61241275e-07 ?-1.57030283e-06 ? 4.43909204e-06 ? 6.66069772e-08
> ?-1.64478684e-06 ? 4.03578664e-06 ? 6.81269187e-07 ? 9.74726616e-06
> ?-5.92372950e-06 ? 7.85634341e-06 ?-1.48669281e-06 ? 2.67525449e-07
> ? 3.50545615e-07 ? 1.44128199e-06 ? 2.71466860e-06 ? 4.23270815e-06
> ?-2.20851113e-05]
>
> The output of check_grad at xopt: 0.000106523006716
>
> So, apparently it looks like my gradient function is correct,
> doesn't it? ?However, when I pass the gradient to fmin_bfgs I get
> this:
>
> Warning: Desired error not necessarily achieveddue to precision loss
> ? ? ? ? Current function value: 653.494345
> ? ? ? ? Iterations: 5
> ? ? ? ? Function evaluations: 45
> ? ? ? ? Gradient evaluations: 44
>
> Notice that the minimum is higher now.
>
> My gradient evaluated at xopt is far from zero this time:
>
> [ ?5.99210031 ? 7.78372931 ? 2.17685535 ? 8.62438169 ? 6.78737246
> ? 4.59089064 ? 6.28766488 ? 3.74376886 ? 5.21582577 ? 1.20448784
> ? 0.26857912 ? 5.17257475 ? 5.32668068 ? 8.14539521 ? 3.21022361
> ? 5.87014267 ? 5.14406772 ? 6.26400519 ? 4.35807008 ? 5.20230664
> ? 1.32962472 ? 6.05407954 ? 2.85062903 ? 5.29204265 ?10.4366293
> ? 1.78770855 ?-2.22449411 ? 5.20648252 ? 4.05410094 ? 6.64206808
> ? 2.19202177 ? 5.33385709 ? 5.30404265 ? 3.73158178 ? 4.44347609
> ? 4.38591199 ? 3.12390498 ? 7.01723668 ? 3.93901794 ? 6.31246349
> ? 3.61374379]
>
> And check_grad at xopt says: 34.3464575331
>
> I can't figure out what's going on. From the output of
> check_grad, it seems that my gradient function calculates the
> gradient rightly at one point and wrongly at another point.
> Is that correct?
>
> Ernest
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From sccolbert at gmail.com  Sat Jul 18 12:43:02 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Sat, 18 Jul 2009 12:43:02 -0400
Subject: [SciPy-user] non-linear multi-variate optimization
In-Reply-To: <ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com>
References: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>
	<ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com>
Message-ID: <7f014ea60907180943h18b27318k4a5c5feb9bf8e4e4@mail.gmail.com>

I'm not quite understanding what you're saying. According to this:
http://en.wikipedia.org/wiki/Gradient
the Gradient and Jacobian are one in the same.

what would be the form of the gradient in my case?

Thanks for the help!

Chris

On Wed, Jul 15, 2009 at 3:32 AM, Sebastian
Walter<sebastian.walter at gmail.com> wrote:
> The gradient g is only defined for functions f: R^N --> R and is
> simply an array with shape (N,)
>
> what you sketched in your post is the Jacobian J of a function f: R^N --> R^M
> Typically, the Jacobian J is defined to have the shape (M,N), but
> there are exceptions.
>
> hope that helps a little
> Sebastian
>
>
>
> On Wed, Jul 15, 2009 at 2:00 AM, Chris Colbert<sccolbert at gmail.com> wrote:
>> The routines for non-linear optimization in scipy.optimize take an
>> argument for a function that computes the gradient.
>>
>> What should be the format of return value of this function? I am
>> assuming that its the gradient of the functions with repect to the
>> independent variables in row vector format.
>>
>> for example say we have:
>>
>> f(x,y,z; a1, a2, a3) where a1, a2, and a3 are the independent variables.
>>
>> Should the gradient of N x,y,z points then be of the form:
>>
>> df/da ?= ? [[df(X0)/da1, df(X0)/da2, df(X0)/da3],
>> ? ? ? ? ? ? ? [df(X1)/da1, df(X1)/da2, df(X1)/da3],
>> ? ? ? ? ? ? ? ....
>> ? ? ? ? ? ? ? [df(Xn)/da1, df(Xn)/da2, df(Xn)/da3]]
>>
>> where Xn is the set of (xn, yn, zn) ?
>>
>> cheers!
>>
>> Chris
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From sebastian.walter at gmail.com  Sat Jul 18 12:53:26 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Sat, 18 Jul 2009 18:53:26 +0200
Subject: [SciPy-user] non-linear multi-variate optimization
In-Reply-To: <7f014ea60907180943h18b27318k4a5c5feb9bf8e4e4@mail.gmail.com>
References: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>
	<ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com>
	<7f014ea60907180943h18b27318k4a5c5feb9bf8e4e4@mail.gmail.com>
Message-ID: <ec9f80fa0907180953w2acca265w1de9806b2f6cbecf@mail.gmail.com>

What I've been trying to say is:

The function you want to optimize must be of the form

f: R^N -> R
      x  -> y = f(x)

i.e. mapping to the real numbers.

then the derivative f'(x) is in R^N
i.e. numpy.shape(f'(x)) == (N,)


If you could be more precise on which optimization function you are
trying to use and what your objective function looks like it would be
easier to help.


Sebastian








On Sat, Jul 18, 2009 at 6:43 PM, Chris Colbert<sccolbert at gmail.com> wrote:
> I'm not quite understanding what you're saying. According to this:
> http://en.wikipedia.org/wiki/Gradient
> the Gradient and Jacobian are one in the same.
>
> what would be the form of the gradient in my case?
>
> Thanks for the help!
>
> Chris
>
> On Wed, Jul 15, 2009 at 3:32 AM, Sebastian
> Walter<sebastian.walter at gmail.com> wrote:
>> The gradient g is only defined for functions f: R^N --> R and is
>> simply an array with shape (N,)
>>
>> what you sketched in your post is the Jacobian J of a function f: R^N --> R^M
>> Typically, the Jacobian J is defined to have the shape (M,N), but
>> there are exceptions.
>>
>> hope that helps a little
>> Sebastian
>>
>>
>>
>> On Wed, Jul 15, 2009 at 2:00 AM, Chris Colbert<sccolbert at gmail.com> wrote:
>>> The routines for non-linear optimization in scipy.optimize take an
>>> argument for a function that computes the gradient.
>>>
>>> What should be the format of return value of this function? I am
>>> assuming that its the gradient of the functions with repect to the
>>> independent variables in row vector format.
>>>
>>> for example say we have:
>>>
>>> f(x,y,z; a1, a2, a3) where a1, a2, and a3 are the independent variables.
>>>
>>> Should the gradient of N x,y,z points then be of the form:
>>>
>>> df/da ?= ? [[df(X0)/da1, df(X0)/da2, df(X0)/da3],
>>> ? ? ? ? ? ? ? [df(X1)/da1, df(X1)/da2, df(X1)/da3],
>>> ? ? ? ? ? ? ? ....
>>> ? ? ? ? ? ? ? [df(Xn)/da1, df(Xn)/da2, df(Xn)/da3]]
>>>
>>> where Xn is the set of (xn, yn, zn) ?
>>>
>>> cheers!
>>>
>>> Chris
>>> _______________________________________________
>>> SciPy-user mailing list
>>> SciPy-user at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From sccolbert at gmail.com  Sat Jul 18 13:08:00 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Sat, 18 Jul 2009 13:08:00 -0400
Subject: [SciPy-user] non-linear multi-variate optimization
In-Reply-To: <ec9f80fa0907180953w2acca265w1de9806b2f6cbecf@mail.gmail.com>
References: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>
	<ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com>
	<7f014ea60907180943h18b27318k4a5c5feb9bf8e4e4@mail.gmail.com>
	<ec9f80fa0907180953w2acca265w1de9806b2f6cbecf@mail.gmail.com>
Message-ID: <7f014ea60907181008v244ac55epaa423a463a625dec@mail.gmail.com>

I'm using fmin_l_bfgs_b

here's the code of my objective function, its the equation of a
superquadric with 11 free variables (a1, a2, a3, e1, e2, px, py, pz,
phi, theta, tsai) the variables (xw, yw, zw) are length N vectors
representing the world coordinates to which I'm fitting the
superquadric.  (i know i spelled psi wrong, i need to change it :) )

def superQuadricFit((a1, a2, a3, e1, e2, phi, theta, tsai, px, py, pz),

                     *args):



    a1 = float(a1)

    a2 = float(a2)

    a3 = float(a3)

    e1 = float(e1)

    e2 = float(e2)

    phi = float(phi)

    theta = float(theta)

    tsai = float(tsai)

    px = float(px)

    py = float(py)

    pz = float(pz)







    xw = args[0]

    yw = args[1]

    zw = args[2]



    cphi = math.cos(phi)

    ctheta = math.cos(theta)

    ctsai = math.cos(tsai)

    sphi = math.sin(phi)

    stheta = math.sin(theta)

    stsai = math.sin(tsai)



    nx = cphi * ctheta * ctsai - sphi * stsai

    ny = sphi * ctheta * ctsai + cphi * stsai

    nz = -stheta * ctsai

    ox = -cphi * ctheta * stsai - sphi * ctsai

    oy = -sphi * ctheta * stsai + cphi * ctsai

    oz = stheta * stsai

    ax = cphi * stheta

    ay = sphi * stheta

    az = ctheta







    f1 = ((nx * xw + ny * yw + nz * zw - px * nx - py * ny - pz * nz) / a1)

    f2 = ((ox * xw + oy * yw + oz * zw - px * ox - py * oy - pz * oz) / a2)

    f3 = ((ax * xw + ay * yw + az * zw - px * ax - py * ay - pz * az) / a3)



    F = ((f1**2)**(1/e2) + (f2**2)**(1/e2))**(e2/e1) + (f3**2)**(1/e1)



    err = (math.sqrt(a1 * a2 * a3) * (F**(e1) - 1))**2


    sumerr = err.sum()


    print err



    return sumerr



So I would think the gradient should express the steepness of the
function wrt the 11 variables, and that steepness will be different at
every point (xw, yw, zw)_i . I cant see how to give any useful
gradient information in a single N-length vector.

Chris

On Sat, Jul 18, 2009 at 12:53 PM, Sebastian
Walter<sebastian.walter at gmail.com> wrote:
> What I've been trying to say is:
>
> The function you want to optimize must be of the form
>
> f: R^N -> R
> ? ? ?x ?-> y = f(x)
>
> i.e. mapping to the real numbers.
>
> then the derivative f'(x) is in R^N
> i.e. numpy.shape(f'(x)) == (N,)
>
>
> If you could be more precise on which optimization function you are
> trying to use and what your objective function looks like it would be
> easier to help.
>
>
> Sebastian
>
>
>
>
>
>
>
>
> On Sat, Jul 18, 2009 at 6:43 PM, Chris Colbert<sccolbert at gmail.com> wrote:
>> I'm not quite understanding what you're saying. According to this:
>> http://en.wikipedia.org/wiki/Gradient
>> the Gradient and Jacobian are one in the same.
>>
>> what would be the form of the gradient in my case?
>>
>> Thanks for the help!
>>
>> Chris
>>
>> On Wed, Jul 15, 2009 at 3:32 AM, Sebastian
>> Walter<sebastian.walter at gmail.com> wrote:
>>> The gradient g is only defined for functions f: R^N --> R and is
>>> simply an array with shape (N,)
>>>
>>> what you sketched in your post is the Jacobian J of a function f: R^N --> R^M
>>> Typically, the Jacobian J is defined to have the shape (M,N), but
>>> there are exceptions.
>>>
>>> hope that helps a little
>>> Sebastian
>>>
>>>
>>>
>>> On Wed, Jul 15, 2009 at 2:00 AM, Chris Colbert<sccolbert at gmail.com> wrote:
>>>> The routines for non-linear optimization in scipy.optimize take an
>>>> argument for a function that computes the gradient.
>>>>
>>>> What should be the format of return value of this function? I am
>>>> assuming that its the gradient of the functions with repect to the
>>>> independent variables in row vector format.
>>>>
>>>> for example say we have:
>>>>
>>>> f(x,y,z; a1, a2, a3) where a1, a2, and a3 are the independent variables.
>>>>
>>>> Should the gradient of N x,y,z points then be of the form:
>>>>
>>>> df/da ?= ? [[df(X0)/da1, df(X0)/da2, df(X0)/da3],
>>>> ? ? ? ? ? ? ? [df(X1)/da1, df(X1)/da2, df(X1)/da3],
>>>> ? ? ? ? ? ? ? ....
>>>> ? ? ? ? ? ? ? [df(Xn)/da1, df(Xn)/da2, df(Xn)/da3]]
>>>>
>>>> where Xn is the set of (xn, yn, zn) ?
>>>>
>>>> cheers!
>>>>
>>>> Chris
>>>> _______________________________________________
>>>> SciPy-user mailing list
>>>> SciPy-user at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>>
>>> _______________________________________________
>>> SciPy-user mailing list
>>> SciPy-user at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From gokhansever at gmail.com  Sat Jul 18 13:35:45 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Sat, 18 Jul 2009 12:35:45 -0500
Subject: [SciPy-user] Today's Scientific Computing Webinar
In-Reply-To: <h3srns$9js$2@ger.gmane.org>
References: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>
	<h3srns$9js$2@ger.gmane.org>
Message-ID: <49d6b3500907181035i34b97943i10a3e221ba9c8329@mail.gmail.com>

On Sat, Jul 18, 2009 at 11:03 AM, Tim Michelsen <timmichelsen at gmx-topmail.de
> wrote:

> Thanks a lot for providing this! A real big thing.
>
> I have issues with the WMV videos on a Linux (Ubuntu) system:
>
> I can see the video correctly:
>
> http://download.enthought.com/media/2009-07Scientific_Computing_with_Python_Webinar.wmv
>
> The earlier ones do not show a video. I can only hear the audio track:
>
> http://download.enthought.com/media/2009-06-19Scientific_Computing_with_Python_Webinar.wmv
>
> http://download.enthought.com/media/2009-05-22_Scientific_Computing_with_Python_Webinar.wmv
>
> I use VLC.
>
> Thanks and regards,
> Timmie
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


For some reasons some wmv's don't work on my Fedora 10 either. Neither with
VLC nor with mplayer. Seems to me an encoding issue. One technique might be
to convert wmv's to avi's using mencoder to fix this viewing problem.

However, the very last recorded wmv works pretty nicely with mplayer. This
said, there are cases where video was working on vlc, but not audio, and
reverse situation for mplayer :)

In the case where I can't find any solution, I just switch to windows and
use kml player. It can play almost every codec without any problem.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090718/8362b981/attachment.html>

From robert.kern at gmail.com  Sat Jul 18 14:02:40 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 18 Jul 2009 13:02:40 -0500
Subject: [SciPy-user] non-linear multi-variate optimization
In-Reply-To: <7f014ea60907181008v244ac55epaa423a463a625dec@mail.gmail.com>
References: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com> 
	<ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com> 
	<7f014ea60907180943h18b27318k4a5c5feb9bf8e4e4@mail.gmail.com> 
	<ec9f80fa0907180953w2acca265w1de9806b2f6cbecf@mail.gmail.com> 
	<7f014ea60907181008v244ac55epaa423a463a625dec@mail.gmail.com>
Message-ID: <3d375d730907181102g5779c122v700703dbf4a8cf8d@mail.gmail.com>

On Sat, Jul 18, 2009 at 12:08, Chris Colbert<sccolbert at gmail.com> wrote:
> I'm using fmin_l_bfgs_b
>
> here's the code of my objective function, its the equation of a
> superquadric with 11 free variables (a1, a2, a3, e1, e2, px, py, pz,
> phi, theta, tsai) the variables (xw, yw, zw) are length N vectors
> representing the world coordinates to which I'm fitting the
> superquadric. ?(i know i spelled psi wrong, i need to change it :) )
>
> def superQuadricFit((a1, a2, a3, e1, e2, phi, theta, tsai, px, py, pz),
>
> ? ? ? ? ? ? ? ? ? ? *args):
>
>
>
> ? ?a1 = float(a1)
>
> ? ?a2 = float(a2)
>
> ? ?a3 = float(a3)
>
> ? ?e1 = float(e1)
>
> ? ?e2 = float(e2)
>
> ? ?phi = float(phi)
>
> ? ?theta = float(theta)
>
> ? ?tsai = float(tsai)
>
> ? ?px = float(px)
>
> ? ?py = float(py)
>
> ? ?pz = float(pz)
>
>
>
>
>
>
>
> ? ?xw = args[0]
>
> ? ?yw = args[1]
>
> ? ?zw = args[2]
>
>
>
> ? ?cphi = math.cos(phi)
>
> ? ?ctheta = math.cos(theta)
>
> ? ?ctsai = math.cos(tsai)
>
> ? ?sphi = math.sin(phi)
>
> ? ?stheta = math.sin(theta)
>
> ? ?stsai = math.sin(tsai)
>
>
>
> ? ?nx = cphi * ctheta * ctsai - sphi * stsai
>
> ? ?ny = sphi * ctheta * ctsai + cphi * stsai
>
> ? ?nz = -stheta * ctsai
>
> ? ?ox = -cphi * ctheta * stsai - sphi * ctsai
>
> ? ?oy = -sphi * ctheta * stsai + cphi * ctsai
>
> ? ?oz = stheta * stsai
>
> ? ?ax = cphi * stheta
>
> ? ?ay = sphi * stheta
>
> ? ?az = ctheta
>
>
>
>
>
>
>
> ? ?f1 = ((nx * xw + ny * yw + nz * zw - px * nx - py * ny - pz * nz) / a1)
>
> ? ?f2 = ((ox * xw + oy * yw + oz * zw - px * ox - py * oy - pz * oz) / a2)
>
> ? ?f3 = ((ax * xw + ay * yw + az * zw - px * ax - py * ay - pz * az) / a3)
>
>
>
> ? ?F = ((f1**2)**(1/e2) + (f2**2)**(1/e2))**(e2/e1) + (f3**2)**(1/e1)
>
>
>
> ? ?err = (math.sqrt(a1 * a2 * a3) * (F**(e1) - 1))**2
>
>
> ? ?sumerr = err.sum()
>
>
> ? ?print err
>
>
>
> ? ?return sumerr
>
>
>
> So I would think the gradient should express the steepness of the
> function wrt the 11 variables, and that steepness will be different at
> every point (xw, yw, zw)_i . I cant see how to give any useful
> gradient information in a single N-length vector.

The gradient that fmin_l_bfgs_b needs is not of F, but of sumerr.
sumerr is a single scalar that is a function of the 11 free variables
and thus has an 11-vector as its gradient.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From sccolbert at gmail.com  Sat Jul 18 14:10:49 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Sat, 18 Jul 2009 14:10:49 -0400
Subject: [SciPy-user] non-linear multi-variate optimization
In-Reply-To: <3d375d730907181102g5779c122v700703dbf4a8cf8d@mail.gmail.com>
References: <7f014ea60907141700w2f73f8e1yca076104054122ea@mail.gmail.com>
	<ec9f80fa0907150032w36445900x272b25acad33244c@mail.gmail.com>
	<7f014ea60907180943h18b27318k4a5c5feb9bf8e4e4@mail.gmail.com>
	<ec9f80fa0907180953w2acca265w1de9806b2f6cbecf@mail.gmail.com>
	<7f014ea60907181008v244ac55epaa423a463a625dec@mail.gmail.com>
	<3d375d730907181102g5779c122v700703dbf4a8cf8d@mail.gmail.com>
Message-ID: <7f014ea60907181110s4e987012jed23fc69613c96c7@mail.gmail.com>

AH!, ok I see now, makes perfect sense. Thanks guys for the help!


On Sat, Jul 18, 2009 at 2:02 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Sat, Jul 18, 2009 at 12:08, Chris Colbert<sccolbert at gmail.com> wrote:
>> I'm using fmin_l_bfgs_b
>>
>> here's the code of my objective function, its the equation of a
>> superquadric with 11 free variables (a1, a2, a3, e1, e2, px, py, pz,
>> phi, theta, tsai) the variables (xw, yw, zw) are length N vectors
>> representing the world coordinates to which I'm fitting the
>> superquadric. ?(i know i spelled psi wrong, i need to change it :) )
>>
>> def superQuadricFit((a1, a2, a3, e1, e2, phi, theta, tsai, px, py, pz),
>>
>> ? ? ? ? ? ? ? ? ? ? *args):
>>
>>
>>
>> ? ?a1 = float(a1)
>>
>> ? ?a2 = float(a2)
>>
>> ? ?a3 = float(a3)
>>
>> ? ?e1 = float(e1)
>>
>> ? ?e2 = float(e2)
>>
>> ? ?phi = float(phi)
>>
>> ? ?theta = float(theta)
>>
>> ? ?tsai = float(tsai)
>>
>> ? ?px = float(px)
>>
>> ? ?py = float(py)
>>
>> ? ?pz = float(pz)
>>
>>
>>
>>
>>
>>
>>
>> ? ?xw = args[0]
>>
>> ? ?yw = args[1]
>>
>> ? ?zw = args[2]
>>
>>
>>
>> ? ?cphi = math.cos(phi)
>>
>> ? ?ctheta = math.cos(theta)
>>
>> ? ?ctsai = math.cos(tsai)
>>
>> ? ?sphi = math.sin(phi)
>>
>> ? ?stheta = math.sin(theta)
>>
>> ? ?stsai = math.sin(tsai)
>>
>>
>>
>> ? ?nx = cphi * ctheta * ctsai - sphi * stsai
>>
>> ? ?ny = sphi * ctheta * ctsai + cphi * stsai
>>
>> ? ?nz = -stheta * ctsai
>>
>> ? ?ox = -cphi * ctheta * stsai - sphi * ctsai
>>
>> ? ?oy = -sphi * ctheta * stsai + cphi * ctsai
>>
>> ? ?oz = stheta * stsai
>>
>> ? ?ax = cphi * stheta
>>
>> ? ?ay = sphi * stheta
>>
>> ? ?az = ctheta
>>
>>
>>
>>
>>
>>
>>
>> ? ?f1 = ((nx * xw + ny * yw + nz * zw - px * nx - py * ny - pz * nz) / a1)
>>
>> ? ?f2 = ((ox * xw + oy * yw + oz * zw - px * ox - py * oy - pz * oz) / a2)
>>
>> ? ?f3 = ((ax * xw + ay * yw + az * zw - px * ax - py * ay - pz * az) / a3)
>>
>>
>>
>> ? ?F = ((f1**2)**(1/e2) + (f2**2)**(1/e2))**(e2/e1) + (f3**2)**(1/e1)
>>
>>
>>
>> ? ?err = (math.sqrt(a1 * a2 * a3) * (F**(e1) - 1))**2
>>
>>
>> ? ?sumerr = err.sum()
>>
>>
>> ? ?print err
>>
>>
>>
>> ? ?return sumerr
>>
>>
>>
>> So I would think the gradient should express the steepness of the
>> function wrt the 11 variables, and that steepness will be different at
>> every point (xw, yw, zw)_i . I cant see how to give any useful
>> gradient information in a single N-length vector.
>
> The gradient that fmin_l_bfgs_b needs is not of F, but of sumerr.
> sumerr is a single scalar that is a function of the 11 free variables
> and thus has an 11-vector as its gradient.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From jkington at wisc.edu  Sat Jul 18 14:11:04 2009
From: jkington at wisc.edu (Joe Kington)
Date: Sat, 18 Jul 2009 13:11:04 -0500
Subject: [SciPy-user] Today's Scientific Computing Webinar
In-Reply-To: <49d6b3500907181035i34b97943i10a3e221ba9c8329@mail.gmail.com>
References: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>
	<h3srns$9js$2@ger.gmane.org>
	<49d6b3500907181035i34b97943i10a3e221ba9c8329@mail.gmail.com>
Message-ID: <f6951adf0907181111y15b29ff6v1ba1cb8215b5658d@mail.gmail.com>

For what it's worth, they work for me on openSuse with mplayer.

I've got ffmpeg installed, which is apparently what it's using...

> File: 2009-06-19Scientific_Computing_with_Python_Webinar.wmv
> <snip>
> Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
> [wmv3 @ 0x88c6470]Extra data: 8 bits left, value: 0
> Selected video codec: [ffwmv3] vfm: ffmpeg (FFmpeg WMV3/WMV9)
> <snip>
>

Hope that helps in some way...
-Joe

On Sat, Jul 18, 2009 at 12:35 PM, G?khan SEVER <gokhansever at gmail.com>wrote:

>
>
> On Sat, Jul 18, 2009 at 11:03 AM, Tim Michelsen <
> timmichelsen at gmx-topmail.de> wrote:
>
>> Thanks a lot for providing this! A real big thing.
>>
>> I have issues with the WMV videos on a Linux (Ubuntu) system:
>>
>> I can see the video correctly:
>>
>> http://download.enthought.com/media/2009-07Scientific_Computing_with_Python_Webinar.wmv
>>
>> The earlier ones do not show a video. I can only hear the audio track:
>>
>> http://download.enthought.com/media/2009-06-19Scientific_Computing_with_Python_Webinar.wmv
>>
>> http://download.enthought.com/media/2009-05-22_Scientific_Computing_with_Python_Webinar.wmv
>>
>> I use VLC.
>>
>> Thanks and regards,
>> Timmie
>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>
>
> For some reasons some wmv's don't work on my Fedora 10 either. Neither with
> VLC nor with mplayer. Seems to me an encoding issue. One technique might be
> to convert wmv's to avi's using mencoder to fix this viewing problem.
>
> However, the very last recorded wmv works pretty nicely with mplayer. This
> said, there are cases where video was working on vlc, but not audio, and
> reverse situation for mplayer :)
>
> In the case where I can't find any solution, I just switch to windows and
> use kml player. It can play almost every codec without any problem.
>
> --
> G?khan
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090718/20bca7c6/attachment.html>

From oliphant at enthought.com  Sat Jul 18 14:17:14 2009
From: oliphant at enthought.com (Travis Oliphant)
Date: Sat, 18 Jul 2009 13:17:14 -0500
Subject: [SciPy-user] Today's Scientific Computing Webinar
In-Reply-To: <h3srns$9js$2@ger.gmane.org>
References: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>
	<h3srns$9js$2@ger.gmane.org>
Message-ID: <81861419-8B2F-4029-9697-3CB591C8940E@enthought.com>


On Jul 18, 2009, at 11:03 AM, Tim Michelsen wrote:

> Thanks a lot for providing this! A real big thing.
>
> I have issues with the WMV videos on a Linux (Ubuntu) system:
>
> I can see the video correctly:
> http://download.enthought.com/media/2009-07Scientific_Computing_with_Python_Webinar.wmv
>
> The earlier ones do not show a video. I can only hear the audio track:
> http://download.enthought.com/media/2009-06-19Scientific_Computing_with_Python_Webinar.wmv
> http://download.enthought.com/media/2009-05-22_Scientific_Computing_with_Python_Webinar.wmv
>
> I use VLC.

Thank you for the feedback.  We definitely have some work to do to  
improve the archived video delivery.

Best regards,

-Travis

--
Travis Oliphant
Enthought Inc.
1-512-536-1057
http://www.enthought.com
oliphant at enthought.com







From Omer.Khalid at cern.ch  Sat Jul 18 16:15:53 2009
From: Omer.Khalid at cern.ch (Omer Khalid)
Date: Sat, 18 Jul 2009 22:15:53 +0200
Subject: [SciPy-user] Unable to build SciPy on OS X 10.5.7
Message-ID: <77e5896b0907181315k4823b83ch8bf020979d23cbff@mail.gmail.com>

Hi,

I am trying to install SciPy following these instructions:
http://www.scipy.org/Download

I already have installed Gfortran and installed numpy. But when I try to
build scipy, I constantly getting error to build them for OS X Lepeord
10.5.7:

dyld: lazy symbol binding failed: Symbol not found: _iconv_open
  Referenced from: /usr/lib/libaprutil-1.0.dylib
  Expected in: /opt/local/lib/libiconv.2.dylib

dyld: Symbol not found: _iconv_open
  Referenced from: /usr/lib/libaprutil-1.0.dylib
  Expected in: /opt/local/lib/libiconv.2.dylib

Traceback (most recent call last):
  File "setup.py", line 82, in <module>
    FULLVERSION += svn_version()
  File "setup.py", line 74, in svn_version
    raise ValueError("Error while parsing svn version ?")
ValueError: Error while parsing svn version ?


Thanks for your help!
Omer

**************************************
CERN, Engineering Department,
CH-1211 Geneva 23, Switzerland

Email:  Omer.Khalid at cern.ch
**************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090718/75d2c78f/attachment.html>

From pgmdevlist at gmail.com  Sat Jul 18 21:31:45 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sat, 18 Jul 2009 21:31:45 -0400
Subject: [SciPy-user] scikits.timeseries : Problem using tsfromtxt
	withconverter
In-Reply-To: <4A605E35.63BA.009B.0@twdb.state.tx.us>
References: <4A5EDFE70200009B0002007A@GWWEB.twdb.state.tx.us>
	<A77581A3-9F8D-416A-9A50-E891C23CE62E@gmail.com>
	<4A605E35.63BA.009B.0@twdb.state.tx.us>
Message-ID: <3332DCA3-FA01-4FD2-82B0-67B02EAB3840@gmail.com>


On Jul 17, 2009, at 12:19 PM, Dharhas Pothina wrote:

>
>> Nothing. It's a bug, sorry about that. I'll try to correct that later
>> today or tmw at worst.
>>
>> Thanks for reporting,
>
> glad to be of help.

Should be fixed in r2199. Would you mind giving it a try ? And sorry  
for the delay


From dwf at cs.toronto.edu  Sat Jul 18 21:56:13 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sat, 18 Jul 2009 21:56:13 -0400
Subject: [SciPy-user] Unable to build SciPy on OS X 10.5.7
In-Reply-To: <77e5896b0907181315k4823b83ch8bf020979d23cbff@mail.gmail.com>
References: <77e5896b0907181315k4823b83ch8bf020979d23cbff@mail.gmail.com>
Message-ID: <6AA9E7C1-F1B6-4A67-B550-FDB15A1799E7@cs.toronto.edu>

The problem is you have a MacPorts install of libiconv which is  
different from the system iconv in /usr/lib, and evidently linker is  
preferring the MacPorts one over the system one.

You could try 'unset DYLD_LIBRARY_PATH' before you build to make it  
ignore MacPorts' version.

David

On 18-Jul-09, at 4:15 PM, Omer Khalid wrote:

> Hi,
>
> I am trying to install SciPy following these instructions: http://www.scipy.org/Download
>
> I already have installed Gfortran and installed numpy. But when I  
> try to build scipy, I constantly getting error to build them for OS  
> X Lepeord 10.5.7:
>
> dyld: lazy symbol binding failed: Symbol not found: _iconv_open
>   Referenced from: /usr/lib/libaprutil-1.0.dylib
>   Expected in: /opt/local/lib/libiconv.2.dylib
>
> dyld: Symbol not found: _iconv_open
>   Referenced from: /usr/lib/libaprutil-1.0.dylib
>   Expected in: /opt/local/lib/libiconv.2.dylib
>
> Traceback (most recent call last):
>   File "setup.py", line 82, in <module>
>     FULLVERSION += svn_version()
>   File "setup.py", line 74, in svn_version
>     raise ValueError("Error while parsing svn version ?")
> ValueError: Error while parsing svn version ?
>
>
> Thanks for your help!
> Omer
>
> **************************************	                        	
> CERN, Engineering Department,	
> CH-1211 Geneva 23, Switzerland
>
> Email:  Omer.Khalid at cern.ch
> **************************************
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From eadrogue at gmx.net  Sun Jul 19 07:05:43 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Sun, 19 Jul 2009 13:05:43 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with
	gradient	information
In-Reply-To: <ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
Message-ID: <20090719110543.GA7138@doriath.local>

Hi,
18/07/09 @ 18:36 (+0200), thus spake Sebastian Walter:
> I don't find it so hard to believe that you got your gradient function wrong.
> Could you post the code of your objective function?

Here it goes:

def fobj(self, x):

    # x = (alpha1...alphan, beta0..betan, gamma, rho)

    n = self.n

	# use absolute values for alphas and betas

    y = [abs(i) for i in x[:-2]]

    # alpha0 = n - sum(alpha1...alphan)

    y.insert(0, abs(n-sum(y[:n-1])))

    alpha = dict(zip(self.names, y[:n]))
    beta = dict(zip(self.names, y[n:]))

    gamma = abs(x[-2])
    rho = x[-1]

    pseudo_likelihood = 0

    for obs in self.observations:

        mu1 = alpha[obs.ht] * beta[obs.at] * gamma
        mu2 = alpha[obs.at] * beta[obs.ht]
        tau = self.tau(mu1, mu2, rho, obs.hg, obs.ag)

        # avoid log(0)
        mu1 = mu1 > 0 and mu1 or 1e-10
        mu2 = mu2 > 0 and mu2 or 1e-10
        tau = tau > 0 and tau or 1e-10

        pseudo_likelihood += math.log(tau)
        pseudo_likelihood += obs.hg * math.log(mu1) - mu1
        pseudo_likelihood += obs.ag * math.log(mu2) - mu2

    return -pseudo_likelihood

> Maybe you've just got the wrong sign. Near the optimum this would be OK,
> but trying to do a descent step away from the optimizer is to fail.

Yes, it must be the gradient that is wrong. It occurs to me
that it could be related to the fact that I'm changing the value
of tau() in the objective function when tau is < 0, and I don't think
tau_prime() used in the gradient reflects this.

Thanks.


Ernest


From eadrogue at gmx.net  Sun Jul 19 07:20:09 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Sun, 19 Jul 2009 13:20:09 +0200
Subject: [SciPy-user] Today's Scientific Computing Webinar
In-Reply-To: <h3srns$9js$2@ger.gmane.org>
References: <529C3B8C-3BDE-426A-A4F3-8FF66B76E9D6@enthought.com>
	<h3srns$9js$2@ger.gmane.org>
Message-ID: <20090719112009.GB7138@doriath.local>

18/07/09 @ 18:03 (+0200), thus spake Tim Michelsen:
> I have issues with the WMV videos on a Linux (Ubuntu) system:
> 
> I can see the video correctly:
> http://download.enthought.com/media/2009-07Scientific_Computing_with_Python_Webinar.wmv
> 
> The earlier ones do not show a video. I can only hear the audio track:
> http://download.enthought.com/media/2009-06-19Scientific_Computing_with_Python_Webinar.wmv
> http://download.enthought.com/media/2009-05-22_Scientific_Computing_with_Python_Webinar.wmv

I can play the first two, but the last one shows no video.

mplayer says:

Requested video codec family [gotomeeting] (vfm=dmo) not available.
Enable it at compilation.

and vlc:
[00000456] main decoder error: no suitable decoder module for fourcc `G2M3'.
VLC probably does not support this sound or video format.

This is on a Debian system.

Ernest


From yosefmel at post.tau.ac.il  Sun Jul 19 08:08:55 2009
From: yosefmel at post.tau.ac.il (Yosef Meller)
Date: Sun, 19 Jul 2009 15:08:55 +0300
Subject: [SciPy-user] Assignment of oversize values.
Message-ID: <200907191508.55617.yosefmel@post.tau.ac.il>

Hello all,

I found today that I can assign to an array some larger array, resulting in 
only the first values being assigned:

In [1]: from numpy import * 

In [2]: t = r_[1, 2, 3]

In [3]: tt = array([[1, 1, 1], [2, 2, 2]])

In [4]: tt
Out[4]:   
array([[1, 1, 1],
       [2, 2, 2]])

In [5]: t
Out[5]: array([1, 2, 3])

In [9]: t[t > 0] = tt

In [10]: t
Out[10]: array([1, 1, 1])

I expected this to raise an exception about shape error or something, but it 
did not. 

The NumPy documentation [1] doesn't mention this case, so I just wanted to ask 
if that's the intended behaveiour.

Thanks,
Yosef.



From sebastian.walter at gmail.com  Sun Jul 19 11:41:44 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Sun, 19 Jul 2009 17:41:44 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with gradient
	information
In-Reply-To: <20090719110543.GA7138@doriath.local>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
	<20090719110543.GA7138@doriath.local>
Message-ID: <ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>

Hello,
I would like to add your objective function as part of the unit test
for on of the automatic differentiation tools I've been developing:
PYADOLC (download at http://github.com/b45ch1/pyadolc).

Unfortunately, the description of the objective function is
incomplete: there are some references to self.n and so on in the code.
Would it be possible for you to post the full description? That would
be great :)


Sebastian



2009/7/19 Ernest Adrogu? <eadrogue at gmx.net>:
> Hi,
> 18/07/09 @ 18:36 (+0200), thus spake Sebastian Walter:
>> I don't find it so hard to believe that you got your gradient function wrong.
>> Could you post the code of your objective function?
>
> Here it goes:
>
> def fobj(self, x):
>
> ? ?# x = (alpha1...alphan, beta0..betan, gamma, rho)
>
> ? ?n = self.n
>
> ? ? ? ?# use absolute values for alphas and betas
>
> ? ?y = [abs(i) for i in x[:-2]]
>
> ? ?# alpha0 = n - sum(alpha1...alphan)
>
> ? ?y.insert(0, abs(n-sum(y[:n-1])))
>
> ? ?alpha = dict(zip(self.names, y[:n]))
> ? ?beta = dict(zip(self.names, y[n:]))
>
> ? ?gamma = abs(x[-2])
> ? ?rho = x[-1]
>
> ? ?pseudo_likelihood = 0
>
> ? ?for obs in self.observations:
>
> ? ? ? ?mu1 = alpha[obs.ht] * beta[obs.at] * gamma
> ? ? ? ?mu2 = alpha[obs.at] * beta[obs.ht]
> ? ? ? ?tau = self.tau(mu1, mu2, rho, obs.hg, obs.ag)
>
> ? ? ? ?# avoid log(0)
> ? ? ? ?mu1 = mu1 > 0 and mu1 or 1e-10
> ? ? ? ?mu2 = mu2 > 0 and mu2 or 1e-10
> ? ? ? ?tau = tau > 0 and tau or 1e-10
>
> ? ? ? ?pseudo_likelihood += math.log(tau)
> ? ? ? ?pseudo_likelihood += obs.hg * math.log(mu1) - mu1
> ? ? ? ?pseudo_likelihood += obs.ag * math.log(mu2) - mu2
>
> ? ?return -pseudo_likelihood
>
>> Maybe you've just got the wrong sign. Near the optimum this would be OK,
>> but trying to do a descent step away from the optimizer is to fail.
>
> Yes, it must be the gradient that is wrong. It occurs to me
> that it could be related to the fact that I'm changing the value
> of tau() in the objective function when tau is < 0, and I don't think
> tau_prime() used in the gradient reflects this.
>
> Thanks.
>
>
> Ernest
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From eadrogue at gmx.net  Sun Jul 19 14:39:23 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Sun, 19 Jul 2009 20:39:23 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with
	gradient	information
In-Reply-To: <ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
	<20090719110543.GA7138@doriath.local>
	<ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>
Message-ID: <20090719183923.GA21105@doriath.local>

19/07/09 @ 17:41 (+0200), thus spake Sebastian Walter:
> Hello,
> I would like to add your objective function as part of the unit test
> for on of the automatic differentiation tools I've been developing:
> PYADOLC (download at http://github.com/b45ch1/pyadolc).
> 
> Unfortunately, the description of the objective function is
> incomplete: there are some references to self.n and so on in the code.
> Would it be possible for you to post the full description? That would
> be great :)

Yes, of course.
You can use this version that works independently. The original
one feeds on data from a dataset, where as this one uses randomly-generated
data. Example usage:

In [6]: m = Model(random_obs())

In [7]: x0 = m.guess()

In [8]: x0
Out[8]: 
[1.3414634146341464,
 1.4634146341463414,
 1.6341463414634145,
 1.5609756097560976,
 1.4878048780487805,
 1.8048780487804879,
 ........
 1.3902439024390243,
 1.6097560975609757,
 1.5,
 0]

In [9]: m.fobj(x0)
Out[9]: 1281.027571347247



------------------------------
import random
import math

class Observation(object):
    def __init__(self, data):
        self.ht = data[0]
        self.at = data[1]
        self.hg = data[2]
        self.ag = data[3]

def random_obs():
    names = 'abcdefghijklmnopqrstuvwxyz'
    random_data = [(i,j,random.randint(0,5),random.randint(0,5))
                   for i in names for j in names if i != j]
    return [Observation(i) for i in random_data]

class Model(object):
    def __init__(self, observations):
        self.observations = tuple(observations)
        self.names = [i.ht for i in observations]
        self.names.extend([i.at for i in observations])
        self.names = tuple(set(self.names))
        self.n = len(self.names)
    def guess(self):
        a, b = [], []
        total_hg = sum([i.hg for i in self.observations])
        for j in self.names:
            sh_j = sum([i.hg for i in self.observations if i.ht == j])
            ca_j = sum([i.hg for i in self.observations if i.at == j])
            a.append(sh_j/math.sqrt(total_hg))
            b.append(ca_j/math.sqrt(total_hg))
        return a + b + [1.5, 0]
    def tau(self, mu1, mu2, rho, x, y):
        if x == 0 and y == 0:
            t = 1 - mu1 * mu2 * rho
        elif x == 0 and y == 1:
            t = 1 + mu1 * rho
        elif x == 1 and y == 0:
            t = 1 + mu2 * rho
        elif x == 1 and y == 1:
            t = 1 - rho
        else:
            t = 1
        return t
    def fobj(self, x):
        n = self.n
        y = [abs(i) for i in x[:-2]]
        y.insert(0, abs(n - sum(y[:n-1])))
        a = dict(zip(self.names, y[:n]))
        b = dict(zip(self.names, y[n:]))
        g = abs(x[-2])
        r = x[-1]
        pseudo_loglikelihood = 0
        for m in self.observations:
            x = m.hg
            y = m.ag
            mu1 = a[m.ht] * b[m.at] * g
            mu2 = a[m.at] * b[m.ht]
            tau = self.tau(mu1, mu2, r, m.hg, m.ag)
            mu1 = mu1 > 0 and mu1 or 1e-10
            mu2 = mu2 > 0 and mu2 or 1e-10
            tau = tau > 0 and tau or 1e-10
            pseudo_loglikelihood += math.log(tau)
            pseudo_loglikelihood += m.hg * math.log(mu1) - mu1
            pseudo_loglikelihood += m.ag * math.log(mu2) - mu2
        return -pseudo_loglikelihood




From dwf at cs.toronto.edu  Mon Jul 20 00:30:18 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Mon, 20 Jul 2009 00:30:18 -0400
Subject: [SciPy-user] Assignment of oversize values.
In-Reply-To: <200907191508.55617.yosefmel@post.tau.ac.il>
References: <200907191508.55617.yosefmel@post.tau.ac.il>
Message-ID: <38E67D7F-B411-43CD-9296-42D18E17614C@cs.toronto.edu>

On 19-Jul-09, at 8:08 AM, Yosef Meller wrote:

> I expected this to raise an exception about shape error or  
> something, but it
> did not.
>
> The NumPy documentation [1] doesn't mention this case, so I just  
> wanted to ask
> if that's the intended behaveiour.

I'm pretty sure it is intended behaviour; this is perfectly consistent  
with the way sequence modification works even with normal Python lists.

In [13]: X = range(10)

In [14]: Y = range(10, 20)

In [15]: X[:5] = Y

In [16]: X
Out[16]: [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 5, 6, 7, 8, 9]

Similarly if you try to zip(a, b) on two things together of different  
lengths, you get back a sequence of tuples that is length min(len(a),  
len(b)) - the extras are discarded.

Regards,

David




From yosefmel at post.tau.ac.il  Mon Jul 20 02:28:21 2009
From: yosefmel at post.tau.ac.il (Yosef Meller)
Date: Mon, 20 Jul 2009 09:28:21 +0300
Subject: [SciPy-user] Assignment of oversize values.
In-Reply-To: <38E67D7F-B411-43CD-9296-42D18E17614C@cs.toronto.edu>
References: <200907191508.55617.yosefmel@post.tau.ac.il>
	<38E67D7F-B411-43CD-9296-42D18E17614C@cs.toronto.edu>
Message-ID: <200907200928.21401.yosefmel@post.tau.ac.il>

On Monday 20 July 2009 07:30:18 David Warde-Farley wrote:
> On 19-Jul-09, at 8:08 AM, Yosef Meller wrote:
> > I expected this to raise an exception about shape error or
> > something, but it
> > did not.
> >
> > The NumPy documentation [1] doesn't mention this case, so I just
> > wanted to ask
> > if that's the intended behaveiour.
>
> I'm pretty sure it is intended behaviour; this is perfectly consistent
> with the way sequence modification works even with normal Python lists.

Good to know. Thanks.



From sebastian.walter at gmail.com  Mon Jul 20 04:43:32 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Mon, 20 Jul 2009 10:43:32 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with gradient
	information
In-Reply-To: <20090719183923.GA21105@doriath.local>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
	<20090719110543.GA7138@doriath.local>
	<ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>
	<20090719183923.GA21105@doriath.local>
Message-ID: <ec9f80fa0907200143j6a6748dx62d605991341d74d@mail.gmail.com>

Hi,

thanks for the function :).
 It is now part of pyadolc's unit test.
http://github.com/b45ch1/pyadolc/blob/3cfff80c062d43e812379f4606eda0ebaaf4e82c/tests/complicated_tests.py
 , Line 246

I added two functions: one is providing the gradient by finite
differences and the other by using automatic differentiation.
The finite differences gradient has very poor accuracy, only the first
three digits are correct.


Sebastian



2009/7/19 Ernest Adrogu? <eadrogue at gmx.net>:
> 19/07/09 @ 17:41 (+0200), thus spake Sebastian Walter:
>> Hello,
>> I would like to add your objective function as part of the unit test
>> for on of the automatic differentiation tools I've been developing:
>> PYADOLC (download at http://github.com/b45ch1/pyadolc).
>>
>> Unfortunately, the description of the objective function is
>> incomplete: there are some references to self.n and so on in the code.
>> Would it be possible for you to post the full description? That would
>> be great :)
>
> Yes, of course.
> You can use this version that works independently. The original
> one feeds on data from a dataset, where as this one uses randomly-generated
> data. Example usage:
>
> In [6]: m = Model(random_obs())
>
> In [7]: x0 = m.guess()
>
> In [8]: x0
> Out[8]:
> [1.3414634146341464,
>  1.4634146341463414,
>  1.6341463414634145,
>  1.5609756097560976,
>  1.4878048780487805,
>  1.8048780487804879,
>  ........
>  1.3902439024390243,
>  1.6097560975609757,
>  1.5,
>  0]
>
> In [9]: m.fobj(x0)
> Out[9]: 1281.027571347247
>
>
>
> ------------------------------
> import random
> import math
>
> class Observation(object):
>    def __init__(self, data):
>        self.ht = data[0]
>        self.at = data[1]
>        self.hg = data[2]
>        self.ag = data[3]
>
> def random_obs():
>    names = 'abcdefghijklmnopqrstuvwxyz'
>    random_data = [(i,j,random.randint(0,5),random.randint(0,5))
>                   for i in names for j in names if i != j]
>    return [Observation(i) for i in random_data]
>
> class Model(object):
>    def __init__(self, observations):
>        self.observations = tuple(observations)
>        self.names = [i.ht for i in observations]
>        self.names.extend([i.at for i in observations])
>        self.names = tuple(set(self.names))
>        self.n = len(self.names)
>    def guess(self):
>        a, b = [], []
>        total_hg = sum([i.hg for i in self.observations])
>        for j in self.names:
>            sh_j = sum([i.hg for i in self.observations if i.ht == j])
>            ca_j = sum([i.hg for i in self.observations if i.at == j])
>            a.append(sh_j/math.sqrt(total_hg))
>            b.append(ca_j/math.sqrt(total_hg))
>        return a + b + [1.5, 0]
>    def tau(self, mu1, mu2, rho, x, y):
>        if x == 0 and y == 0:
>            t = 1 - mu1 * mu2 * rho
>        elif x == 0 and y == 1:
>            t = 1 + mu1 * rho
>        elif x == 1 and y == 0:
>            t = 1 + mu2 * rho
>        elif x == 1 and y == 1:
>            t = 1 - rho
>        else:
>            t = 1
>        return t
>    def fobj(self, x):
>        n = self.n
>        y = [abs(i) for i in x[:-2]]
>        y.insert(0, abs(n - sum(y[:n-1])))
>        a = dict(zip(self.names, y[:n]))
>        b = dict(zip(self.names, y[n:]))
>        g = abs(x[-2])
>        r = x[-1]
>        pseudo_loglikelihood = 0
>        for m in self.observations:
>            x = m.hg
>            y = m.ag
>            mu1 = a[m.ht] * b[m.at] * g
>            mu2 = a[m.at] * b[m.ht]
>            tau = self.tau(mu1, mu2, r, m.hg, m.ag)
>            mu1 = mu1 > 0 and mu1 or 1e-10
>            mu2 = mu2 > 0 and mu2 or 1e-10
>            tau = tau > 0 and tau or 1e-10
>            pseudo_loglikelihood += math.log(tau)
>            pseudo_loglikelihood += m.hg * math.log(mu1) - mu1
>            pseudo_loglikelihood += m.ag * math.log(mu2) - mu2
>        return -pseudo_loglikelihood
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From Dharhas.Pothina at twdb.state.tx.us  Mon Jul 20 09:20:27 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Mon, 20 Jul 2009 08:20:27 -0500
Subject: [SciPy-user] scikits.timeseries: unable
	to	stack	timeserieswithdifferent	number of columns
In-Reply-To: <57965819-8A49-4048-82F5-406A3647CB54@gmail.com>
References: <4A5EDC75.63BA.009B.0@twdb.state.tx.us>
	<3FE87F21-23D5-43E7-A04E-0983DA249567@gmail.com>
	<4A605E00.63BA.009B.0@twdb.state.tx.us>
	<57965819-8A49-4048-82F5-406A3647CB54@gmail.com>
Message-ID: <4A6428CB.63BA.009B.0@twdb.state.tx.us>


thanks. I'll try that.

- dharhas

>>> Pierre GM <pgmdevlist at gmail.com> 7/17/2009 11:25 AM >>>

On Jul 17, 2009, at 12:18 PM, Dharhas Pothina wrote:

>
>
>>>> Pierre GM <pgmdevlist at gmail.com> 7/16/2009 4:21 PM >>>
>
>> Meanwhile, you could:
>> * make sure that your series are compatible: same frequency, same
>> starting and endings dates, same intervals between the dates
>> * stack the series using ma.column_stack : that gives you a masked  
>> array
>> * view the output as a TimeSeries, and set its dates to the dates of
>> the input.
>
> not sure if I completely understood this.
>
> so if I have a_ts and b_ts that are timeseries objects I do
>
> c = ma.stack_column((a_ts,b_ts))
>
> then how do I "view the output as a TimeSeries, and set its dates to  
> the dates of the input"


Provided that a_ts and b_ts are compatible (see previous post):
 >>> c_ts = ma.column_stack((a_ts, b_ts))
 >>> c_ts = c_ts..view(ts.TimeSeries)
 >>> c_ts.dates = a_ts.dates


_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.org 
http://mail.scipy.org/mailman/listinfo/scipy-user



From Omer.Khalid at cern.ch  Mon Jul 20 11:52:16 2009
From: Omer.Khalid at cern.ch (Omer Khalid)
Date: Mon, 20 Jul 2009 17:52:16 +0200
Subject: [SciPy-user] CDF/PDF Stats with SciPy
Message-ID: <77e5896b0907200852o71083dbh92d1cbb055416e6a@mail.gmail.com>

Hi Everybody,

I am new to Python and new to SciPy libraries. I wanted to take some ques
from the experts here on the list before dive into SciPy world.

I was wondering if some one could provide a rough guide about how to run two
stats functions: Cumulative Distribution Function (CDF) and Probability
Distribution Function (PDF).

My use case is the following: I have a sampleSpaceList [] which have 1000
floating point values. When a new floating point value is generated in my
program, I would like to run both CDF and PDF on the sampleList for it and
get the probabilty of value less or equal for CDF and probablity
distribution for PDF.

Many thanks in advance!

Omer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/38531494/attachment.html>

From ivo.maljevic at gmail.com  Mon Jul 20 12:07:10 2009
From: ivo.maljevic at gmail.com (Ivo Maljevic)
Date: Mon, 20 Jul 2009 12:07:10 -0400
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <77e5896b0907200852o71083dbh92d1cbb055416e6a@mail.gmail.com>
References: <77e5896b0907200852o71083dbh92d1cbb055416e6a@mail.gmail.com>
Message-ID: <826c64da0907200907m2775caa5jfe53ed515a456e5@mail.gmail.com>

Hi Omer,

For histogram you can either use histogram function from numpy/scipy:

from scipy import *

x = .... # some vector
h,bins = histogram(x,50,normed=True,new=True) # find the histogram, number
of bins = 50

or you can use pylab's version (good for plotting):

import matplotlib.pyplot as plt

count, bins, ignored = plt.hist(x, 50, normed=True)

For CDF you can use cumsum function (standard approach), but for smaller
number of data points I
prefer to use all the points, which is a neat trick:


from scipy import *

def my_cdf(x):

  bins = sort(x)
  cdf  = linspace(0,1,len(bins))
  return [bins, cdf]


2009/7/20 Omer Khalid <Omer.Khalid at cern.ch>

> Hi Everybody,
>
> I am new to Python and new to SciPy libraries. I wanted to take some ques
> from the experts here on the list before dive into SciPy world.
>
> I was wondering if some one could provide a rough guide about how to run
> two stats functions: Cumulative Distribution Function (CDF) and Probability
> Distribution Function (PDF).
>
> My use case is the following: I have a sampleSpaceList [] which have 1000
> floating point values. When a new floating point value is generated in my
> program, I would like to run both CDF and PDF on the sampleList for it and
> get the probabilty of value less or equal for CDF and probablity
> distribution for PDF.
>
> Many thanks in advance!
>
> Omer
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/bb4aebe5/attachment.html>

From Omer.Khalid at cern.ch  Mon Jul 20 16:11:29 2009
From: Omer.Khalid at cern.ch (Omer Khalid)
Date: Mon, 20 Jul 2009 22:11:29 +0200
Subject: [SciPy-user] CDF/PDF Stats with SciPy
Message-ID: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>

Hi Ivo,

Thanks for your reply. But I am getting a little confused here now. It seems
there are multiple ways to get the CDF for a distribution. You mean linspace
function returns a CDF for a normal distribution.

As far as I understood from other sources is that scipy.stats.norm.cdf
(mean, std) will return the CDF for the normal distribution or for
non-normal distribution given one replace *norm* with the distributions
name.

And what about scipy.stats.<dist>.fit function?
Thanks,
Omer

>
>
> Message: 7
> Date: Mon, 20 Jul 2009 12:07:10 -0400
> From: Ivo Maljevic <ivo.maljevic at gmail.com>
> Subject: Re: [SciPy-user] CDF/PDF Stats with SciPy
> To: SciPy Users List <scipy-user at scipy.org>
> Message-ID:
>        <826c64da0907200907m2775caa5jfe53ed515a456e5 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Omer,
>
> For histogram you can either use histogram function from numpy/scipy:
>
> from scipy import *
>
> x = .... # some vector
> h,bins = histogram(x,50,normed=True,new=True) # find the histogram, number
> of bins = 50
>
> or you can use pylab's version (good for plotting):
>
> import matplotlib.pyplot as plt
>
> count, bins, ignored = plt.hist(x, 50, normed=True)
>
> For CDF you can use cumsum function (standard approach), but for smaller
> number of data points I
> prefer to use all the points, which is a neat trick:
>
>
> from scipy import *
>
> def my_cdf(x):
>
>  bins = sort(x)
>  cdf  = linspace(0,1,len(bins))
>  return [bins, cdf]
>
>
> 2009/7/20 Omer Khalid <Omer.Khalid at cern.ch>
>
> > Hi Everybody,
> >
> > I am new to Python and new to SciPy libraries. I wanted to take some ques
> > from the experts here on the list before dive into SciPy world.
> >
> > I was wondering if some one could provide a rough guide about how to run
> > two stats functions: Cumulative Distribution Function (CDF) and
> Probability
> > Distribution Function (PDF).
> >
> > My use case is the following: I have a sampleSpaceList [] which have 1000
> > floating point values. When a new floating point value is generated in my
> > program, I would like to run both CDF and PDF on the sampleList for it
> and
> > get the probabilty of value less or equal for CDF and probablity
> > distribution for PDF.
> >
> > Many thanks in advance!
> >
> > Omer
> >
> >
> > _______________________________________________
> > SciPy-user mailing list
> > SciPy-user at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/bb4aebe5/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
> End of SciPy-user Digest, Vol 71, Issue 32
> ******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/cbd0f83e/attachment.html>

From robert.kern at gmail.com  Mon Jul 20 16:16:54 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 Jul 2009 15:16:54 -0500
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
Message-ID: <3d375d730907201316x1732fbe7y7c66d697389c4625@mail.gmail.com>

On Mon, Jul 20, 2009 at 15:11, Omer Khalid<Omer.Khalid at cern.ch> wrote:
> Hi Ivo,
>
> Thanks for your reply. But I am getting a little confused here now. It seems
> there are multiple ways to get the CDF for a distribution. You mean linspace
> function returns a CDF for a normal distribution.

No, not at all. linspace() returns uniformly increasing numbers
between the given endpoints. Ivo showed you how to find the empirical
CDF of the dataset. It is not related to any particular family of
probability distributions.

> As far as I understood from other sources is that scipy.stats.norm.cdf
> (mean, std) will return the CDF for the normal distribution or for
> non-normal distribution given one replace *norm* with the distributions
> name.

Yes.

> And what about scipy.stats.<dist>.fit function?

It will find the maximum likelihood parameters for fitting the given
distribution to your dataset. These parameters can be then used the
the <dist> object to return the fitted CDF, PDF, etc. The .fit()
method is not very flexible, though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ivo.maljevic at gmail.com  Mon Jul 20 16:22:05 2009
From: ivo.maljevic at gmail.com (Ivo Maljevic)
Date: Mon, 20 Jul 2009 16:22:05 -0400
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
Message-ID: <826c64da0907201322s290bef52x4ec362c4e510a8ca@mail.gmail.com>

Hi Omer,
The function you are mentioning, scipy.stats.norm.cdf (mean, std), will
return you the CDF for the normally distributed r.v. simply by using the
well known formula, but I typically don't know the distribution of the data
I'm processing, and I am interested at finding the discrete r.v.
distribution(s) from the actual data set. That's where histogram can be used
to represent the PDF and cumsum(histogram) or my "special" function for CDF
come into place.

I never used scipy.stats.<dist>.fit function, so I cannot help you with
that.

The bottom line is, are you interested in:

a) determening the distribution from the actual data without bothering to
know the exact formula and drawing conclusions (that is find moments,
probabilities,etc) from it (that is what I normally do)

b) try to determine what distribution your data fits the best (i.e., is it
normal, ricean, rayleigh, nakagammi, etc)

c) you just want to play with scipy for the purpose of learning and plot
various distributions using scipy.stats.<dist>.cdf

Ivo

2009/7/20 Omer Khalid <Omer.Khalid at cern.ch>

> Hi Ivo,
>
> Thanks for your reply. But I am getting a little confused here now. It
> seems there are multiple ways to get the CDF for a distribution. You mean
> linspace function returns a CDF for a normal distribution.
>
> As far as I understood from other sources is that scipy.stats.norm.cdf
> (mean, std) will return the CDF for the normal distribution or for
> non-normal distribution given one replace *norm* with the distributions
> name.
>
> And what about scipy.stats.<dist>.fit function?
> Thanks,
> Omer
>
>>
>>
>> Message: 7
>> Date: Mon, 20 Jul 2009 12:07:10 -0400
>> From: Ivo Maljevic <ivo.maljevic at gmail.com>
>> Subject: Re: [SciPy-user] CDF/PDF Stats with SciPy
>> To: SciPy Users List <scipy-user at scipy.org>
>> Message-ID:
>>        <826c64da0907200907m2775caa5jfe53ed515a456e5 at mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>>
>> Hi Omer,
>>
>> For histogram you can either use histogram function from numpy/scipy:
>>
>> from scipy import *
>>
>> x = .... # some vector
>> h,bins = histogram(x,50,normed=True,new=True) # find the histogram, number
>> of bins = 50
>>
>> or you can use pylab's version (good for plotting):
>>
>> import matplotlib.pyplot as plt
>>
>> count, bins, ignored = plt.hist(x, 50, normed=True)
>>
>> For CDF you can use cumsum function (standard approach), but for smaller
>> number of data points I
>> prefer to use all the points, which is a neat trick:
>>
>>
>> from scipy import *
>>
>> def my_cdf(x):
>>
>>  bins = sort(x)
>>  cdf  = linspace(0,1,len(bins))
>>  return [bins, cdf]
>>
>>
>> 2009/7/20 Omer Khalid <Omer.Khalid at cern.ch>
>>
>> > Hi Everybody,
>> >
>> > I am new to Python and new to SciPy libraries. I wanted to take some
>> ques
>> > from the experts here on the list before dive into SciPy world.
>> >
>> > I was wondering if some one could provide a rough guide about how to run
>> > two stats functions: Cumulative Distribution Function (CDF) and
>> Probability
>> > Distribution Function (PDF).
>> >
>> > My use case is the following: I have a sampleSpaceList [] which have
>> 1000
>> > floating point values. When a new floating point value is generated in
>> my
>> > program, I would like to run both CDF and PDF on the sampleList for it
>> and
>> > get the probabilty of value less or equal for CDF and probablity
>> > distribution for PDF.
>> >
>> > Many thanks in advance!
>> >
>> > Omer
>> >
>> >
>> > _______________________________________________
>> > SciPy-user mailing list
>> > SciPy-user at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-user
>> >
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/bb4aebe5/attachment-0001.html
>>
>> ------------------------------
>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>
>> End of SciPy-user Digest, Vol 71, Issue 32
>> ******************************************
>>
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/7486fc8e/attachment.html>

From ivo.maljevic at gmail.com  Mon Jul 20 16:25:14 2009
From: ivo.maljevic at gmail.com (Ivo Maljevic)
Date: Mon, 20 Jul 2009 16:25:14 -0400
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <3d375d730907201316x1732fbe7y7c66d697389c4625@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
	<3d375d730907201316x1732fbe7y7c66d697389c4625@mail.gmail.com>
Message-ID: <826c64da0907201325h4def7868n29fb128de66b10c7@mail.gmail.com>

Thanks Robert, I wasn't paying attention to that part of Omer's question.

2009/7/20 Robert Kern <robert.kern at gmail.com>

> On Mon, Jul 20, 2009 at 15:11, Omer Khalid<Omer.Khalid at cern.ch> wrote:
> > Hi Ivo,
> >
> > Thanks for your reply. But I am getting a little confused here now. It
> seems
> > there are multiple ways to get the CDF for a distribution. You mean
> linspace
> > function returns a CDF for a normal distribution.
>
> No, not at all. linspace() returns uniformly increasing numbers
> between the given endpoints. Ivo showed you how to find the empirical
> CDF of the dataset. It is not related to any particular family of
> probability distributions.
>
> > As far as I understood from other sources is that scipy.stats.norm.cdf
> > (mean, std) will return the CDF for the normal distribution or for
> > non-normal distribution given one replace *norm* with the distributions
> > name.
>
> Yes.
>
> > And what about scipy.stats.<dist>.fit function?
>
> It will find the maximum likelihood parameters for fitting the given
> distribution to your dataset. These parameters can be then used the
> the <dist> object to return the fitted CDF, PDF, etc. The .fit()
> method is not very flexible, though.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
>  -- Umberto Eco
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/85f93b0b/attachment.html>

From karl.young at ucsf.edu  Mon Jul 20 16:18:33 2009
From: karl.young at ucsf.edu (Karl Young)
Date: Mon, 20 Jul 2009 13:18:33 -0700
Subject: [SciPy-user] build error for scipy-0.7.1 on CentOS 5
In-Reply-To: <4A588FF0.9050601@lpta.in2p3.fr>
References: <785491.66970.qm@web34405.mail.mud.yahoo.com>
	<4A588FF0.9050601@lpta.in2p3.fr>
Message-ID: <4A64D119.2010105@ucsf.edu>


I get the following error when trying to build scipy-0.7.1 on 64 bit 
CentOS 5 (Linux 2.6.18-128.2.1.el5 #1 SMP x86_64 GNU/Linux):

----------------------------------------------------------------------------------------------

gcc -pthread -shared 
build/temp.linux-x86_64-2.5/build/src.linux-x86_64-2.5/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.o 
-L/usr/local/lib -Lbuild/temp.linux-x86_64-2.5 -lumfpack -lamd 
-lptf77blas -lptcblas -latlas -o 
build/lib.linux-x86_64-2.5/scipy/sparse/linalg/dsolve/umfpack/__umfpack.so
/usr/bin/ld: /usr/local/lib/libumfpack.a(umfpack_di_col_to_triplet.o): 
relocation R_X86_64_32 against `a local symbol' can not be used when 
making a shared object; recompile with -fPIC
/usr/local/lib/libumfpack.a: could not read symbols: Bad value
collect2: ld returned 1 exit status

----------------------------------------------------------------------------------------------

I built UMFPACK (version 5.4.0) as per David's instructions on 
http://www.scipy.org/Installing_SciPy/Linux
Specifically I checked that:

CC = gcc
CFLAGS = -O3 -fexceptions -m64 -fPIC
F77 = gfortran
F77FLAGS = -O -m64 -fPIC

were set in UFconfig/UFconfig.mk.

Has anybody seen this problem and know what to do about it ? Thanks,

-- KY


From Omer.Khalid at cern.ch  Mon Jul 20 16:36:12 2009
From: Omer.Khalid at cern.ch (Omer Khalid)
Date: Mon, 20 Jul 2009 22:36:12 +0200
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <826c64da0907201322s290bef52x4ec362c4e510a8ca@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com> 
	<826c64da0907201322s290bef52x4ec362c4e510a8ca@mail.gmail.com>
Message-ID: <77e5896b0907201336ld9aebd5i2a1e9424cb1c31ac@mail.gmail.com>

Hi Ivo,


> The bottom line is, are you interested in:
>
> a) determining the distribution from the actual data without bothering to
> know the exact formula and drawing conclusions (that is find moments,
> probabilities,etc) from it (that is what I normally do)


Yes, I am interested in this.



> b) try to determine what distribution your data fits the best (i.e., is it
> normal, ricean, rayleigh, nakagammi, etc)


This is partially true..

I think I should have explained more of my research question. My program is
generating a real number variate X for every success. I keep on storing X
for each success cycle of my program and once the sample list is size 1000;
then I would like to use that sample space to determine the probability for
every next X and again store it until the sample space reaches 1000.

I am not really concerned with the distribution type of my sample space, so
i thought (may be out of ignorance) that I first must determine the
distribution type using the fit function and then get the mean/std. Once I
have mean/std, then i get CDF probability for every next X and store it my
sample list replace the previous once.

Basically, I want to get a probability for every X in my program cycle till
the next sample space reaches 1000, and keeps on doing it. This way I am
assuming my algorithm will learn to improve.

But I could not figure out the proper python code  yet for this....

Thanks,
Omer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/57c098c3/attachment.html>

From Omer.Khalid at cern.ch  Mon Jul 20 16:39:06 2009
From: Omer.Khalid at cern.ch (Omer Khalid)
Date: Mon, 20 Jul 2009 22:39:06 +0200
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <3d375d730907201316x1732fbe7y7c66d697389c4625@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com> 
	<3d375d730907201316x1732fbe7y7c66d697389c4625@mail.gmail.com>
Message-ID: <77e5896b0907201339g367af6e5lc086b4d79fc3c053@mail.gmail.com>

Hi Robert,

Thanks for your rpely.


> >
> > Thanks for your reply. But I am getting a little confused here now. It
> seems
> > there are multiple ways to get the CDF for a distribution. You mean
> linspace
> > function returns a CDF for a normal distribution.
>
> No, not at all. linspace() returns uniformly increasing numbers
> between the given endpoints. Ivo showed you how to find the empirical
> CDF of the dataset. It is not related to any particular family of
> probability distributions.


Sorry for my ignorance, but what is Empirical CDF and how's that different
from a CDF lets say for Normal distribution or Piosson other than that it's
calculated differently?

>
>
> > As far as I understood from other sources is that scipy.stats.norm.cdf
> > (mean, std) will return the CDF for the normal distribution or for
> > non-normal distribution given one replace *norm* with the distributions
> > name.
>
> Yes.
>

OK, so that I understood correctly :-)


>
> > And what about scipy.stats.<dist>.fit function?
>
> It will find the maximum likelihood parameters for fitting the given
> distribution to your dataset. These parameters can be then used the
> the <dist> object to return the fitted CDF, PDF, etc. The .fit()
> method is not very flexible, though.


Sorry, what do you mean that .fit() is not very flexible?

And can you explain with some python code as then it would be easier for me
to understand.

Thanks,
Omer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/ce0bd6d9/attachment.html>

From robert.kern at gmail.com  Mon Jul 20 16:45:36 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 Jul 2009 15:45:36 -0500
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <77e5896b0907201339g367af6e5lc086b4d79fc3c053@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com> 
	<3d375d730907201316x1732fbe7y7c66d697389c4625@mail.gmail.com> 
	<77e5896b0907201339g367af6e5lc086b4d79fc3c053@mail.gmail.com>
Message-ID: <3d375d730907201345w6410e89bg7a6fc740a2ea2a35@mail.gmail.com>

On Mon, Jul 20, 2009 at 15:39, Omer Khalid<Omer.Khalid at cern.ch> wrote:
> Hi Robert,
>
> Thanks for your rpely.
>
>>
>> >
>> > Thanks for your reply. But I am getting a little confused here now. It
>> > seems
>> > there are multiple ways to get the CDF for a distribution. You mean
>> > linspace
>> > function returns a CDF for a normal distribution.
>>
>> No, not at all. linspace() returns uniformly increasing numbers
>> between the given endpoints. Ivo showed you how to find the empirical
>> CDF of the dataset. It is not related to any particular family of
>> probability distributions.
>
> Sorry for my ignorance, but what is Empirical CDF and how's that different
> from a CDF lets say for Normal distribution or Piosson other than that it's
> calculated differently?

It is an estimate of the CDF given just the data, without trying to
fit the data to a particular model.

>> > As far as I understood from other sources is that scipy.stats.norm.cdf
>> > (mean, std) will return the CDF for the normal distribution or for
>> > non-normal distribution given one replace *norm* with the distributions
>> > name.
>>
>> Yes.
>
> OK, so that I understood correctly :-)
>
>>
>> > And what about scipy.stats.<dist>.fit function?
>>
>> It will find the maximum likelihood parameters for fitting the given
>> distribution to your dataset. These parameters can be then used the
>> the <dist> object to return the fitted CDF, PDF, etc. The .fit()
>> method is not very flexible, though.
>
> Sorry, what do you mean that .fit() is not very flexible?

It fits all of the parameters for the distribution object. It does not
let you hold certain parameters fixed. This can be a problem because
some of the distribution object have nonstandard parameters. For
example, all of the distribution objects have a "location" parameter,
but a number of distributions like gamma don't normally have such a
parameter. If you wanted to fit a gamma distribution, you would
usually want to hold loc=0 fixed.

> And can you explain with some python code as then it would be easier for me
> to understand.

I do not have time right now.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ivo.maljevic at gmail.com  Mon Jul 20 16:51:21 2009
From: ivo.maljevic at gmail.com (Ivo Maljevic)
Date: Mon, 20 Jul 2009 16:51:21 -0400
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <77e5896b0907201336ld9aebd5i2a1e9424cb1c31ac@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
	<826c64da0907201322s290bef52x4ec362c4e510a8ca@mail.gmail.com>
	<77e5896b0907201336ld9aebd5i2a1e9424cb1c31ac@mail.gmail.com>
Message-ID: <826c64da0907201351w774883b5xc465a99bb2e15093@mail.gmail.com>

I am not sure I quite understand what you are doing (the first criterion is
the success of an experiment, and the second criterion is based on
statistics of the first test?), but regardless of what you are doing, you
can apply my_cdf() function I gave you to get the discrete CDF (or you can
get the cumulative CDF as 1-CDF). To elaborate a little more on why I prefer
this CDF approach. Quite often, r.v.'s have long tails, and they tend to
disapper when you do numerical integration (cumsum is the most basic
approach) on the estimated pdf (which is the histogram). When you use all
the available data points (instead of just 50 or so), you get much better
results.

Once you find the CDF, you should be able the get your probabilities
directly by reading off the plot values or by finding which Y-axis value
(which is the probability) matches whatever bin you are interested in (on
X-axis).

I don't know if this helps. If not, and if you have some real data, maybe I
can write you some more code.

Ivo

2009/7/20 Omer Khalid <Omer.Khalid at cern.ch>

> Hi Ivo,
>
>
>> The bottom line is, are you interested in:
>>
>> a) determining the distribution from the actual data without bothering to
>> know the exact formula and drawing conclusions (that is find moments,
>> probabilities,etc) from it (that is what I normally do)
>
>
> Yes, I am interested in this.
>
>
>
>> b) try to determine what distribution your data fits the best (i.e., is it
>> normal, ricean, rayleigh, nakagammi, etc)
>
>
> This is partially true..
>
> I think I should have explained more of my research question. My program is
> generating a real number variate X for every success. I keep on storing X
> for each success cycle of my program and once the sample list is size 1000;
> then I would like to use that sample space to determine the probability for
> every next X and again store it until the sample space reaches 1000.
>
> I am not really concerned with the distribution type of my sample space, so
> i thought (may be out of ignorance) that I first must determine the
> distribution type using the fit function and then get the mean/std. Once I
> have mean/std, then i get CDF probability for every next X and store it my
> sample list replace the previous once.
>
> Basically, I want to get a probability for every X in my program cycle till
> the next sample space reaches 1000, and keeps on doing it. This way I am
> assuming my algorithm will learn to improve.
>
> But I could not figure out the proper python code  yet for this....
>
> Thanks,
> Omer
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/3d1bb6a7/attachment.html>

From karl.young at ucsf.edu  Mon Jul 20 16:55:43 2009
From: karl.young at ucsf.edu (Karl Young)
Date: Mon, 20 Jul 2009 13:55:43 -0700
Subject: [SciPy-user] build error for scipy-0.7.1 on CentOS 5 - oops
	sorry
In-Reply-To: <4A64D119.2010105@ucsf.edu>
References: <785491.66970.qm@web34405.mail.mud.yahoo.com>	<4A588FF0.9050601@lpta.in2p3.fr>
	<4A64D119.2010105@ucsf.edu>
Message-ID: <4A64D9CF.70204@ucsf.edu>


Sorry for the noise - I found the problem re. this question; it's that I 
didn't notice that UFconfig/UFconfig.mk had multiple places for 
specifying CFLAGS and there was one below the one I modified which was 
uncommented and overrode  the one I set containing -fPIC.


> I get the following error when trying to build scipy-0.7.1 on 64 bit 
> CentOS 5 (Linux 2.6.18-128.2.1.el5 #1 SMP x86_64 GNU/Linux):
>
> ----------------------------------------------------------------------------------------------
>
> gcc -pthread -shared 
> build/temp.linux-x86_64-2.5/build/src.linux-x86_64-2.5/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.o 
> -L/usr/local/lib -Lbuild/temp.linux-x86_64-2.5 -lumfpack -lamd 
> -lptf77blas -lptcblas -latlas -o 
> build/lib.linux-x86_64-2.5/scipy/sparse/linalg/dsolve/umfpack/__umfpack.so
> /usr/bin/ld: /usr/local/lib/libumfpack.a(umfpack_di_col_to_triplet.o): 
> relocation R_X86_64_32 against `a local symbol' can not be used when 
> making a shared object; recompile with -fPIC
> /usr/local/lib/libumfpack.a: could not read symbols: Bad value
> collect2: ld returned 1 exit status
>
> ----------------------------------------------------------------------------------------------
>
> I built UMFPACK (version 5.4.0) as per David's instructions on 
> http://www.scipy.org/Installing_SciPy/Linux
> Specifically I checked that:
>
> CC = gcc
> CFLAGS = -O3 -fexceptions -m64 -fPIC
> F77 = gfortran
> F77FLAGS = -O -m64 -fPIC
>
> were set in UFconfig/UFconfig.mk.
>
> Has anybody seen this problem and know what to do about it ? Thanks,
>
> -- KY
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>   



From ivo.maljevic at gmail.com  Mon Jul 20 19:12:01 2009
From: ivo.maljevic at gmail.com (Ivo Maljevic)
Date: Mon, 20 Jul 2009 19:12:01 -0400
Subject: [SciPy-user] CDF/PDF Stats with SciPy
In-Reply-To: <826c64da0907201351w774883b5xc465a99bb2e15093@mail.gmail.com>
References: <77e5896b0907201311w28107eaaje4e4dcf88401e6b5@mail.gmail.com>
	<826c64da0907201322s290bef52x4ec362c4e510a8ca@mail.gmail.com>
	<77e5896b0907201336ld9aebd5i2a1e9424cb1c31ac@mail.gmail.com>
	<826c64da0907201351w774883b5xc465a99bb2e15093@mail.gmail.com>
Message-ID: <826c64da0907201612t2533459crd35d65987ea0ce6c@mail.gmail.com>

I meant complementary CDF (CCDF=1-CDF).

2009/7/20 Ivo Maljevic <ivo.maljevic at gmail.com>

> I am not sure I quite understand what you are doing (the first criterion is
> the success of an experiment, and the second criterion is based on
> statistics of the first test?), but regardless of what you are doing, you
> can apply my_cdf() function I gave you to get the discrete CDF (or you can
> get the cumulative CDF as 1-CDF). To elaborate a little more on why I prefer
> this CDF approach. Quite often, r.v.'s have long tails, and they tend to
> disapper when you do numerical integration (cumsum is the most basic
> approach) on the estimated pdf (which is the histogram). When you use all
> the available data points (instead of just 50 or so), you get much better
> results.
>
> Once you find the CDF, you should be able the get your probabilities
> directly by reading off the plot values or by finding which Y-axis value
> (which is the probability) matches whatever bin you are interested in (on
> X-axis).
>
> I don't know if this helps. If not, and if you have some real data, maybe I
> can write you some more code.
>
> Ivo
>
> 2009/7/20 Omer Khalid <Omer.Khalid at cern.ch>
>
>> Hi Ivo,
>>
>>
>>
>>> The bottom line is, are you interested in:
>>>
>>> a) determining the distribution from the actual data without bothering to
>>> know the exact formula and drawing conclusions (that is find moments,
>>> probabilities,etc) from it (that is what I normally do)
>>
>>
>> Yes, I am interested in this.
>>
>>
>>
>>> b) try to determine what distribution your data fits the best (i.e., is
>>> it normal, ricean, rayleigh, nakagammi, etc)
>>
>>
>> This is partially true..
>>
>> I think I should have explained more of my research question. My program
>> is generating a real number variate X for every success. I keep on storing X
>> for each success cycle of my program and once the sample list is size 1000;
>> then I would like to use that sample space to determine the probability for
>> every next X and again store it until the sample space reaches 1000.
>>
>> I am not really concerned with the distribution type of my sample space,
>> so i thought (may be out of ignorance) that I first must determine the
>> distribution type using the fit function and then get the mean/std. Once I
>> have mean/std, then i get CDF probability for every next X and store it my
>> sample list replace the previous once.
>>
>> Basically, I want to get a probability for every X in my program cycle
>> till the next sample space reaches 1000, and keeps on doing it. This way I
>> am assuming my algorithm will learn to improve.
>>
>> But I could not figure out the proper python code  yet for this....
>>
>> Thanks,
>> Omer
>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090720/b1929197/attachment.html>

From brennan.williams at visualreservoir.com  Mon Jul 20 23:26:13 2009
From: brennan.williams at visualreservoir.com (Brennan Williams)
Date: Tue, 21 Jul 2009 15:26:13 +1200
Subject: [SciPy-user] piecewise continuous distribution
Message-ID: <4A653555.102@visualreservoir.com>

I have a user-specified  piecewise continuous distribution defined by 
pairs of (value, cumulative probability).

For example...

Value ,    Cumulative Probability
---------------------------
1.0, 0.0
1.5, 0.2
2.3, 0.4
3.5, 0.6
5.2, 0.8
8.5, 1.0

So no values below 1.0, none above 8.5.
Between adjacent specified values the generated values will be uniformly 
distributed,
i.e. the cumulative probability graph is made up of straight lines 
between adjacent points/values.

The generated values are created by a piece of TCL code.

What I need to do is assign a pdf probability value to each generated value.

How would I do this? What is the best way to do it in scipy.stats?

BTW I'm new to the stats capabilities of scipy. My app also supports 
normal, log normal and triangular distributions
and I think I've worked out how to use scipy.stats.norm, .lognorm and 
...triang

Thanks

Brennan




From robert.kern at gmail.com  Mon Jul 20 23:45:32 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 Jul 2009 22:45:32 -0500
Subject: [SciPy-user] piecewise continuous distribution
In-Reply-To: <4A653555.102@visualreservoir.com>
References: <4A653555.102@visualreservoir.com>
Message-ID: <3d375d730907202045h19631459t5026dee458a1f514@mail.gmail.com>

On Mon, Jul 20, 2009 at 22:26, Brennan
Williams<brennan.williams at visualreservoir.com> wrote:
> I have a user-specified ?piecewise continuous distribution defined by
> pairs of (value, cumulative probability).
>
> For example...
>
> Value , ? ?Cumulative Probability
> ---------------------------
> 1.0, 0.0
> 1.5, 0.2
> 2.3, 0.4
> 3.5, 0.6
> 5.2, 0.8
> 8.5, 1.0
>
> So no values below 1.0, none above 8.5.
> Between adjacent specified values the generated values will be uniformly
> distributed,
> i.e. the cumulative probability graph is made up of straight lines
> between adjacent points/values.
>
> The generated values are created by a piece of TCL code.
>
> What I need to do is assign a pdf probability value to each generated value.
>
> How would I do this?

In [25]: x0 = np.array([1.0, 1.5, 2.3, 3.5, 5.2, 8.5])

In [26]: cdf = np.linspace(0, 1, len(x0))

In [27]: def pdf(x):
    y = np.zeros_like(x)
    inside = (x >= x0[0]) & (x <= x0[-1])
    x = x[inside]
    i = x0.searchsorted(x).clip(0, len(x0)-2)
    block_pdf = np.diff(cdf) / np.diff(x0)
    y[inside] = block_pdf[i]
    return y
   ....:

In [35]: pdf(np.linspace(0.0, 10.0))
Out[35]:
array([ 0.        ,  0.        ,  0.        ,  0.        ,  0.        ,
        0.25      ,  0.25      ,  0.25      ,  0.16666667,  0.16666667,
        0.16666667,  0.16666667,  0.11764706,  0.11764706,  0.11764706,
        0.11764706,  0.11764706,  0.11764706,  0.06060606,  0.06060606,
        0.06060606,  0.06060606,  0.06060606,  0.06060606,  0.06060606,
        0.06060606,  0.06060606,  0.06060606,  0.06060606,  0.06060606,
        0.06060606,  0.06060606,  0.06060606,  0.06060606,  0.06060606,
        0.06060606,  0.06060606,  0.06060606,  0.06060606,  0.06060606,
        0.06060606,  0.06060606,  0.        ,  0.        ,  0.        ,
        0.        ,  0.        ,  0.        ,  0.        ,  0.        ])


> What is the best way to do it in scipy.stats?

Neither here nor there, really.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From cimrman3 at ntc.zcu.cz  Tue Jul 21 09:40:56 2009
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Tue, 21 Jul 2009 15:40:56 +0200
Subject: [SciPy-user] ANN: SfePy 2009.3
Message-ID: <4A65C568.1070801@ntc.zcu.cz>

I am pleased to announce release 2009.3 of SfePy.

Description
-----------
SfePy (simple finite elements in Python) is a software, distributed 
under the BSD license, for solving systems of coupled partial 
differential equations by the finite element method. The code is based 
on NumPy and SciPy packages.

Mailing lists, issue tracking, git repository: http://sfepy.org
Home page: http://sfepy.kme.zcu.cz

Highlights of this release
--------------------------
Finally, SfePy has a basic support for Windows installation via numpy 
distutils:
- still very experimental!
- the tests will not finish if umfpack is not installed, as the default 
direct solver in scipy cannot handle some problems (see recent 
sfepy-devel mailing list discussions).

Major improvements
------------------
- new scripts:
   - cylindergen.py: cylindrical mesh generator
- updated scripts:
   - postproc.py:
     - quite usable now for fast first glance at the results
     - plots point, cell data of all kinds (scalar, vector, tensor)
     - Viewer is much more configurable
   - probe.py:
     - can probe selected quantities only
   - isfepy:
     - Viewer is much more configurable
- new tests and terms

Applications
------------
- phononic materials:
    - plotting improved
    - caching of eigen-problem solution and Christoffel acoustic tensor
- schroedinger.py:
    - choose and call DFT solver via solver interface

People who contributed to this release: Vladimir Lukes.

For more information on this release, see
http://sfepy.googlecode.com/svn/web/releases/2009.3_RELEASE_NOTES.txt

Best regards,
Robert Cimrman



From millman at berkeley.edu  Wed Jul 22 12:01:17 2009
From: millman at berkeley.edu (Jarrod Millman)
Date: Wed, 22 Jul 2009 09:01:17 -0700
Subject: [SciPy-user] ANN: SciPy 2009 early registration ends today
Message-ID: <c7009a550907220901i16081db0v5e2dbb3ed094851f@mail.gmail.com>

Today is the last day to register for SciPy 2009 at the early bird rates.
Please register (http://conference.scipy.org/to_register )
by the end of the day to take advantage of the reduced early registration
rate.

The conference schedule is available here:
 http://conference.scipy.org/schedule

The special group rate for the Marriot Hotel is no longer available.  However,
there are a number of closer and less expensive choices still available:
 http://admissions.caltech.edu/visiting/accommodations
I've been staying at the Vagabond Inn for the last several years:
 http://www.vagabondinn-pasadena-hotel.com/
It is within easy walking distance of the conference and has just been
completely renovated.  Rooms at the Vagabond start at $79/night.

About the conference
--------------------

SciPy 2009, the 8th Python in Science conference, will be held from
August 18-23, 2009 at Caltech in Pasadena, CA, USA.  The conference
starts with two days of tutorials to the scientific Python tools.
There will be two tracks, one for introduction of the basic tools to
beginners, and one for more advanced tools.  The tutorials will be
followed by two days of talks.  Both days of talks will begin with a
keynote address.  The first day?s keynote will be given by Peter
Norvig, the Director of Research at Google; while, the second keynote
will be delivered by Jon Guyer, a Materials Scientist in the
Thermodynamics and Kinetics Group at NIST.  The program committee will
select the remaining talks from submissions to our call for papers.
All selected talks will be included in our conference proceedings
edited by the program committee.  After the talks each day we will
provide several rooms for impromptu birds of a feather discussions.
Finally, the last two days of the conference will be used for a number
of coding sprints on the major software projects in our community.

For the 8th consecutive year, the conference will bring together the
developers and users of the open source software stack for scientific
computing with Python.  Attendees have the opportunity to review the
available tools and how they apply to specific problems.  By providing
a forum for developers to share their Python expertise with the wider
commercial, academic, and research communities, this conference
fosters collaboration and facilitates the sharing of software
components, techniques, and a vision for high level language use in
scientific computing.

For further information, please visit the conference homepage:
http://conference.scipy.org.

Important Dates
---------------

* Friday, July 3: Abstracts Due
* Wednesday, July 15: Announce accepted talks, post schedule
* Wednesday, July 22: Early Registration ends
* Tuesday-Wednesday, August 18-19: Tutorials
* Thursday-Friday, August 20-21: Conference
* Saturday-Sunday, August 22-23: Sprints
* Friday, September 4: Papers for proceedings due

Executive Committee
-------------------

* Jarrod Millman, UC Berkeley, USA (Conference Chair)
* Ga?l Varoquaux, INRIA Saclay, France (Program Co-Chair)
* St?fan van der Walt, University of Stellenbosch, South Africa
(Program Co-Chair)
* Fernando P?rez, UC Berkeley, USA (Tutorial Chair)


From eadrogue at gmx.net  Wed Jul 22 12:33:32 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Wed, 22 Jul 2009 18:33:32 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with
	gradient	information
In-Reply-To: <ec9f80fa0907200143j6a6748dx62d605991341d74d@mail.gmail.com>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
	<20090719110543.GA7138@doriath.local>
	<ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>
	<20090719183923.GA21105@doriath.local>
	<ec9f80fa0907200143j6a6748dx62d605991341d74d@mail.gmail.com>
Message-ID: <20090722163332.GA3515@doriath.local>

20/07/09 @ 10:43 (+0200), thus spake Sebastian Walter:
> thanks for the function :).
>  It is now part of pyadolc's unit test.
> http://github.com/b45ch1/pyadolc/blob/3cfff80c062d43e812379f4606eda0ebaaf4e82c/tests/complicated_tests.py
>  , Line 246
> 
> I added two functions: one is providing the gradient by finite
> differences and the other by using automatic differentiation.
> The finite differences gradient has very poor accuracy, only the first
> three digits are correct.

So what your automatic differentiation does when it reaches
a point where the function is not differentiable?

Scipy's optimization.approx_fprime() returns an arbitrarily
large value. For example, let's suppose that f(x) is

In [164]: def f(x):
    if x[0] > 2:
        return x[0]**2
    return 0

then the derivative of f(2) doesn't exist mathematically
speaking. f'(x<2) is 0, and f'(x>2) is 2*x, if I understand
correctly. This is the output of approx_fprime for function f:

In [162]: [opt.approx_fprime((i,),f,eps) for i in numpy.linspace(1.95,2.05,9)]
Out[162]: 
[array([ 0.]),
 array([ 0.]),
 array([ 0.]),
 array([ 0.]),
 array([  2.68435460e+08]),
 array([ 4.02500004]),
 array([ 4.05000001]),
 array([ 4.07500005]),
 array([ 4.09999996])]

As you can see, it returns a large number for f'(2).
My question is, for the purposes of optimising f(x), what
should my gradient function return at x=2, so that the
optimisation algorithm works well. I would have said it should
return 0, but seeing what approx_fprime does, I'm not sure any more.

Ernest



From perfreem at gmail.com  Wed Jul 22 23:02:31 2009
From: perfreem at gmail.com (per freem)
Date: Wed, 22 Jul 2009 23:02:31 -0400
Subject: [SciPy-user] a pdf for multivariate normal distribution?
Message-ID: <e95b09750907222002s11b8fe4ep7f0cffba8ff5fbb6@mail.gmail.com>

hi all,

i'm trying to find the function for the pdf of a multivariate normal pdf. i
know that multivariate_normal can be used to sample from the multivariate
normal distribution, but i just want to get the pdf for a given vector of
means and a covariance matrix. is there a function to do this?

thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090722/37a2ece5/attachment.html>

From sebastian.walter at gmail.com  Thu Jul 23 04:42:21 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Thu, 23 Jul 2009 10:42:21 +0200
Subject: [SciPy-user] optimization using fmin_bfgs with gradient
	information
In-Reply-To: <20090722163332.GA3515@doriath.local>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
	<20090719110543.GA7138@doriath.local>
	<ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>
	<20090719183923.GA21105@doriath.local>
	<ec9f80fa0907200143j6a6748dx62d605991341d74d@mail.gmail.com>
	<20090722163332.GA3515@doriath.local>
Message-ID: <ec9f80fa0907230142q2b8c103co497ecca8ad9255a5@mail.gmail.com>

1)
approx_fprime uses probably finite differences to approximate the
gradient, so at x = 2
it computes  (f(2+epsilon) - f(2))/epsilon =  (2+epsilon)**2/epsilon
which can be very large

2) When using AD to evaluate derivatives, only one path through the
control flow graph (which is defined by the if statement in your
function) is taken.
I.e. if x<2, AD will not know that for x>=2 it would have taken
another way through the control flow graph of your algorithm.

I.e. it  would compute
f'(x<2) = 0 and f'(x>=2) = x**2  without realizing that the function
is nondifferentiable at that point.

3)
If you have such an objective function as you have shown above, then
your optimization problem is not well-formed because it does not have
a real minimizer x_*.
I'd assume that your objective function looks more like
f(x) = abs(x)
which is non-differentiable at x=0.
This kind of objective function destroys the convergence properties of
algorithms that assume continuously differentiable objective
functions, i.e. the convergence to the minimizer can be very slow.
For such problems, special purpose algorithms, e.g.  the so-called
"bundle methods" can be used which converge superlinearly, as far as I
know.


Sebastian





2009/7/22 Ernest Adrogu? <eadrogue at gmx.net>:
> 20/07/09 @ 10:43 (+0200), thus spake Sebastian Walter:
>> thanks for the function :).
>> ?It is now part of pyadolc's unit test.
>> http://github.com/b45ch1/pyadolc/blob/3cfff80c062d43e812379f4606eda0ebaaf4e82c/tests/complicated_tests.py
>> ?, Line 246
>>
>> I added two functions: one is providing the gradient by finite
>> differences and the other by using automatic differentiation.
>> The finite differences gradient has very poor accuracy, only the first
>> three digits are correct.
>
> So what your automatic differentiation does when it reaches
> a point where the function is not differentiable?
>
> Scipy's optimization.approx_fprime() returns an arbitrarily
> large value. For example, let's suppose that f(x) is
>
> In [164]: def f(x):
> ? ?if x[0] > 2:
> ? ? ? ?return x[0]**2
> ? ?return 0
>
> then the derivative of f(2) doesn't exist mathematically
> speaking. f'(x<2) is 0, and f'(x>2) is 2*x, if I understand
> correctly. This is the output of approx_fprime for function f:
>
> In [162]: [opt.approx_fprime((i,),f,eps) for i in numpy.linspace(1.95,2.05,9)]
> Out[162]:
> [array([ 0.]),
> ?array([ 0.]),
> ?array([ 0.]),
> ?array([ 0.]),
> ?array([ ?2.68435460e+08]),
> ?array([ 4.02500004]),
> ?array([ 4.05000001]),
> ?array([ 4.07500005]),
> ?array([ 4.09999996])]
>
> As you can see, it returns a large number for f'(2).
> My question is, for the purposes of optimising f(x), what
> should my gradient function return at x=2, so that the
> optimisation algorithm works well. I would have said it should
> return 0, but seeing what approx_fprime does, I'm not sure any more.
>
> Ernest
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From emmanuelle.gouillart at normalesup.org  Thu Jul 23 10:17:37 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Thu, 23 Jul 2009 16:17:37 +0200
Subject: [SciPy-user] Numpy/Scipy and the Python African Tour
Message-ID: <20090723141737.GF10321@phare.normalesup.org>

Dear users of Numpy and Scipy,

here is an informal report on the last event of the Python African Tour,
which took place in Dakar (Senegal) on July 6-10th. It might interest
only a fraction of the lists, so I apologize for the spamming.

What is the Python African Tour?
----------------------------------

It is a recent initiative to promote the use of Python in African
countries, that we owe to Kamon Ayeva (http://kayeva.wordpress.com/).
Born a Togolese, Kamon now lives in Paris (France) where he works as a
web developer and a trainer in web technologies. It occurred to him that
but a few African developers (Hi, Stefan! :D) participate in
Python-related development or conferences. IT-related technologies and
web development can contribute to economic growth in developing
countries, and, because of the clarity and flexibility of the language,
Python-trained developers may have a competitive advantage to develop
successful businesses.

The Python African Tour sends a few volunteers to places in Africa where
a small core of Python users or developers already exists. These deliver
a training over a few days, about Python and some of its applications.
The goal is to create a small but dynamical community of users that will
keep on growing after the volunteers have left.

A first event was held in Morocco last December. Whereas the emphasis had
been put on an introduction to the Python language and to the Django web
framework, I proposed to give this time an additional course about the
use of Python in scientific computing, which was nicknamed "Scientific
Python". Numpy/Scipy is an opportunity to present the language as
attractive for not only geeks, but also academic staff -- the people who
may precisely teach the language in the universities and engineering
departments.

Organization of the Dakar event
----------------------------------

This year, three European trainers flew to Dakar for the event: Kamon
Ayeva, Daniel Nouri and myself. For all courses we worked in pairs with
three local organizers and Python experts: Thomas No?l, Ousmane Wilane
and Sergiu Mihai. The course was organized as follows:

* 1.5 day for a general introduction to Python
* 1 day of specialization, either on the Django web framework (Kamon and
  Daniel) or on Python for scientific computing (Emmanuelle)
* 1 day of barcamp, or informal talks showing how we would use Python in
  our daily work.
* 1.5 day of "sprint", I should rather say practical work.

The hosting organizations were the AUF (Agence universitaire de la
Francophonie), the Dakar Linux User Group (DakarLUG) and the ESP (Ecole
Sup?rieure Polytechnique, a master-level engineer school).

The local team did a great job on announcing the event, rounding up
potential attendees (and sponsors!), and selecting people. The course was
free of charge, but there were only a limited number of positions.
Finally, about 50 people showed up at the course (20 having chosen the
Scientific Python course). Facilities at AUF were excellent: three rooms
with a total of 70 PCs running the latest Ubuntu.

The course on "Scientific Python"
----------------------------------

Course material
...............

Whereas we reused the slides of the Moroccan event on the introduction to
Python, I wrote the slides for the 3-4 hour course about Scientific
Python. The slides were written in French, English being an issue for a
part of the students. They can be found on
http://www.dakarlug.org/pat/scientifique/html/ for French-reading people
(my apologies to the others! I'm considering translating the slides to
English but I will do it more rapidly if I'm given the incentive to do
so, so tell me if you may be interested by an English version!).

Given the duration of the course and the fact that the trainees had
already been introduced to Python, I chose to concentrate mostly on
Numpy, and on Scipy to a lesser extent. However, I tried to explain as
soon as possible how to use numpy and scipy in a "real life" scientific
workflow: plotting data with matplotlib or mayavi, opening data files,
finding documentation, etc.

How it went
............

Together with Thomas No?l, we followed the same group of 20 people during
the training week. The group was composed of both students (master and
PhD), and a few senior academic staff, professors and researchers. I'm
glad to say that the course was a real success, insofar as everybody was
convinced he or she could use Python with some benefit for his or her
research. Most people were already using either Matlab (with some license
issues...), R or Java for scientific computing. They were attracted by
the following features of Python:
* Python is a free software, so there are no license problems
* with its wide range of scientific modules, almost *everything* can be
  done with Python
Also, people were encouraged when they saw at the end of the week that
they could work on their own data using their usual algorithms in Python,
without too many difficulties.

Speaking about difficulties, one may say that there was some disparity in
the level of experience inside the group, some being very well at ease
with array programming while others were obviously discovering
programming. It was therefore difficult to meet everybody's needs, but
well, I hope everybody learned something!

Some pictures of the event are on http://dakarlug.org/pat/galerie/.

What's next: outlook on the PAT
--------------------------------

The Dakar event was a very encouraging one, and some other countries
(Togo or Zambia) are already considered for the next stop of the PAT. Of
course, before keeping moving towards the next event, we first would
like to see what happens in Dakar after the PAT has left. Some
pedagogical staff decided there would be a course on Python next academic
year, and this is a very positive decision: the one-week course was
definitely too short for the students. Now the big question is who is
going to teach the course, as all professors were beginners in Python. A
new mailing-list has also been created to gather the trainees together
after the event.

If you are interested by the PAT and you want to know more about past and
future events, you may write to the dedicated mailing-list
python-african-tour-discussion at lists.openplans.org, or take a look at the
website http://www.coactivate.org/projects/python-african-tour.

Thanks a lot for your patience if you have read so far!

Cheers,

Emmanuelle


From perfreem at gmail.com  Thu Jul 23 15:44:43 2009
From: perfreem at gmail.com (per freem)
Date: Thu, 23 Jul 2009 15:44:43 -0400
Subject: [SciPy-user] pdf for multivariate normal-logistic function?
Message-ID: <e95b09750907231244l71770f29mb66b8636c8a56bf8@mail.gmail.com>

hello all,

does anyone have a python function for evaluating the pdf of the
multivariate logistic-normal function? this would be greatly appreciated.
thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090723/9b93d521/attachment.html>

From atmyers at berkeley.edu  Fri Jul 24 00:10:41 2009
From: atmyers at berkeley.edu (atmyers at berkeley.edu)
Date: Thu, 23 Jul 2009 21:10:41 -0700 (PDT)
Subject: [SciPy-user] 64-Bit Scipy on OS X
Message-ID: <15903.169.229.69.169.1248408641.squirrel@calmail.berkeley.edu>

Hello All,

I have successfully built numpy 1.3.0 with a 64-bit installation of Python
2.6. I am trying to do the same with scipy-0.7.1, but I seem to be having
the same problem as Dan Yamis <a
href="http://www.nabble.com/building-64bit-scipy-ts23118382.html#a23118382">
here</a> - the "x86_64" flag gets passed to the c compiler just fine, but
NOT to gfortran. Consequently, the modules that don't use fortran work
just fine, but the ones that do fail to import.

The solution that worked for Dan:

LDFLAGS="-arch x86_64 -undefined dynamic_lookup -bundle" FFLAGS="-arch
x86_64 -O2 -Wall -fPIC" python setup.py install

does not work for me. The output of python setup.py install still has
things like

gfortran:f77:
build/src.macosx-10.5-universal-2.6/scipy/stats/mvn-f2pywrappers.f
/usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle
build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/scipy/stats/mvnmodule.o
build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/fortranobject.o
build/temp.macosx-10.5-universal-2.6/scipy/stats/mvndst.o
build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/scipy/stats/mvn-f2pywrappers.o
-L/usr/local/lib/gcc/i386-apple-darwin9.0.0/4.3.0
-Lbuild/temp.macosx-10.5-universal-2.6 -lgfortran -o
build/lib.macosx-10.5-universal-2.6/scipy/stats/mvn.so

so the flag still isn't getting passed. I don't really have any experience
with this kind of thing, so I'd appreciate any help you all could offer.

Thanks,
Andrew Myers





From tritemio at gmail.com  Fri Jul 24 05:26:29 2009
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Fri, 24 Jul 2009 11:26:29 +0200
Subject: [SciPy-user] Building 64bit scipy with ATLAS/LAPACK/UMFPACK in
	Fedora core 8
Message-ID: <5486cca80907240226y56a55508rb7ed64c1782578b4@mail.gmail.com>

Hi to the list,

I have fedora core 8 64bit in an 8 core server. I've installed the
latest numpy/scipy from sources but using Sfepy[1] I found that the
tests run very slowly.

Here it is the procedure I used to install numpy and scipy. Firstly I installed:

    lapack.x86_64
    lapack-devel.x86_64
    atlas.x86_64
    atlas-devel.x86_64
    suitesparse.x86_64
    suitesparse-devel.x86_64

then I downloaded numpy 1.3.0, edited site.cfg adding the following lines:

    [umfpack]
    library_dirs=/usr/lib64
    include_dirs = /usr/include/suitesparse

and built it with

    python setup.py build
    python setup.py install --prefix=/my/dir

After that I built scipy in exactly the same way (including site.cfg editing).

For umfpack I installed the scikits using "python setup.py install
--prefix=/my/dir"

I can I check if this scipy is correctly using the fast
atlas/lapack/umfpack libraries?

Are there any pitfalls on fedora core 8 regarding the numerical
libraries? I've read that some libraries are broken...

Any help is appreciated.

    ~ Antonio


[1] http://code.google.com/p/sfepy/


From david at ar.media.kyoto-u.ac.jp  Fri Jul 24 05:15:28 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 24 Jul 2009 18:15:28 +0900
Subject: [SciPy-user] Building 64bit scipy with ATLAS/LAPACK/UMFPACK in
 Fedora core 8
In-Reply-To: <5486cca80907240226y56a55508rb7ed64c1782578b4@mail.gmail.com>
References: <5486cca80907240226y56a55508rb7ed64c1782578b4@mail.gmail.com>
Message-ID: <4A697BB0.9070706@ar.media.kyoto-u.ac.jp>

Antonino Ingargiola wrote:
> Hi to the list,
>
> I have fedora core 8 64bit in an 8 core server. I've installed the
> latest numpy/scipy from sources but using Sfepy[1] I found that the
> tests run very slowly.
>
> Here it is the procedure I used to install numpy and scipy. Firstly I installed:
>
>     lapack.x86_64
>     lapack-devel.x86_64
>     atlas.x86_64
>     atlas-devel.x86_64
>     suitesparse.x86_64
>     suitesparse-devel.x86_64
>
> then I downloaded numpy 1.3.0, edited site.cfg adding the following lines:
>
>     [umfpack]
>     library_dirs=/usr/lib64
>     include_dirs = /usr/include/suitesparse
>
> and built it with
>
>     python setup.py build
>     python setup.py install --prefix=/my/dir
>
> After that I built scipy in exactly the same way (including site.cfg editing).
>
> For umfpack I installed the scikits using "python setup.py install
> --prefix=/my/dir"
>
> I can I check if this scipy is correctly using the fast
> atlas/lapack/umfpack libraries?
>   

You can check as follows:
    >>> import scipy
    >>> scipy.show_config()
    >>> import scipy.linalg.atlas_version

To check the speed, you can just use numpy.dot. Basic blas vs. optimized
ATLAS can be one order of magnitude different for large matrices (say
1000x1000, for example).

> Are there any pitfalls on fedora core 8 regarding the numerical
> libraries? I've read that some libraries are broken...
>   

Generally, problems are with the packaged versions - ATLAS in particular
is very hard to package correctly. If you build them by yourself
correctly, there should be no problem. Just make sure you use the last
released ATLAS (3.8.3 I believe at this time), and avoid LAPACK 3.2.0
(i.e. use 3.1.1).

cheers,

David


From tritemio at gmail.com  Fri Jul 24 06:09:45 2009
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Fri, 24 Jul 2009 12:09:45 +0200
Subject: [SciPy-user] Building 64bit scipy with ATLAS/LAPACK/UMFPACK in
	Fedora core 8
In-Reply-To: <4A697BB0.9070706@ar.media.kyoto-u.ac.jp>
References: <5486cca80907240226y56a55508rb7ed64c1782578b4@mail.gmail.com>
	<4A697BB0.9070706@ar.media.kyoto-u.ac.jp>
Message-ID: <5486cca80907240309m3cf3833em1694f6fbcd226774@mail.gmail.com>

Hi David,

2009/7/24 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
[cut]
>> I can I check if this scipy is correctly using the fast
>> atlas/lapack/umfpack libraries?
>>
>
> You can check as follows:
> ? ?>>> import scipy
> ? ?>>> scipy.show_config()
> ? ?>>> import scipy.linalg.atlas_version
>
> To check the speed, you can just use numpy.dot. Basic blas vs. optimized
> ATLAS can be one order of magnitude different for large matrices (say
> 1000x1000, for example).

I get this:

    import numpy, scipy
    a=numpy.random.rand(1000,1000)
    b=numpy.random.rand(1000,1000)
    time c = numpy.dot(a,b)
    CPU times: user 0.55 s, sys: 0.01 s, total: 0.56 s
    Wall time: 0.56 s

so it seems is using the fast lapack? (On an Ubuntu machine in which
sfepy is very fast I get the same in 0.6 s)

Furthermore scipy.show_config() gives:

    amd_info:
        libraries = ['amd']
        library_dirs = ['/usr/lib64']

    umfpack_info:
        libraries = ['umfpack', 'amd']
        library_dirs = ['/usr/lib64']
        define_macros = [('SCIPY_UMFPACK_H', None)]
        swig_opts = ['-I/usr/include/suitesparse']
        include_dirs = ['/usr/include/suitesparse']

    atlas_threads_info:
      NOT AVAILABLE

    blas_opt_info:
        libraries = ['f77blas', 'cblas', 'atlas']
        library_dirs = ['/usr/lib64/atlas']
        define_macros = [('ATLAS_INFO', '"\\"3.6.0\\""')]
        language = c
        include_dirs = ['/usr/include/atlas']

    atlas_blas_threads_info:
      NOT AVAILABLE

    lapack_opt_info:
        libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
        library_dirs = ['/usr/lib64/atlas']
        define_macros = [('ATLAS_INFO', '"\\"3.6.0\\""')]
        language = f77
        include_dirs = ['/usr/include/atlas']

    atlas_info:
        libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
        library_dirs = ['/usr/lib64/atlas']
        language = f77
        include_dirs = ['/usr/include/atlas']

    lapack_mkl_info:
      NOT AVAILABLE

    blas_mkl_info:
      NOT AVAILABLE

    atlas_blas_info:
        libraries = ['f77blas', 'cblas', 'atlas']
        library_dirs = ['/usr/lib64/atlas']
        language = c
        include_dirs = ['/usr/include/atlas']

    mkl_info:
      NOT AVAILABLE

is it ok?

>> Are there any pitfalls on fedora core 8 regarding the numerical
>> libraries? I've read that some libraries are broken...
>>
>
> Generally, problems are with the packaged versions - ATLAS in particular
> is very hard to package correctly. If you build them by yourself
> correctly, there should be no problem. Just make sure you use the last
> released ATLAS (3.8.3 I believe at this time), and avoid LAPACK 3.2.0
> (i.e. use 3.1.1).

Given the above results the packages version of ATLAS/LAPACK should work fine.

How about testing if UMFPACK is used... maybe I should test a dot
product between sparse matrices but I don't know how to do it.

> cheers,
>
> David

Thanks,

   Antonio


From stefan at sun.ac.za  Fri Jul 24 09:32:55 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 24 Jul 2009 15:32:55 +0200
Subject: [SciPy-user] Numpy/Scipy and the Python African Tour
In-Reply-To: <20090723141737.GF10321@phare.normalesup.org>
References: <20090723141737.GF10321@phare.normalesup.org>
Message-ID: <9457e7c80907240632t3ba83b3avc4d82ef88628a20f@mail.gmail.com>

2009/7/23 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
> Whereas we reused the slides of the Moroccan event on the introduction to
> Python, I wrote the slides for the 3-4 hour course about Scientific
> Python. The slides were written in French, English being an issue for a
> part of the students. They can be found on
> http://www.dakarlug.org/pat/scientifique/html/ for French-reading people
> (my apologies to the others! I'm considering translating the slides to
> English but I will do it more rapidly if I'm given the incentive to do
> so, so tell me if you may be interested by an English version!).

That's a fantastic set of notes!  I wonder if a person can't find a
volunteer to do the translation, saving you some time.

> Thanks a lot for your patience if you have read so far!

Thank you for an interesting and inspiring post.

Regards
St?fan


From ce at vejnar.eu  Fri Jul 24 11:47:32 2009
From: ce at vejnar.eu (Charles Vejnar)
Date: Fri, 24 Jul 2009 17:47:32 +0200
Subject: [SciPy-user] Selection of dominant points
Message-ID: <200907241747.33691.ce@vejnar.eu>

Hi,

I have a curve with millions of points which is non-standard (i.e. it's not a 
gaussian or ...).

I would like to select about 500 points which best describe the curve 
(something like selecting dominant points).

Do you know any existing implementation ?

Thank you.

Charles



From dyamins at gmail.com  Fri Jul 24 16:57:17 2009
From: dyamins at gmail.com (Dan Yamins)
Date: Fri, 24 Jul 2009 16:57:17 -0400
Subject: [SciPy-User] [SciPy-user] 64-Bit Scipy on OS X
In-Reply-To: <15903.169.229.69.169.1248408641.squirrel@calmail.berkeley.edu>
References: <15903.169.229.69.169.1248408641.squirrel@calmail.berkeley.edu>
Message-ID: <15e4667e0907241357u5e74b18cn4f02d3a4c2e7b333@mail.gmail.com>

> The solution that worked for Dan:
>
> LDFLAGS="-arch x86_64 -undefined dynamic_lookup -bundle" FFLAGS="-arch
> x86_64 -O2 -Wall -fPIC" python setup.py install
>
> does not work for me. The output of python setup.py install still has
> things like
>
> gfortran:f77:
> build/src.macosx-10.5-universal-2.6/scipy/stats/mvn-f2pywrappers.f
> /usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle
>
> build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/scipy/stats/mvnmodule.o
>
> build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/fortranobject.o
> build/temp.macosx-10.5-universal-2.6/scipy/stats/mvndst.o
>
> build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/scipy/stats/mvn-f2pywrappers.o
> -L/usr/local/lib/gcc/i386-apple-darwin9.0.0/4.3.0
> -Lbuild/temp.macosx-10.5-universal-2.6 -lgfortran -o
> build/lib.macosx-10.5-universal-2.6/scipy/stats/mvn.so
>

I'm not an expert at doing this sort of thing either -- I think David
Cornapeau and Robert Kern are really the experts there -- but I would like
to ask if you have any more specific error message output during the
compile?  Are there any lines with a message like "not of proper
architecture" or something similar?   If you actually are able, could you
shunt all the output to a file, and attach it?  Maybe I could try to
diagnose the problem.  (Quite a few other things came up that had to be
solved _before_ the passing of the flags at the command prompt worked as a
solution.)

Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090724/55af81a3/attachment.html>

From atmyers at berkeley.edu  Fri Jul 24 18:04:12 2009
From: atmyers at berkeley.edu (atmyers at berkeley.edu)
Date: Fri, 24 Jul 2009 15:04:12 -0700 (PDT)
Subject: [SciPy-User] [SciPy-user] 64-Bit Scipy on OS X
In-Reply-To: <15e4667e0907241357u5e74b18cn4f02d3a4c2e7b333@mail.gmail.com>
References: <15903.169.229.69.169.1248408641.squirrel@calmail.berkeley.edu>
	<15e4667e0907241357u5e74b18cn4f02d3a4c2e7b333@mail.gmail.com>
Message-ID: <60169.136.152.134.151.1248473052.squirrel@calmail.berkeley.edu>

Hi Dan,

Thanks for responding. I actually just solved the problem on my own: as
David Cournapeau tried to tell me in your thread, sudo affects environment
variables, so I wasn't actually passing the flag like I thought I was.
After that I needed to upgrade my gfortran to a universal capable version,
and the install went fine. Thanks for the help, and sorry for spamming the
list.

~Andrew

>> The solution that worked for Dan:
>>
>> LDFLAGS="-arch x86_64 -undefined dynamic_lookup -bundle" FFLAGS="-arch
>> x86_64 -O2 -Wall -fPIC" python setup.py install
>>
>> does not work for me. The output of python setup.py install still has
>> things like
>>
>> gfortran:f77:
>> build/src.macosx-10.5-universal-2.6/scipy/stats/mvn-f2pywrappers.f
>> /usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle
>>
>> build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/scipy/stats/mvnmodule.o
>>
>> build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/fortranobject.o
>> build/temp.macosx-10.5-universal-2.6/scipy/stats/mvndst.o
>>
>> build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/scipy/stats/mvn-f2pywrappers.o
>> -L/usr/local/lib/gcc/i386-apple-darwin9.0.0/4.3.0
>> -Lbuild/temp.macosx-10.5-universal-2.6 -lgfortran -o
>> build/lib.macosx-10.5-universal-2.6/scipy/stats/mvn.so
>>
>
> I'm not an expert at doing this sort of thing either -- I think David
> Cornapeau and Robert Kern are really the experts there -- but I would like
> to ask if you have any more specific error message output during the
> compile?  Are there any lines with a message like "not of proper
> architecture" or something similar?   If you actually are able, could you
> shunt all the output to a file, and attach it?  Maybe I could try to
> diagnose the problem.  (Quite a few other things came up that had to be
> solved _before_ the passing of the flags at the command prompt worked as a
> solution.)
>
> Dan
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>




From dwf at cs.toronto.edu  Sat Jul 25 00:45:09 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sat, 25 Jul 2009 00:45:09 -0400
Subject: [SciPy-User] [SciPy-user] Selection of dominant points
In-Reply-To: <200907241747.33691.ce@vejnar.eu>
References: <200907241747.33691.ce@vejnar.eu>
Message-ID: <AAEFA82E-C1B5-4E63-9709-080C6483704E@cs.toronto.edu>

On 24-Jul-09, at 11:47 AM, Charles Vejnar wrote:

> Hi,
>
> I have a curve with millions of points which is non-standard (i.e.  
> it's not a
> gaussian or ...).
>
> I would like to select about 500 points which best describe the curve
> (something like selecting dominant points).

It would be helpful to know more about the problem.  Like:
- are the points evenly spaced along the independent axis?
- are they assumed to be samples from a continuous signal?
- does the signal have any known structure at all? If you could  
describe where these data come from it might help nail down what  
methods would be appropriate.

Regards,

David 


From jkington at wisc.edu  Sat Jul 25 03:02:33 2009
From: jkington at wisc.edu (Joe Kington)
Date: Sat, 25 Jul 2009 02:02:33 -0500
Subject: [SciPy-User] [SciPy-user] Selection of dominant points
In-Reply-To: <AAEFA82E-C1B5-4E63-9709-080C6483704E@cs.toronto.edu>
References: <200907241747.33691.ce@vejnar.eu>
	<AAEFA82E-C1B5-4E63-9709-080C6483704E@cs.toronto.edu>
Message-ID: <f6951adf0907250002l1273874y5cc8cdb050d0c651@mail.gmail.com>

Well, I thought shapely <http://trac.gispython.org/lab/wiki/Shapely> would
have some sort of line-generalization capabilities, but apparently it
doesn't.

You may already be aware of this, but algorithm wise, you're looking for the
Douglas-Peucker
algorithm<http://en.wikipedia.org/wiki/Ramer-Douglas-Peucker_algorithm>,
or something similar.

If you don't want to mess with implementing it yourself, you might want to
look at grass <http://grass.itc.it/>.  Grass has a
generalize<http://grass.osgeo.org/wiki/V.generalize_tutorial>function
that should do what you need, and does have at least some python
bindings.  I've never actually used the python bindings for
grass<http://download.osgeo.org/grass/grass6_progman/pythonlib.html>,
so I can't help you too much there...

Hope that helps, at any rate!
-Joe

On Fri, Jul 24, 2009 at 11:45 PM, David Warde-Farley <dwf at cs.toronto.edu>wrote:

> On 24-Jul-09, at 11:47 AM, Charles Vejnar wrote:
>
> > Hi,
> >
> > I have a curve with millions of points which is non-standard (i.e.
> > it's not a
> > gaussian or ...).
> >
> > I would like to select about 500 points which best describe the curve
> > (something like selecting dominant points).
>
> It would be helpful to know more about the problem.  Like:
> - are the points evenly spaced along the independent axis?
> - are they assumed to be samples from a continuous signal?
> - does the signal have any known structure at all? If you could
> describe where these data come from it might help nail down what
> methods would be appropriate.
>
> Regards,
>
> David
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090725/c18c66de/attachment.html>

From stef.mientki at gmail.com  Sat Jul 25 17:13:23 2009
From: stef.mientki at gmail.com (Stef Mientki)
Date: Sat, 25 Jul 2009 23:13:23 +0200
Subject: [SciPy-User] encryption of numpy array (wav file) ?
Message-ID: <4A6B7573.5000501@gmail.com>

hello,

I need to encrypt a wav file (conversation between doctor and his patient).
Now it's easy to convert a wav file ( or maybe even the sound stream 
itself) into a numpy array.
Everything runs on a slow machine (netbook),
so I hope numpy can encrypt on the flight.
I don't need very secure encryption, just good enough to protect the 
patient information from accidental listening by others.

Any idea of a relative simple (and fast) encryption method with numpy 
arrays ?

thanks,
Stef Mientki


From robert.kern at gmail.com  Sat Jul 25 17:17:17 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 25 Jul 2009 16:17:17 -0500
Subject: [SciPy-User] encryption of numpy array (wav file) ?
In-Reply-To: <4A6B7573.5000501@gmail.com>
References: <4A6B7573.5000501@gmail.com>
Message-ID: <3d375d730907251417ia47c953h8a99a75cd4be6061@mail.gmail.com>

On Sat, Jul 25, 2009 at 16:13, Stef Mientki<stef.mientki at gmail.com> wrote:
> hello,
>
> I need to encrypt a wav file (conversation between doctor and his patient).
> Now it's easy to convert a wav file ( or maybe even the sound stream
> itself) into a numpy array.
> Everything runs on a slow machine (netbook),
> so I hope numpy can encrypt on the flight.
> I don't need very secure encryption, just good enough to protect the
> patient information from accidental listening by others.
>
> Any idea of a relative simple (and fast) encryption method with numpy
> arrays ?

Not on numpy arrays, no. Just use the normal crypto tools on strings.
Preferably, use a complete tool like GPG (for files) or SSL (for
network connections) than just using a library like PyCrypto.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From david_baddeley at yahoo.com.au  Sun Jul 26 20:16:26 2009
From: david_baddeley at yahoo.com.au (David Baddeley)
Date: Sun, 26 Jul 2009 17:16:26 -0700 (PDT)
Subject: [SciPy-User] encryption of numpy array (wav file) ?
In-Reply-To: <3d375d730907251417ia47c953h8a99a75cd4be6061@mail.gmail.com>
References: <4A6B7573.5000501@gmail.com>
	<3d375d730907251417ia47c953h8a99a75cd4be6061@mail.gmail.com>
Message-ID: <759002.48107.qm@web33008.mail.mud.yahoo.com>


If you're only looking to protect the data from accidental listening you could always just bitwise xor it with a (random) key:

encrypted = data ^ key

decrypted = encrypted ^ key

if you had a reasonable key (which you kept safe), and potentially cast your data to some type with loads of precision (eg int64 or float128) before doing the xor's it might not be too insecure either.

David





----- Original Message ----
From: Robert Kern <robert.kern at gmail.com>
To: SciPy Users List <scipy-user at scipy.org>
Sent: Sunday, 26 July, 2009 9:17:17 AM
Subject: Re: [SciPy-User] encryption of numpy array (wav file) ?

On Sat, Jul 25, 2009 at 16:13, Stef Mientki<stef.mientki at gmail.com> wrote:
> hello,
>
> I need to encrypt a wav file (conversation between doctor and his patient).
> Now it's easy to convert a wav file ( or maybe even the sound stream
> itself) into a numpy array.
> Everything runs on a slow machine (netbook),
> so I hope numpy can encrypt on the flight.
> I don't need very secure encryption, just good enough to protect the
> patient information from accidental listening by others.
>
> Any idea of a relative simple (and fast) encryption method with numpy
> arrays ?

Not on numpy arrays, no. Just use the normal crypto tools on strings.
Preferably, use a complete tool like GPG (for files) or SSL (for
network connections) than just using a library like PyCrypto.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
_______________________________________________
SciPy-User mailing list
SciPy-User at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user



      


From robert.kern at gmail.com  Sun Jul 26 21:06:19 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 26 Jul 2009 20:06:19 -0500
Subject: [SciPy-User] encryption of numpy array (wav file) ?
In-Reply-To: <759002.48107.qm@web33008.mail.mud.yahoo.com>
References: <4A6B7573.5000501@gmail.com>
	<3d375d730907251417ia47c953h8a99a75cd4be6061@mail.gmail.com> 
	<759002.48107.qm@web33008.mail.mud.yahoo.com>
Message-ID: <3d375d730907261806g60578809q9be777808beb1b5@mail.gmail.com>

On Sun, Jul 26, 2009 at 19:16, David
Baddeley<david_baddeley at yahoo.com.au> wrote:
>
> If you're only looking to protect the data from accidental listening you could always just bitwise xor it with a (random) key:
>
> encrypted = data ^ key
>
> decrypted = encrypted ^ key
>
> if you had a reasonable key (which you kept safe), and potentially cast your data to some type with loads of precision (eg int64 or float128) before doing the xor's it might not be too insecure either.

Upcasting the data will make lots of predictable zeros and thus
exposing much of your key.

If you have any concerns about privacy, it is much, much easier to use
real cryptography written by security professionals than to devise
some scheme yourself. This is even more important when you are
protecting someone else's confidential medical information. If it's
important enough to do at all, it's important enough to do it right.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From massimiliano.cannata at gmail.com  Mon Jul 27 07:09:56 2009
From: massimiliano.cannata at gmail.com (massimiliano cannata)
Date: Mon, 27 Jul 2009 13:09:56 +0200
Subject: [SciPy-User] timeseries manipulate
Message-ID: <df5c57470907270409t21716c19x41b079fde6c43e0c@mail.gmail.com>

Hi folk,
I'm quite new of python, numpy and timeseries.
I have the following problem, and maybe you can help me:
- I have files with irregular time series data (at different seconds)
- I want 10m regular time series

Looking around I sow that I can get 10min freq by masking a minutely
timeseries, but how to get there?

should I manipulate arrays on my own (loop into dates array and values array
and extrapolate a new dates10min and vals10min) or there is a cheaper and
faster approach I should follow?

Up to now I figure out I could:
1. read the txt file
2. create a time series at "second" frequency
3. convert to "minute" freq with convert('MINUTE',func=ma.sum)

and then?..... I should somehow make a sum of all the values within every
minute and derive a minute freq timeseries with values only every 10min...

I hope you can understand what I mean.. :-)

Thanks,
Maxi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090727/8262b9fc/attachment.html>

From Dharhas.Pothina at twdb.state.tx.us  Mon Jul 27 11:40:27 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Mon, 27 Jul 2009 10:40:27 -0500
Subject: [SciPy-User] timeseries manipulate
In-Reply-To: <df5c57470907270409t21716c19x41b079fde6c43e0c@mail.gmail.com>
References: <df5c57470907270409t21716c19x41b079fde6c43e0c@mail.gmail.com>
Message-ID: <4A6D841B.63BA.009B.0@twdb.state.tx.us>

Hi,

>From what I understand there is no direct way to do this in the timeseries package but if you search the list archives for 'pytseries custom frequencies' there is a post describing a workaround to do what you are trying to do.

- dharjhas

>>> massimiliano cannata <massimiliano.cannata at gmail.com> 7/27/2009 6:09 AM >>>
Hi folk,
I'm quite new of python, numpy and timeseries.
I have the following problem, and maybe you can help me:
- I have files with irregular time series data (at different seconds)
- I want 10m regular time series

Looking around I sow that I can get 10min freq by masking a minutely
timeseries, but how to get there?

should I manipulate arrays on my own (loop into dates array and values array
and extrapolate a new dates10min and vals10min) or there is a cheaper and
faster approach I should follow?

Up to now I figure out I could:
1. read the txt file
2. create a time series at "second" frequency
3. convert to "minute" freq with convert('MINUTE',func=ma.sum)

and then?..... I should somehow make a sum of all the values within every
minute and derive a minute freq timeseries with values only every 10min...

I hope you can understand what I mean.. :-)

Thanks,
Maxi



From daniel.farrell at imperial.ac.uk  Mon Jul 27 03:33:37 2009
From: daniel.farrell at imperial.ac.uk (Daniel J Farrell)
Date: Mon, 27 Jul 2009 08:33:37 +0100
Subject: [SciPy-User] Scipy and statistics: probability density function
Message-ID: <C1C4F27B-636C-4A00-A169-A50440DEF3E4@imperial.ac.uk>

Dear list,

I am looking for some of the functionality provided by the GNU  
Scientific Library histogram module (http://www.gnu.org/software/gsl/manual/html_node/Histograms.html 
).

In particular, a need to be able to create a probability density  
function from my histogram of data. This will allow inverse look-ups  
to be performed, i.e. for a random number (0-->1) find the associated  
probability (http://www.gnu.org/software/gsl/manual/html_node/The-histogram-probability-distribution-struct.html 
). This allows the distribution and for samples to be returned  
weighted by the probability of the distribution -- which is a common  
task!

Is is easier just to call GSL using C-types? I just wanted to check  
that this functionality doesn't already exist before I stray too far,  
or implement it my self.

Best regards

Dan


From robert.kern at gmail.com  Mon Jul 27 11:59:36 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 27 Jul 2009 10:59:36 -0500
Subject: [SciPy-User] Scipy and statistics: probability density function
In-Reply-To: <C1C4F27B-636C-4A00-A169-A50440DEF3E4@imperial.ac.uk>
References: <C1C4F27B-636C-4A00-A169-A50440DEF3E4@imperial.ac.uk>
Message-ID: <3d375d730907270859w42327684gb726549eea094147@mail.gmail.com>

On Mon, Jul 27, 2009 at 02:33, Daniel J
Farrell<daniel.farrell at imperial.ac.uk> wrote:
> Dear list,
>
> I am looking for some of the functionality provided by the GNU
> Scientific Library histogram module (http://www.gnu.org/software/gsl/manual/html_node/Histograms.html
> ).
>
> In particular, a need to be able to create a probability density
> function from my histogram of data. This will allow inverse look-ups
> to be performed, i.e. for a random number (0-->1) find the associated
> probability (http://www.gnu.org/software/gsl/manual/html_node/The-histogram-probability-distribution-struct.html
> ). This allows the distribution and for samples to be returned
> weighted by the probability of the distribution -- which is a common
> task!

It looks like you want a CDF (or rather it's inverse, the PPF) rather
than a PDF. Anyways, this is straightforward. Compute the histogram
using normed=False. Find the cumulative sum and divide by the sum to
get the (smoothed) empirical CDF. Prepend a 0.0 to this, and then this
will align with the edges array that is also returned. Then you can
use linear interpolation to do lookups. If you use the edges array as
"X" and the empirical CDF as "Y", then this is a CDF. If you use the
empircal CDF array as "X" and the edges as "Y", then this is a PPF.


In [12]: x = np.random.uniform(0, 10, size=1000)

In [13]: hist, edges = np.histogram(x)

In [15]: hist
Out[15]: array([100, 101, 104, 108,  80, 111,  96,  88, 108, 104])

In [16]: edges
Out[16]:
array([  3.53879571e-04,   1.00026423e+00,   2.00017458e+00,
         3.00008493e+00,   3.99999528e+00,   4.99990563e+00,
         5.99981598e+00,   6.99972633e+00,   7.99963668e+00,
         8.99954704e+00,   9.99945739e+00])

In [18]: ecdf = np.hstack([0.0, hist.cumsum() / float(hist.sum())])

In [19]: ecdf
Out[19]:
array([ 0.   ,  0.1  ,  0.201,  0.305,  0.413,  0.493,  0.604,  0.7  ,
        0.788,  0.896,  1.   ])

In [20]: np.interp(np.linspace(0, 10), edges, ecdf)
Out[20]:
array([ 0.        ,  0.0203746 ,  0.04078459,  0.06119459,  0.08160458,
        0.10203472,  0.12264881,  0.14326291,  0.163877  ,  0.18449109,
        0.20522712,  0.22645351,  0.24767991,  0.2689063 ,  0.29013269,
        0.31160366,  0.33364646,  0.35568925,  0.37773204,  0.39977483,
        0.41953158,  0.43585957,  0.45218756,  0.46851556,  0.48484355,
        0.50433802,  0.52699311,  0.54964821,  0.5723033 ,  0.59495839,
        0.61577382,  0.63536742,  0.65496101,  0.6745546 ,  0.6941482 ,
        0.71259664,  0.73055743,  0.74851823,  0.76647902,  0.78443982,
        0.80567348,  0.82771627,  0.84975906,  0.87180185,  0.89384465,
        0.91515087,  0.93637726,  0.95760365,  0.97883004,  1.        ])

In [21]: np.interp(np.linspace(0, 1), ecdf, edges)
Out[21]:
array([  3.53879571e-04,   2.04417216e-01,   4.08480553e-01,
         6.12543890e-01,   8.16607227e-01,   1.02046852e+00,
         1.22251143e+00,   1.42455434e+00,   1.62659724e+00,
         1.82864015e+00,   2.02980301e+00,   2.22601775e+00,
         2.42223250e+00,   2.61844725e+00,   2.81466200e+00,
         3.01047705e+00,   3.19942458e+00,   3.38837211e+00,
         3.57731965e+00,   3.76626718e+00,   3.95521472e+00,
         4.19462069e+00,   4.44969986e+00,   4.70477903e+00,
         4.95985820e+00,   5.15488346e+00,   5.33872431e+00,
         5.52256515e+00,   5.70640600e+00,   5.89024684e+00,
         6.08569264e+00,   6.29825861e+00,   6.51082459e+00,
         6.72339057e+00,   6.93595654e+00,   7.16204944e+00,
         7.39393960e+00,   7.62582975e+00,   7.85771991e+00,
         8.07294833e+00,   8.26189586e+00,   8.45084340e+00,
         8.63979093e+00,   8.82873846e+00,   9.01838365e+00,
         9.21459840e+00,   9.41081315e+00,   9.60702789e+00,
         9.80324264e+00,   9.99945739e+00])


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ce at vejnar.eu  Mon Jul 27 12:20:29 2009
From: ce at vejnar.eu (Charles Vejnar)
Date: Mon, 27 Jul 2009 18:20:29 +0200
Subject: [SciPy-User] [SciPy-user] Selection of dominant points
In-Reply-To: <f6951adf0907250002l1273874y5cc8cdb050d0c651@mail.gmail.com>
References: <200907241747.33691.ce@vejnar.eu>
	<AAEFA82E-C1B5-4E63-9709-080C6483704E@cs.toronto.edu>
	<f6951adf0907250002l1273874y5cc8cdb050d0c651@mail.gmail.com>
Message-ID: <200907271820.30118.ce@vejnar.eu>

Hi,

Thank you. I wasn't looking in the right direction.

After intensive googling, I found Python implementations of Ramer-Douglas-
Peucker algorithm:
http://mappinghacks.com/2008/05/05/douglas-peucker-line-simplification-in-
python/
It was derived from this one:
http://mappinghacks.com/code/PolyLineReduction/
I also found one here:
http://cvs.linuxcnc.org/cvs/emc2/lib/python/rs274/author.py?rev=1.1

This algorithm is working perfectly on my data. Thanks again.

Charles

On Saturday 25 July 2009, Joe Kington wrote:
> Well, I thought shapely <http://trac.gispython.org/lab/wiki/Shapely> would
> have some sort of line-generalization capabilities, but apparently it
> doesn't.
>
> You may already be aware of this, but algorithm wise, you're looking for
> the Douglas-Peucker
> algorithm<http://en.wikipedia.org/wiki/Ramer-Douglas-Peucker_algorithm>,
> or something similar.
>
> If you don't want to mess with implementing it yourself, you might want to
> look at grass <http://grass.itc.it/>.  Grass has a
> generalize<http://grass.osgeo.org/wiki/V.generalize_tutorial>function
> that should do what you need, and does have at least some python
> bindings.  I've never actually used the python bindings for
> grass<http://download.osgeo.org/grass/grass6_progman/pythonlib.html>,
> so I can't help you too much there...
>
> Hope that helps, at any rate!
> -Joe
>
> On Fri, Jul 24, 2009 at 11:45 PM, David Warde-Farley 
<dwf at cs.toronto.edu>wrote:
> > On 24-Jul-09, at 11:47 AM, Charles Vejnar wrote:
> > > Hi,
> > >
> > > I have a curve with millions of points which is non-standard (i.e.
> > > it's not a
> > > gaussian or ...).
> > >
> > > I would like to select about 500 points which best describe the curve
> > > (something like selecting dominant points).
> >
> > It would be helpful to know more about the problem.  Like:
> > - are the points evenly spaced along the independent axis?
> > - are they assumed to be samples from a continuous signal?
> > - does the signal have any known structure at all? If you could
> > describe where these data come from it might help nail down what
> > methods would be appropriate.
> >
> > Regards,
> >
> > David
> > _______________________________________________
> > SciPy-User mailing list
> > SciPy-User at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user


From permafacture at gmail.com  Mon Jul 27 12:34:37 2009
From: permafacture at gmail.com (Elliot Hallmark)
Date: Mon, 27 Jul 2009 11:34:37 -0500
Subject: [SciPy-User] optical ray tracing
Message-ID: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>

howdy,

I'm looking to do some raytracing for a nonimaging design and there
are no opensource packages that address the geometry needed.  Scipy
seems like a good package to become familiar with anyway (I miss
mathematica now that i'm no longer a university student), but i was
wondering if anyone has already done some optical ray tracing work in
scipy that i could build off of.

Wouldn't be to hard to do from scratch, but my focus is on the design
and the fewer excursions into building new tools the better.

thanks,

elliot


From daniel.farrell at imperial.ac.uk  Mon Jul 27 12:40:50 2009
From: daniel.farrell at imperial.ac.uk (Daniel J Farrell)
Date: Mon, 27 Jul 2009 17:40:50 +0100
Subject: [SciPy-User] optical ray tracing
In-Reply-To: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
References: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
Message-ID: <33B90ACC-16FB-4E22-B0B0-EE84D6218C4F@imperial.ac.uk>

Dear Elliot,

I am doing some optical ray tracing and would be happy to share my  
code with you. I started yesterday believe it or not! What is your  
application? I'm working on photovoltaics.

So far I have written the ray tracing code for plane and axis aligned  
box intersection. Today I have added materials properties such as  
refractive index and absorption emission coefficients, but this bit is  
far from complete.

Does that sound like something you want to work with me on? It would  
be great if we have a similar common goal. If so I could dump  
something on Google code.

Cheers,

Dan

On 27 Jul 2009, at 17:34, Elliot Hallmark wrote:

> howdy,
>
> I'm looking to do some raytracing for a nonimaging design and there
> are no opensource packages that address the geometry needed.  Scipy
> seems like a good package to become familiar with anyway (I miss
> mathematica now that i'm no longer a university student), but i was
> wondering if anyone has already done some optical ray tracing work in
> scipy that i could build off of.
>
> Wouldn't be to hard to do from scratch, but my focus is on the design
> and the fewer excursions into building new tools the better.
>
> thanks,
>
> elliot
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



From stefan at sun.ac.za  Mon Jul 27 12:41:18 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 27 Jul 2009 18:41:18 +0200
Subject: [SciPy-User] optical ray tracing
In-Reply-To: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
References: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
Message-ID: <9457e7c80907270941o8eacc38i57f7519febb5b14b@mail.gmail.com>

2009/7/27 Elliot Hallmark <permafacture at gmail.com>:
> howdy,
>
> I'm looking to do some raytracing for a nonimaging design and there
> are no opensource packages that address the geometry needed. ?Scipy
> seems like a good package to become familiar with anyway (I miss
> mathematica now that i'm no longer a university student), but i was
> wondering if anyone has already done some optical ray tracing work in
> scipy that i could build off of.

The only work I'm aware of is Matthieu's real time raytracer with
Python bindings:

http://matt.eifelle.com/2009/05/19/interactive-raytracer/

Regards
St?fan


From permafacture at gmail.com  Mon Jul 27 13:31:00 2009
From: permafacture at gmail.com (Elliot Hallmark)
Date: Mon, 27 Jul 2009 12:31:00 -0500
Subject: [SciPy-User] optical ray tracing
In-Reply-To: <9457e7c80907270941o8eacc38i57f7519febb5b14b@mail.gmail.com>
References: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
	<9457e7c80907270941o8eacc38i57f7519febb5b14b@mail.gmail.com>
Message-ID: <a07a0f1c0907271031m34f162afv52ed5f7905ff455f@mail.gmail.com>

Daniel,

I'm working on a non imaging concentrator for solar thermal.  This
involves parabolas translated off the optical axis and tilted with
their axii (?) crossing the optical axis.

Then, nonimaging optics uses the edge ray principal, So I would have
two rays divergent by some angle originating at the very edges of the
entry aperature.

Pretty simple, tilting the parabolas is the hardest part, followed by
determining when a ray hits a reflector.

I dont need refraction but its good to have it available.

I'm still looking into some other options but I'll let you know if i
go the scipy route.

--Stefan--

there lots of ray tracing programs like that out there, for rendering
images rather than designing optics.  Not useful for me.  But thanks.

-elliot


From matthieu.brucher at gmail.com  Mon Jul 27 13:43:44 2009
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Mon, 27 Jul 2009 19:43:44 +0200
Subject: [SciPy-User] optical ray tracing
In-Reply-To: <a07a0f1c0907271031m34f162afv52ed5f7905ff455f@mail.gmail.com>
References: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
	<9457e7c80907270941o8eacc38i57f7519febb5b14b@mail.gmail.com>
	<a07a0f1c0907271031m34f162afv52ed5f7905ff455f@mail.gmail.com>
Message-ID: <e76aa17f0907271043q59b114bdr9d3ae1586cd19aea@mail.gmail.com>

> --Stefan--
>
> there lots of ray tracing programs like that out there, for rendering
> images rather than designing optics. ?Not useful for me. ?But thanks.

Indeed, my raytracer is not adequate to this (besides, it's no where
near completion :()

Matthieu
-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From josephsmidt at gmail.com  Tue Jul 28 21:34:43 2009
From: josephsmidt at gmail.com (Joseph Smidt)
Date: Tue, 28 Jul 2009 18:34:43 -0700
Subject: [SciPy-User] Easy Way To Find Confidence Intervals For 2D Data?
Message-ID: <142682e10907281834w6bd3f7ebm7df2ddc0b28ecc1b@mail.gmail.com>

I have a function on a 2d grid that looks like a skewed mound. I would
like to calculate everything in the 68% (one sigma or standard
deviation) confidence interval, and 95% (two sigma) as seen done in
this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
 (The first line is 68% and second line is 95% confidence intervals.)

I would like to the set everything in the 68% confidence interval to
1, 95% to 2 and everything else to 0.  Thanks.

                   Joseph Smidt

-- 
------------------------------------------------------------------------
Joseph Smidt <josephsmidt at gmail.com>

Physics and Astronomy
4129 Frederick Reines Hall
Irvine, CA 92697-4575
Office: 949-824-3269


From robert.kern at gmail.com  Tue Jul 28 22:33:17 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 28 Jul 2009 21:33:17 -0500
Subject: [SciPy-User] Easy Way To Find Confidence Intervals For 2D Data?
In-Reply-To: <142682e10907281834w6bd3f7ebm7df2ddc0b28ecc1b@mail.gmail.com>
References: <142682e10907281834w6bd3f7ebm7df2ddc0b28ecc1b@mail.gmail.com>
Message-ID: <3d375d730907281933l40aa77d8p72d9172c55cedfe@mail.gmail.com>

On Tue, Jul 28, 2009 at 20:34, Joseph Smidt<josephsmidt at gmail.com> wrote:
> I have a function on a 2d grid that looks like a skewed mound. I would
> like to calculate everything in the 68% (one sigma or standard
> deviation) confidence interval, and 95% (two sigma) as seen done in
> this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
> ?(The first line is 68% and second line is 95% confidence intervals.)
>
> I would like to the set everything in the 68% confidence interval to
> 1, 95% to 2 and everything else to 0. ?Thanks.

It depends somewhat on what that function is. Is it a proper PDF?
Let's say it is.

I'll assume that the data has been normalized such that the sum of all
the entries is 1. If not you'll have to keep track of that
normalization constant. Let's call this array normed_pdf. What you are
going to do is make a function that takes a value x, computes the sum
normed_pdf[normed_pdf > x].sum(), and then subtracts the target value,
say 0.68. Then use a root finder from scipy.optimize to find out where
this function equals 0. Repeat for 0.95. Now just do a contour plot
using those two values as the contour lines; e.g. in matplotlib, use
the contour(Z, V) form.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From josephsmidt at gmail.com  Wed Jul 29 01:07:09 2009
From: josephsmidt at gmail.com (Joseph Smidt)
Date: Tue, 28 Jul 2009 22:07:09 -0700
Subject: [SciPy-User] Easy Way To Find Confidence Intervals For 2D Data?
In-Reply-To: <3d375d730907281933l40aa77d8p72d9172c55cedfe@mail.gmail.com>
References: <142682e10907281834w6bd3f7ebm7df2ddc0b28ecc1b@mail.gmail.com>
	<3d375d730907281933l40aa77d8p72d9172c55cedfe@mail.gmail.com>
Message-ID: <142682e10907282207n28ae8e36w4729637046e9f327@mail.gmail.com>

Thanks, that's what i needed!

On Tue, Jul 28, 2009 at 7:33 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Tue, Jul 28, 2009 at 20:34, Joseph Smidt<josephsmidt at gmail.com> wrote:
>> I have a function on a 2d grid that looks like a skewed mound. I would
>> like to calculate everything in the 68% (one sigma or standard
>> deviation) confidence interval, and 95% (two sigma) as seen done in
>> this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
>> ?(The first line is 68% and second line is 95% confidence intervals.)
>>
>> I would like to the set everything in the 68% confidence interval to
>> 1, 95% to 2 and everything else to 0. ?Thanks.
>
> It depends somewhat on what that function is. Is it a proper PDF?
> Let's say it is.
>
> I'll assume that the data has been normalized such that the sum of all
> the entries is 1. If not you'll have to keep track of that
> normalization constant. Let's call this array normed_pdf. What you are
> going to do is make a function that takes a value x, computes the sum
> normed_pdf[normed_pdf > x].sum(), and then subtracts the target value,
> say 0.68. Then use a root finder from scipy.optimize to find out where
> this function equals 0. Repeat for 0.95. Now just do a contour plot
> using those two values as the contour lines; e.g. in matplotlib, use
> the contour(Z, V) form.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



-- 
------------------------------------------------------------------------
Joseph Smidt <josephsmidt at gmail.com>

Physics and Astronomy
4129 Frederick Reines Hall
Irvine, CA 92697-4575
Office: 949-824-3269


From emmanuelle.gouillart at normalesup.org  Wed Jul 29 04:42:54 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Wed, 29 Jul 2009 10:42:54 +0200
Subject: [SciPy-User] [Numpy-discussion] [SciPy-user] Numpy/Scipy and
	the Python	African Tour
In-Reply-To: <db6b5ecc0907281039q7adfe9d6ic3ac652239633072@mail.gmail.com>
References: <20090723141737.GF10321@phare.normalesup.org>
	<db6b5ecc0907281039q7adfe9d6ic3ac652239633072@mail.gmail.com>
Message-ID: <20090729084254.GB22115@phare.normalesup.org>

	Dear Fernando and St?fan,

	thank you for the very encouraging answers!

On Tue, Jul 28, 2009 at 10:39:44AM -0700, Fernando Perez wrote:
> I'm glad to hear that you've set up a mailing list for your students
> to continue growing, but I'd also encourage them to subscribe to the
> 'normal' numpy/scipy lists.  Even if they initially feel a bit shy
> about posting, simply 'lurking' on the lists for a while and reading
> the flow of information can be very useful both to learn technical
> ideas and to get familiar with the discussion patterns of a community.
>  This will ease them into naturally engaging the upstream projects
> when they feel comfortable, which I think in the long term is both
> useful and necessary.

	I totally agree: I've already prompted the scientists on the list
to suscribe also to scipy-user and numpy-discussion. I also forwarded
your mail to the Dakar list, saying that if the Great Creator of Ipython
is saying the same thing, they really have no choice but to do it :D! The
local list is only meant to be a temporary solution to help the students
to gain more confidence; of course, we all know of temporary fixes that
have stayed unchanged for years :D, so we should be wary of inertia. The
good thing is that, as you say, it is possible to start posting on the
local list and get answers to short-term problems, and, at the same time,
lurk on the normal lists and learn tons of things.

> Mmh, I just had a thought.  A number of us have by now taught
> reasonably organized workshops of this kind in different places and
> institutions. I wonder if it wouldn't be a good idea to have a page on
> the scipy site collecting this information, with a short paragraph
> about the event and a link to any additional external materials that
> may exist.  This could encourage others to create similar things in
> their own communities, whether with the help of one of the previous
> presenters or fully independently.  Does that sound useful?

	This sounds as a great idea to me. This page could be linked from
the main documentation page. If the page is created, I'd be glad to
contribute with my slides (including sources, of course), and write a
short blurb about the Dakar event. Regarding St?fan's suggestion to find
a volunteer for translating the slides from French into English, I will
ask the Dakar students as well as the students in my lab. If nobody
volunteers I'll end up doing it myself, because slides only in French are
of limited use... Of course, if you know enthusiastic people with a lot
of free time, that know French and English, you're very welcome to probe
their interest in this task!

	Cheers,

	Emmanuelle


From harald.schilly at gmail.com  Wed Jul 29 05:12:39 2009
From: harald.schilly at gmail.com (Harald Schilly)
Date: Wed, 29 Jul 2009 11:12:39 +0200
Subject: [SciPy-User] [Numpy-discussion] [SciPy-user] Numpy/Scipy and
	the Python African Tour
In-Reply-To: <20090729084254.GB22115@phare.normalesup.org>
References: <20090723141737.GF10321@phare.normalesup.org>
	<db6b5ecc0907281039q7adfe9d6ic3ac652239633072@mail.gmail.com> 
	<20090729084254.GB22115@phare.normalesup.org>
Message-ID: <20548feb0907290212j263c0eccm4a7030f8b9e9e48a@mail.gmail.com>

On Wed, Jul 29, 2009 at 10:42, Emmanuelle
Gouillart<emmanuelle.gouillart at normalesup.org> wrote:
> Regarding St?fan's suggestion to find
> a volunteer for translating the slides from French into English, I will
> ask the Dakar students as well as the students in my lab. If nobody
> volunteers I'll end up doing it myself, because slides only in French are
> of limited use...

There is some new tool by google that might help you, translator
toolkit, where you can see the text document side by side with aided
translation and collaborate and share your work. right now, only
english as source language, but i guess this will change soon.
(http://translate.google.com/support/toolkit/bin/answer.py?hl=en&answer=152100)
it's here: http://translate.google.com/toolkit

Harald


From gokhansever at gmail.com  Wed Jul 29 10:30:20 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Wed, 29 Jul 2009 09:30:20 -0500
Subject: [SciPy-User] Curse of recursive directory walking
Message-ID: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>

I have a simple Python script which is written with a purpose to walk
through the given root directory and spot and read the right data files and
plot some of the desired variables within them for the first stage data
inspection.

The script named (dccn_plot_all.py) is given as below --while dccn_plot
being the main action script:


#!/usr/bin/env python

# dccn_plot_all

import os
from subprocess import call

init = os.getcwd()

for root, dirs, files in os.walk('.'):
    for file in files:
        if file.split('.')[1] == 'sau' and not file.endswith('.nc'):
            print file
            os.chdir(root)
            print os.getcwd()
            call(['dccn_plot', file])
            os.chdir(init)


Here is the fun part: While I was testing the script, I mistakenly called
this dccn_plot_all script itself within the os.walk suite. And this was
resulted with a almost complete lose of my Bash shell access on my Fedora 11
and made windows manager also unreachable. To recover, I had to restart the
system.

The lesson learnt: be extremely cautious while working in the nearby of
recursive functions.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090729/8e2af667/attachment.html>

From josef.pktd at gmail.com  Wed Jul 29 10:48:47 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 29 Jul 2009 10:48:47 -0400
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
Message-ID: <1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>

On Wed, Jul 29, 2009 at 10:30 AM, G?khan SEVER<gokhansever at gmail.com> wrote:
> I have a simple Python script which is written with a purpose to walk
> through the given root directory and spot and read the right data files and
> plot some of the desired variables within them for the first stage data
> inspection.
>
> The script named (dccn_plot_all.py) is given as below --while dccn_plot
> being the main action script:
>
>
> #!/usr/bin/env python
>
> # dccn_plot_all
>
> import os
> from subprocess import call
>
> init = os.getcwd()
>
> for root, dirs, files in os.walk('.'):
> ??? for file in files:
> ??????? if file.split('.')[1] == 'sau' and not file.endswith('.nc'):
> ??????????? print file
> ??????????? os.chdir(root)
> ??????????? print os.getcwd()
> ??????????? call(['dccn_plot', file])
> ??????????? os.chdir(init)
>
>
> Here is the fun part: While I was testing the script, I mistakenly called
> this dccn_plot_all script itself within the os.walk suite. And this was
> resulted with a almost complete lose of my Bash shell access on my Fedora 11
> and made windows manager also unreachable. To recover, I had to restart the
> system.
>
> The lesson learnt: be extremely cautious while working in the nearby of
> recursive functions.

To me it looks more like it is a problem with relative file paths and
changing directories and not with recursion.
I never had problems with os.walk or any home made recursive directory
walker, but I always used absolute paths.

from the python 2.5 docs

Note: If you pass a relative pathname, don't change the current
working directory between resumptions of walk(). walk() never changes
the current directory, and assumes that its caller doesn't either.

Josef

>
> --
> G?khan
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


From gokhansever at gmail.com  Wed Jul 29 17:14:12 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Wed, 29 Jul 2009 16:14:12 -0500
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
Message-ID: <49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>

On Wed, Jul 29, 2009 at 9:48 AM, <josef.pktd at gmail.com> wrote:

> To me it looks more like it is a problem with relative file paths and
> changing directories and not with recursion.
> I never had problems with os.walk or any home made recursive directory
> walker, but I always used absolute paths.
>
> from the python 2.5 docs
>
> Note: If you pass a relative pathname, don't change the current
> working directory between resumptions of walk(). walk() never changes
> the current directory, and assumes that its caller doesn't either.
>
> Josef



I have one more question on os.walk()

For some reason while this function working on my local folders I walks
through them in a randomly manner. However when I run the same script on a
network mounted drive (using the same directory structure) it walks through
them in an alphabetically sorted order.

Have you encountered a similar behaviour before?

Thanks.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090729/edf5e06c/attachment.html>

From josef.pktd at gmail.com  Wed Jul 29 19:25:50 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 29 Jul 2009 19:25:50 -0400
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
Message-ID: <1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com>

On Wed, Jul 29, 2009 at 5:14 PM, G?khan SEVER<gokhansever at gmail.com> wrote:
>
>
> On Wed, Jul 29, 2009 at 9:48 AM, <josef.pktd at gmail.com> wrote:
>>
>> To me it looks more like it is a problem with relative file paths and
>> changing directories and not with recursion.
>> I never had problems with os.walk or any home made recursive directory
>> walker, but I always used absolute paths.
>>
>> from the python 2.5 docs
>>
>> Note: If you pass a relative pathname, don't change the current
>> working directory between resumptions of walk(). walk() never changes
>> the current directory, and assumes that its caller doesn't either.
>>
>> Josef
>
>
> I have one more question on os.walk()
>
> For some reason while this function working on my local folders I walks
> through them in a randomly manner. However when I run the same script on a
> network mounted drive (using the same directory structure) it walks through
> them in an alphabetically sorted order.
>
> Have you encountered a similar behaviour before?

I only use windows, and it looks like I get windows fileordering,
alphabetically, independent of capitalization, and not capitalized
names sorted first. So I assume that the ordering depends on the
operating system.

to get deterministic ordering, inplace sorting should be possible, but
I never tried
python help: When topdown is true, the caller can modify the dirnames
list in-place

help for os.listdir says about the list of directory entries:  "The
list is in arbitrary order"

Josef


>
> Thanks.
>
> --
> G?khan
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


From gokhansever at gmail.com  Wed Jul 29 23:19:10 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Wed, 29 Jul 2009 22:19:10 -0500
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
	<1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com>
Message-ID: <49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>

On Wed, Jul 29, 2009 at 6:25 PM, <josef.pktd at gmail.com> wrote:

> On Wed, Jul 29, 2009 at 5:14 PM, G?khan SEVER<gokhansever at gmail.com>
> wrote:
> >
> >
> > On Wed, Jul 29, 2009 at 9:48 AM, <josef.pktd at gmail.com> wrote:
> >>
> >> To me it looks more like it is a problem with relative file paths and
> >> changing directories and not with recursion.
> >> I never had problems with os.walk or any home made recursive directory
> >> walker, but I always used absolute paths.
> >>
> >> from the python 2.5 docs
> >>
> >> Note: If you pass a relative pathname, don't change the current
> >> working directory between resumptions of walk(). walk() never changes
> >> the current directory, and assumes that its caller doesn't either.
> >>
> >> Josef
> >
> >
> > I have one more question on os.walk()
> >
> > For some reason while this function working on my local folders I walks
> > through them in a randomly manner. However when I run the same script on
> a
> > network mounted drive (using the same directory structure) it walks
> through
> > them in an alphabetically sorted order.
> >
> > Have you encountered a similar behaviour before?
>
> I only use windows, and it looks like I get windows fileordering,
> alphabetically, independent of capitalization, and not capitalized
> names sorted first. So I assume that the ordering depends on the
> operating system.
>
> to get deterministic ordering, inplace sorting should be possible, but
> I never tried
> python help: When topdown is true, the caller can modify the dirnames
> list in-place
>
> help for os.listdir says about the list of directory entries:  "The
> list is in arbitrary order"
>
> Josef
>
>
This is Fedora 11.

I still couldn't figure out why os.walk() preserves the fileordering
properly on samba mounted folders but not on my same local copies. The weird
thing is it always walks the given directory tree in the same arbitrary
order whenever I called the script.

Inplace listing may not be a good option, 1st I don't know how to do that,
and second when a directory under a given path is reached and if the desired
file exists under that directory it will execute my plotting script. It
works but a bizarre way :)

Maybe I should ask on a python mailing list.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090729/5d16120b/attachment.html>

From robert.kern at gmail.com  Thu Jul 30 00:42:35 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 29 Jul 2009 23:42:35 -0500
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com> 
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com> 
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com> 
	<1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com> 
	<49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>
Message-ID: <3d375d730907292142l604379c4y8a286e30a899b353@mail.gmail.com>

On Wed, Jul 29, 2009 at 22:19, G?khan SEVER<gokhansever at gmail.com> wrote:

> I still couldn't figure out why os.walk() preserves the fileordering
> properly on samba mounted folders but not on my same local copies. The weird
> thing is it always walks the given directory tree in the same arbitrary
> order whenever I called the script.

The implementation of the filesystem determines what order you get
them back in. That's why you got a consistent arbitrary order in your
local copies. It is also why you get a consistent arbitrary order in
your Samba folders; Samba's arbitrary ordering just happens to be what
you consider to be properly sorted.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From bala.biophysics at gmail.com  Thu Jul 30 01:32:50 2009
From: bala.biophysics at gmail.com (Bala subramanian)
Date: Thu, 30 Jul 2009 07:32:50 +0200
Subject: [SciPy-User] fitting data
Message-ID: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>

Friends,

I am new and just joined the scipy community. I want to fit (least sq. fit)
my data to the following equation. I would greatly appreciate if someone
could help me with a sample code on how fitting is done with scipy.

The function is : f = a * fb + (1-a) fo, where f, fb, fo are known.

Regards,
Bala
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090730/3f213d7f/attachment.html>

From sccolbert at gmail.com  Thu Jul 30 01:46:47 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Thu, 30 Jul 2009 01:46:47 -0400
Subject: [SciPy-User] fitting data
In-Reply-To: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
Message-ID: <7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>

maybe i don't understand what you are asking, but if (a) is your only
unknown, just do:

a = (f - f0) / (fb - f0)



On Thu, Jul 30, 2009 at 1:32 AM, Bala
subramanian<bala.biophysics at gmail.com> wrote:
> Friends,
>
> I am new and just joined the scipy community. I want to fit (least sq. fit)
> my data to the following equation. I would greatly appreciate if someone
> could help me with a sample code on how fitting is done with scipy.
>
> The function is : f = a * fb + (1-a) fo, where f, fb, fo are known.
>
> Regards,
> Bala
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


From cohen at lpta.in2p3.fr  Thu Jul 30 01:49:35 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Thu, 30 Jul 2009 07:49:35 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
	<7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>
Message-ID: <4A71346F.3070407@lpta.in2p3.fr>

I thinkk he means that f0 and fb are two known function of x, and he is 
interested in the weight parameter a.
You can try :
from scipy import optimize
def func(x,a): return a * fb(x) + (1-a) fo(x)
fitp,cov=optimize.curve_fit(func,x,p)

that is one of many options....
Johann



Chris Colbert wrote:
> maybe i don't understand what you are asking, but if (a) is your only
> unknown, just do:
>
> a = (f - f0) / (fb - f0)
>
>
>
> On Thu, Jul 30, 2009 at 1:32 AM, Bala
> subramanian<bala.biophysics at gmail.com> wrote:
>   
>> Friends,
>>
>> I am new and just joined the scipy community. I want to fit (least sq. fit)
>> my data to the following equation. I would greatly appreciate if someone
>> could help me with a sample code on how fitting is done with scipy.
>>
>> The function is : f = a * fb + (1-a) fo, where f, fb, fo are known.
>>
>> Regards,
>> Bala
>>
>>
>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>
>>     
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From wierob83 at googlemail.com  Thu Jul 30 03:01:59 2009
From: wierob83 at googlemail.com (wierob)
Date: Thu, 30 Jul 2009 09:01:59 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
Message-ID: <4A714567.1040900@googlemail.com>

maybe this helps: http://www.scipy.org/Cookbook/OLS

Bala subramanian schrieb:
> Friends,
>
> I am new and just joined the scipy community. I want to fit (least sq. 
> fit) my data to the following equation. I would greatly appreciate if 
> someone could help me with a sample code on how fitting is done with 
> scipy.
>
> The function is : f = a * fb + (1-a) fo, where f, fb, fo are known.
>
> Regards,
> Bala
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From emmanuelle.gouillart at normalesup.org  Thu Jul 30 03:08:27 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Thu, 30 Jul 2009 09:08:27 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <4A71346F.3070407@lpta.in2p3.fr>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
	<7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>
	<4A71346F.3070407@lpta.in2p3.fr>
Message-ID: <20090730070827.GA18424@phare.normalesup.org>

> from scipy import optimize
> def func(x,a): return a * fb(x) + (1-a) fo(x)
> fitp,cov=optimize.curve_fit(func,x,p)

	There is no optimize.curve_fit im my version scipy:
>>> from scipy import optimize
>>> help optimize.curve_fit
--> help(optimize.curve_fit)
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
AttributeError: 'module' object has no attribute 'curve_fit'
>>> import scipy
>>> scipy.__version__
'0.7.0'

	Am I missing something?

	Bala, one of the classical solutions is to use
scipy.optimize.leastsq; there is a whole page on the scipy Cookbook
dedicated to fitting problems http://www.scipy.org/Cookbook/FittingData.
The page provides many examples of how to use leastsq for fitting data.

	Cheers,

	Emmanuelle


From scott.sinclair.za at gmail.com  Thu Jul 30 03:21:02 2009
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Thu, 30 Jul 2009 09:21:02 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <20090730070827.GA18424@phare.normalesup.org>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
	<7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>
	<4A71346F.3070407@lpta.in2p3.fr>
	<20090730070827.GA18424@phare.normalesup.org>
Message-ID: <6a17e9ee0907300021p165bb09eu6d23cba43fee1328@mail.gmail.com>

> 2009/7/30 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
> ? ? ? ?There is no optimize.curve_fit im my version scipy:
>>>> from scipy import optimize
>>>> help optimize.curve_fit
> --> help(optimize.curve_fit)
> ------------------------------------------------------------
> Traceback (most recent call last):
> ?File "<ipython console>", line 1, in <module>
> AttributeError: 'module' object has no attribute 'curve_fit'
>>>> import scipy
>>>> scipy.__version__
> '0.7.0'
>
> ? ? ? ?Am I missing something?

No, it's only in SVN. It was added a few days after the 0.7 release.

http://projects.scipy.org/scipy/changeset/5543/trunk/scipy/optimize/minpack.py

Cheers,
Scott


From bala.biophysics at gmail.com  Thu Jul 30 03:22:50 2009
From: bala.biophysics at gmail.com (Bala subramanian)
Date: Thu, 30 Jul 2009 09:22:50 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <4A71346F.3070407@lpta.in2p3.fr>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>
	<7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>
	<4A71346F.3070407@lpta.in2p3.fr>
Message-ID: <288df32a0907300022v5f5e424k7d78ed4e8e826064@mail.gmail.com>

Sorry for not giving clear info. in my previous mail.

Actually, i have two equations 1) f = a * fb + (1-a) fo, where f, fb, fo are
known

and 2) a = 1/2ao[ (ao + x + bo) - { (ao + x + bo)2 - 4 ao bo }0.5 ]

>From eq 1, i should get 'a' and then i use this in eq 2 to find bo.

On Thu, Jul 30, 2009 at 7:49 AM, Johann Cohen-Tanugi <cohen at lpta.in2p3.fr>wrote:

> I thinkk he means that f0 and fb are two known function of x, and he is
> interested in the weight parameter a.
> You can try :
> from scipy import optimize
> def func(x,a): return a * fb(x) + (1-a) fo(x)
> fitp,cov=optimize.curve_fit(func,x,p)
>
> that is one of many options....
> Johann
>
>
>
> Chris Colbert wrote:
> > maybe i don't understand what you are asking, but if (a) is your only
> > unknown, just do:
> >
> > a = (f - f0) / (fb - f0)
> >
> >
> >
> > On Thu, Jul 30, 2009 at 1:32 AM, Bala
> > subramanian<bala.biophysics at gmail.com> wrote:
> >
> >> Friends,
> >>
> >> I am new and just joined the scipy community. I want to fit (least sq.
> fit)
> >> my data to the following equation. I would greatly appreciate if someone
> >> could help me with a sample code on how fitting is done with scipy.
> >>
> >> The function is : f = a * fb + (1-a) fo, where f, fb, fo are known.
> >>
> >> Regards,
> >> Bala
> >>
> >>
> >>
> >> _______________________________________________
> >> SciPy-User mailing list
> >> SciPy-User at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-user
> >>
> >>
> >>
> > _______________________________________________
> > SciPy-User mailing list
> > SciPy-User at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090730/eb5509dd/attachment.html>

From cohen at lpta.in2p3.fr  Thu Jul 30 03:20:03 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Thu, 30 Jul 2009 09:20:03 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <6a17e9ee0907300021p165bb09eu6d23cba43fee1328@mail.gmail.com>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>	<7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>	<4A71346F.3070407@lpta.in2p3.fr>	<20090730070827.GA18424@phare.normalesup.org>
	<6a17e9ee0907300021p165bb09eu6d23cba43fee1328@mail.gmail.com>
Message-ID: <4A7149A3.4090907@lpta.in2p3.fr>

oops sorry, I did not pay attention. Yes Emmanuelle is right, leastsqr 
is the way to go, only very slightly more involved for a beginner, the 
cookbook page is great to learn to use it.

Johann

Scott Sinclair wrote:
>> 2009/7/30 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
>>        There is no optimize.curve_fit im my version scipy:
>>     
>>>>> from scipy import optimize
>>>>> help optimize.curve_fit
>>>>>           
>> --> help(optimize.curve_fit)
>> ------------------------------------------------------------
>> Traceback (most recent call last):
>>  File "<ipython console>", line 1, in <module>
>> AttributeError: 'module' object has no attribute 'curve_fit'
>>     
>>>>> import scipy
>>>>> scipy.__version__
>>>>>           
>> '0.7.0'
>>
>>        Am I missing something?
>>     
>
> No, it's only in SVN. It was added a few days after the 0.7 release.
>
> http://projects.scipy.org/scipy/changeset/5543/trunk/scipy/optimize/minpack.py
>
> Cheers,
> Scott
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From cohen at lpta.in2p3.fr  Thu Jul 30 03:34:36 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Thu, 30 Jul 2009 09:34:36 +0200
Subject: [SciPy-User] fitting data
In-Reply-To: <288df32a0907300022v5f5e424k7d78ed4e8e826064@mail.gmail.com>
References: <288df32a0907292232xef8efc0k6273b3076c5d77d2@mail.gmail.com>	
	<7f014ea60907292246j6218b32cu9be604c792ebafbc@mail.gmail.com>	
	<4A71346F.3070407@lpta.in2p3.fr>
	<288df32a0907300022v5f5e424k7d78ed4e8e826064@mail.gmail.com>
Message-ID: <4A714D0C.4060401@lpta.in2p3.fr>

Bala, I think that you are mistyping your problem : the second equation 
looks a lot like one of the 2 solutions of a second order polynom a0*X^2 
-2*(a0+x+b0)*X+b0
Is that right?
In any case, replace a  by  (f-f0)/(fb-f0) in eq.2  and then  you have 
an implicit function g(x,b0)
Johann

Bala subramanian wrote:
> Sorry for not giving clear info. in my previous mail.
>
> Actually, i have two equations 1) f = a * fb + (1-a) fo, where f, fb, 
> fo are known
>
> and 2) a = 1/2ao[ (ao + x + bo) - { (ao + x + bo)2 - 4 ao bo }0.5 ]
>
> From eq 1, i should get 'a' and then i use this in eq 2 to find bo.
>
> On Thu, Jul 30, 2009 at 7:49 AM, Johann Cohen-Tanugi 
> <cohen at lpta.in2p3.fr <mailto:cohen at lpta.in2p3.fr>> wrote:
>
>     I thinkk he means that f0 and fb are two known function of x, and
>     he is
>     interested in the weight parameter a.
>     You can try :
>     from scipy import optimize
>     def func(x,a): return a * fb(x) + (1-a) fo(x)
>     fitp,cov=optimize.curve_fit(func,x,p)
>
>     that is one of many options....
>     Johann
>
>
>
>     Chris Colbert wrote:
>     > maybe i don't understand what you are asking, but if (a) is your
>     only
>     > unknown, just do:
>     >
>     > a = (f - f0) / (fb - f0)
>     >
>     >
>     >
>     > On Thu, Jul 30, 2009 at 1:32 AM, Bala
>     > subramanian<bala.biophysics at gmail.com
>     <mailto:bala.biophysics at gmail.com>> wrote:
>     >
>     >> Friends,
>     >>
>     >> I am new and just joined the scipy community. I want to fit
>     (least sq. fit)
>     >> my data to the following equation. I would greatly appreciate
>     if someone
>     >> could help me with a sample code on how fitting is done with scipy.
>     >>
>     >> The function is : f = a * fb + (1-a) fo, where f, fb, fo are known.
>     >>
>     >> Regards,
>     >> Bala
>     >>
>     >>
>     >>
>     >> _______________________________________________
>     >> SciPy-User mailing list
>     >> SciPy-User at scipy.org <mailto:SciPy-User at scipy.org>
>     >> http://mail.scipy.org/mailman/listinfo/scipy-user
>     >>
>     >>
>     >>
>     > _______________________________________________
>     > SciPy-User mailing list
>     > SciPy-User at scipy.org <mailto:SciPy-User at scipy.org>
>     > http://mail.scipy.org/mailman/listinfo/scipy-user
>     >
>     _______________________________________________
>     SciPy-User mailing list
>     SciPy-User at scipy.org <mailto:SciPy-User at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


From cohen at lpta.in2p3.fr  Thu Jul 30 04:08:21 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Thu, 30 Jul 2009 10:08:21 +0200
Subject: [SciPy-User] Easy Way To Find Confidence Intervals For 2D Data?
In-Reply-To: <3d375d730907281933l40aa77d8p72d9172c55cedfe@mail.gmail.com>
References: <142682e10907281834w6bd3f7ebm7df2ddc0b28ecc1b@mail.gmail.com>
	<3d375d730907281933l40aa77d8p72d9172c55cedfe@mail.gmail.com>
Message-ID: <4A7154F5.6070803@lpta.in2p3.fr>

sounds like a perfect call for a scipy cookbook page!
JCT

Robert Kern wrote:
> On Tue, Jul 28, 2009 at 20:34, Joseph Smidt<josephsmidt at gmail.com> wrote:
>   
>> I have a function on a 2d grid that looks like a skewed mound. I would
>> like to calculate everything in the 68% (one sigma or standard
>> deviation) confidence interval, and 95% (two sigma) as seen done in
>> this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
>>  (The first line is 68% and second line is 95% confidence intervals.)
>>
>> I would like to the set everything in the 68% confidence interval to
>> 1, 95% to 2 and everything else to 0.  Thanks.
>>     
>
> It depends somewhat on what that function is. Is it a proper PDF?
> Let's say it is.
>
> I'll assume that the data has been normalized such that the sum of all
> the entries is 1. If not you'll have to keep track of that
> normalization constant. Let's call this array normed_pdf. What you are
> going to do is make a function that takes a value x, computes the sum
> normed_pdf[normed_pdf > x].sum(), and then subtracts the target value,
> say 0.68. Then use a root finder from scipy.optimize to find out where
> this function equals 0. Repeat for 0.95. Now just do a contour plot
> using those two values as the contour lines; e.g. in matplotlib, use
> the contour(Z, V) form.
>
>   


From harald.schilly at gmail.com  Thu Jul 30 05:38:52 2009
From: harald.schilly at gmail.com (Harald Schilly)
Date: Thu, 30 Jul 2009 11:38:52 +0200
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com> 
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com> 
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
Message-ID: <20548feb0907300238x35ed7fbbmc61d79ca15fc1322@mail.gmail.com>

On Wed, Jul 29, 2009 at 23:14, G?khan SEVER<gokhansever at gmail.com> wrote:
> For some reason while this function working on my local folders I walks
> through them in a randomly manner.

This simply depends on the file system and the OS. Maybe your random
order is just the sorting given by the inode index number!

H


From josef.pktd at gmail.com  Thu Jul 30 09:04:00 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 30 Jul 2009 09:04:00 -0400
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
	<1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com>
	<49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>
Message-ID: <1cd32cbb0907300604p571eecf7r2b26f2b147f3e9d3@mail.gmail.com>

On Wed, Jul 29, 2009 at 11:19 PM, G?khan SEVER<gokhansever at gmail.com> wrote:
>
>
> On Wed, Jul 29, 2009 at 6:25 PM, <josef.pktd at gmail.com> wrote:
>>
>> On Wed, Jul 29, 2009 at 5:14 PM, G?khan SEVER<gokhansever at gmail.com>
>> wrote:
>> >
>> >
>> > On Wed, Jul 29, 2009 at 9:48 AM, <josef.pktd at gmail.com> wrote:
>> >>
>> >> To me it looks more like it is a problem with relative file paths and
>> >> changing directories and not with recursion.
>> >> I never had problems with os.walk or any home made recursive directory
>> >> walker, but I always used absolute paths.
>> >>
>> >> from the python 2.5 docs
>> >>
>> >> Note: If you pass a relative pathname, don't change the current
>> >> working directory between resumptions of walk(). walk() never changes
>> >> the current directory, and assumes that its caller doesn't either.
>> >>
>> >> Josef
>> >
>> >
>> > I have one more question on os.walk()
>> >
>> > For some reason while this function working on my local folders I walks
>> > through them in a randomly manner. However when I run the same script on
>> > a
>> > network mounted drive (using the same directory structure) it walks
>> > through
>> > them in an alphabetically sorted order.
>> >
>> > Have you encountered a similar behaviour before?
>>
>> I only use windows, and it looks like I get windows fileordering,
>> alphabetically, independent of capitalization, and not capitalized
>> names sorted first. So I assume that the ordering depends on the
>> operating system.
>>
>> to get deterministic ordering, inplace sorting should be possible, but
>> I never tried
>> python help: When topdown is true, the caller can modify the dirnames
>> list in-place
>>
>> help for os.listdir says about the list of directory entries: ?"The
>> list is in arbitrary order"
>>
>> Josef
>>
>
> This is Fedora 11.
>
> I still couldn't figure out why os.walk() preserves the fileordering
> properly on samba mounted folders but not on my same local copies. The weird
> thing is it always walks the given directory tree in the same arbitrary
> order whenever I called the script.
>
> Inplace listing may not be a good option, 1st I don't know how to do that,
> and second when a directory under a given path is reached and if the desired
> file exists under that directory it will execute my plotting script. It
> works but a bizarre way :)
>
> Maybe I should ask on a python mailing list.
>
> --
> G?khan
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>

something like this might work.

>>> for r,d,f in os.walk(r'C:\path\of\directory'):
	d.sort()
	d.reverse()
	print r

this gives me walking the tree in reverse, python-alphabetical order
(capital letters first)

Josef


From jsseabold at gmail.com  Thu Jul 30 11:24:45 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Thu, 30 Jul 2009 11:24:45 -0400
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <3d375d730907292142l604379c4y8a286e30a899b353@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com> 
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com> 
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com> 
	<1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com> 
	<49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com> 
	<3d375d730907292142l604379c4y8a286e30a899b353@mail.gmail.com>
Message-ID: <c048da1c0907300824o3d70e77clb2810dea53713cf2@mail.gmail.com>

On Thu, Jul 30, 2009 at 12:42 AM, Robert Kern<robert.kern at gmail.com> wrote:
> On Wed, Jul 29, 2009 at 22:19, G?khan SEVER<gokhansever at gmail.com> wrote:
>
>> I still couldn't figure out why os.walk() preserves the fileordering
>> properly on samba mounted folders but not on my same local copies. The weird
>> thing is it always walks the given directory tree in the same arbitrary
>> order whenever I called the script.
>
> The implementation of the filesystem determines what order you get
> them back in. That's why you got a consistent arbitrary order in your
> local copies. It is also why you get a consistent arbitrary order in
> your Samba folders; Samba's arbitrary ordering just happens to be what
> you consider to be properly sorted.
>

Also note that Samba is based on the SMB protocol used by the Windows
network file system.  I use Samba to share my Linux folders over my
network to Windows boxes, so it makes sense that it's the same as the
Windows ordering.

Cheers,
Skipper


From gokhansever at gmail.com  Thu Jul 30 15:33:22 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Thu, 30 Jul 2009 14:33:22 -0500
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <20548feb0907300238x35ed7fbbmc61d79ca15fc1322@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
	<20548feb0907300238x35ed7fbbmc61d79ca15fc1322@mail.gmail.com>
Message-ID: <49d6b3500907301233h65820026ka4f406facdf0a2e3@mail.gmail.com>

On Thu, Jul 30, 2009 at 4:38 AM, Harald Schilly <harald.schilly at gmail.com>wrote:

> On Wed, Jul 29, 2009 at 23:14, G?khan SEVER<gokhansever at gmail.com> wrote:
> > For some reason while this function working on my local folders I walks
> > through them in a randomly manner.
>
> This simply depends on the file system and the OS. Maybe your random
> order is just the sorting given by the inode index number!
>
> H
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>

I checked the directory with "ls -li". os.walk() doesn't base off on inodes.
The way inodes ordered and the files are walked through are in different
order.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090730/91f320dc/attachment.html>

From gokhansever at gmail.com  Thu Jul 30 15:39:04 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Thu, 30 Jul 2009 14:39:04 -0500
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <1cd32cbb0907300604p571eecf7r2b26f2b147f3e9d3@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
	<1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com>
	<49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>
	<1cd32cbb0907300604p571eecf7r2b26f2b147f3e9d3@mail.gmail.com>
Message-ID: <49d6b3500907301239o3dedc586h4474ea6abbe6f2c1@mail.gmail.com>

On Thu, Jul 30, 2009 at 8:04 AM, <josef.pktd at gmail.com> wrote:

>
> something like this might work.
>
> >>> for r,d,f in os.walk(r'C:\path\of\directory'):
>        d.sort()
>        d.reverse()
>        print r
>
> this gives me walking the tree in reverse, python-alphabetical order
> (capital letters first)
>
> Josef
>

d.sort() did the trick.

Thanks for your help. Now it behaves as I wanted it to work.


-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090730/97d2f8ef/attachment.html>

From gokhansever at gmail.com  Thu Jul 30 15:41:45 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Thu, 30 Jul 2009 14:41:45 -0500
Subject: [SciPy-User] Curse of recursive directory walking
In-Reply-To: <c048da1c0907300824o3d70e77clb2810dea53713cf2@mail.gmail.com>
References: <49d6b3500907290730o94161dl3a77aa6f39980714@mail.gmail.com>
	<1cd32cbb0907290748yfe30fcctbf69252b96039d03@mail.gmail.com>
	<49d6b3500907291414k7b934179yf35a0f85b19c8e17@mail.gmail.com>
	<1cd32cbb0907291625m5d876beemf721ce98b97461c1@mail.gmail.com>
	<49d6b3500907292019g7e7b91ddsc447717ced787c63@mail.gmail.com>
	<3d375d730907292142l604379c4y8a286e30a899b353@mail.gmail.com>
	<c048da1c0907300824o3d70e77clb2810dea53713cf2@mail.gmail.com>
Message-ID: <49d6b3500907301241t6f0c35a0t3807e7c32418898a@mail.gmail.com>

On Thu, Jul 30, 2009 at 10:24 AM, Skipper Seabold <jsseabold at gmail.com>wrote:

> Also note that Samba is based on the SMB protocol used by the Windows
> network file system.  I use Samba to share my Linux folders over my
> network to Windows boxes, so it makes sense that it's the same as the
> Windows ordering.
>
> Cheers,
> Skipper
>
>
We use Samba in the same way. There is a few big common drives that are
mounted on either Linux or Windows machines.

Thanks for the tip.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090730/1c2289e5/attachment.html>

From amenity at enthought.com  Thu Jul 30 19:48:18 2009
From: amenity at enthought.com (Amenity Applewhite)
Date: Thu, 30 Jul 2009 18:48:18 -0500
Subject: [SciPy-User] EPD Webinar August 7: Speed up Python (& wait list for
	non-subscribers!)
Message-ID: <F67378B9-134A-497D-98C8-0E78E3C69D19@enthought.com>

Friday, August 7th 1pm CDT
How do I...speed up Python?  Exploration of Weave and Cython
We wanted to let you know that next week we'll host another  
installment of our popular EPD webinar series. Although only EPD Basic  
or above subscribers are guaranteed seats at EPD webinars, we invite  
non-subscribers to add their names to the wating list for each event.  
If there are available seats, you will be notified by next Thursday  
and given access to the webinar.  Links to the waiting lists and  
upcoming topics are posted here: http://enthought.com/training/webinars.php

These events feature detailed demonstrations of powerful Python  
techniques that Enthought developers use to enhance our applications  
or development process. Participants are often invited to participate  
in the demonstration, and are welcome to join the interactive VOIP  
discussion later in the session. This is a great opportunity to learn  
new methods and interact with our expert developers.

If you have topics you'd like to see addressed during the webinar,  
feel free to let us know at media at enthought.com.
How do I...speed up Python:  Exploration of Weave and Cython
Python excels as a language for rapid development and interactive  
technical computing.   Sometimes, however, the full speed of the  
machine is needed.  Fortunately, Python is also an excellent "glue"  
language that allows simple integration to compiled languages.

In this webinar, we will present a few of the methods useful for  
speeding up sections of code.  In particular we will focus on weave  
and Cython.   Weave is a sub-package of SciPy (scipy.weave) and allows  
seamless integration with C/C++.    Sections of code that need to be  
sped up can be re-coded in C and seamlessly integrated into the  
remaining Python program using weave.   Cython is a newer Python-like  
language which is compiled to an extension module that can be  
dynamically loaded into Python just like any other Python module.   
Cython also supports writing new built-in Python types and tries to  
compile loops to C-speeds when variable types are indicated.   Both  
weave and Cython integrate well with NumPy which makes them ideal  
whenever you need to iterate and calculate with large amounts of data  
and can't find a built-in (vectorized) approach.

Once again, to add your name to the wait-list:
http://enthought.com/training/webinars.php

We hope to see you there!

Thanks,
Enthought Media
Quick Links...
Enthought.com
Enthought Python Distribution (EPD)
Enthought Webinars

@Facebook
@Twitter
--
Amenity Applewhite
Enthought, Inc.
Scientific Computing Solutions
www.enthought.com









-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090730/823deb06/attachment.html>

From permafacture at gmail.com  Fri Jul 31 00:03:40 2009
From: permafacture at gmail.com (Permafacture)
Date: Thu, 30 Jul 2009 21:03:40 -0700 (PDT)
Subject: [SciPy-User] Solve an arbitrary function
Message-ID: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>

I'm new to this package, and am used to mathematica.

I've been reading through the tutorials and numpy lists of functions,
yet i cannot find the answer to this, though it would be trivial in
mathematica.  Can someone show me an example of how to find the
intersection of a line and a polynomial?

for instance: y1=x1^2, y2=x2+1

Solving a system of linear equations is pretty straight forward, but
it isn't clear to me how to approach this in scipy.  The most general
approach to this would probably be the most useful to me.


thanks.


From robert.kern at gmail.com  Fri Jul 31 00:17:33 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Jul 2009 23:17:33 -0500
Subject: [SciPy-User] Solve an arbitrary function
In-Reply-To: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>
References: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>
Message-ID: <3d375d730907302117n78a97481w80f9742233ffdba9@mail.gmail.com>

On Thu, Jul 30, 2009 at 23:03, Permafacture<permafacture at gmail.com> wrote:
> I'm new to this package, and am used to mathematica.
>
> I've been reading through the tutorials and numpy lists of functions,
> yet i cannot find the answer to this, though it would be trivial in
> mathematica. ?Can someone show me an example of how to find the
> intersection of a line and a polynomial?
>
> for instance: y1=x1^2, y2=x2+1
>
> Solving a system of linear equations is pretty straight forward, but
> it isn't clear to me how to approach this in scipy. ?The most general
> approach to this would probably be the most useful to me.

http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html#scipy.optimize.fsolve


In [1]: from scipy.optimize import fsolve

In [2]: import numpy as np

In [3]: def f(xy):
   ...:     x, y = xy
   ...:     z = np.array([y - x**2, y - x - 1.0])
   ...:     return z
   ...:

In [4]: fsolve(f, [1.0, 2.0])
Out[4]: array([ 1.61803399,  2.61803399])

In [5]: fsolve(f, [-0.5, 1.5])
Out[5]: array([-0.61803399,  0.38196601])


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From vanforeest at gmail.com  Fri Jul 31 03:19:21 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Fri, 31 Jul 2009 09:19:21 +0200
Subject: [SciPy-User] Solve an arbitrary function
In-Reply-To: <3d375d730907302117n78a97481w80f9742233ffdba9@mail.gmail.com>
References: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>
	<3d375d730907302117n78a97481w80f9742233ffdba9@mail.gmail.com>
Message-ID: <fa510ff80907310019u1075e274uf128844999db1c79@mail.gmail.com>

Hi,

If you like a graphical approach you might give PyX a try, and have
postscript solve your problem:

http://pyx.sourceforge.net/examples/path/intersect.html

2009/7/31 Robert Kern <robert.kern at gmail.com>:
> On Thu, Jul 30, 2009 at 23:03, Permafacture<permafacture at gmail.com> wrote:
>> I'm new to this package, and am used to mathematica.
>>
>> I've been reading through the tutorials and numpy lists of functions,
>> yet i cannot find the answer to this, though it would be trivial in
>> mathematica. ?Can someone show me an example of how to find the
>> intersection of a line and a polynomial?
>>
>> for instance: y1=x1^2, y2=x2+1
>>
>> Solving a system of linear equations is pretty straight forward, but
>> it isn't clear to me how to approach this in scipy. ?The most general
>> approach to this would probably be the most useful to me.

What would be the best place to document examples such as Robert's
below? I am afraid such examples get burried in the mailing list while
they are great examples of how to use fsolve, for instance.

Nicky

>
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html#scipy.optimize.fsolve

>
>
> In [1]: from scipy.optimize import fsolve
>
> In [2]: import numpy as np
>
> In [3]: def f(xy):
> ? ...: ? ? x, y = xy
> ? ...: ? ? z = np.array([y - x**2, y - x - 1.0])
> ? ...: ? ? return z
> ? ...:
>
> In [4]: fsolve(f, [1.0, 2.0])
> Out[4]: array([ 1.61803399, ?2.61803399])
>
> In [5]: fsolve(f, [-0.5, 1.5])
> Out[5]: array([-0.61803399, ?0.38196601])
>
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From bala.biophysics at gmail.com  Fri Jul 31 04:00:45 2009
From: bala.biophysics at gmail.com (Bala subramanian)
Date: Fri, 31 Jul 2009 13:30:45 +0530
Subject: [SciPy-User] book on scipy
Message-ID: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>

Friends,

Someone please suggest me some good books on scipy/numpy for a beginner.

Thanks,
Bala
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090731/5ddc5950/attachment.html>

From elcorto at gmx.net  Fri Jul 31 04:02:23 2009
From: elcorto at gmx.net (Steve Schmerler)
Date: Fri, 31 Jul 2009 10:02:23 +0200
Subject: [SciPy-User] Solve an arbitrary function
In-Reply-To: <fa510ff80907310019u1075e274uf128844999db1c79@mail.gmail.com>
References: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>
	<3d375d730907302117n78a97481w80f9742233ffdba9@mail.gmail.com>
	<fa510ff80907310019u1075e274uf128844999db1c79@mail.gmail.com>
Message-ID: <20090731080223.GA25790@ramrod.starsheriffs.de>

On Jul 31 09:19 +0200, nicky van foreest  wrote:
 
> What would be the best place to document examples such as Robert's
> below? I am afraid such examples get burried in the mailing list while
> they are great examples of how to use fsolve, for instance.

http://scipy.org/Cookbook would be the place, I guess.

best,
steve


From gael.varoquaux at normalesup.org  Fri Jul 31 04:03:32 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 31 Jul 2009 10:03:32 +0200
Subject: [SciPy-User] Solve an arbitrary function
In-Reply-To: <fa510ff80907310019u1075e274uf128844999db1c79@mail.gmail.com>
References: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>
	<3d375d730907302117n78a97481w80f9742233ffdba9@mail.gmail.com>
	<fa510ff80907310019u1075e274uf128844999db1c79@mail.gmail.com>
Message-ID: <20090731080332.GA9207@phare.normalesup.org>

On Fri, Jul 31, 2009 at 09:19:21AM +0200, nicky van foreest wrote:
> What would be the best place to document examples such as Robert's
> below? I am afraid such examples get burried in the mailing list while
> they are great examples of how to use fsolve, for instance.

That's a tough question. You can of course edit the Scipy Cookbook
(www.scipy.org/Cookbook), but the problem with this approach is that it
grows messy and may be hard to find.

The best solution is to manage to make the scipy docuementation
(http://docs.scipy.org/doc/scipy/reference/) good enough so that people
find it, read it, and quickly find an answer to their problems. That
requires careful organisation of the answers and examples, and linking
between them. Indeed, everybody has a different question and adding lots
of examples without any organization in the docs will not solve the
problem.

If you want to pitch in, it is easy (as least technically), you can use
your loggin on the doc wiki.

Cheers,

Ga?l


From gael.varoquaux at normalesup.org  Fri Jul 31 04:05:22 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 31 Jul 2009 10:05:22 +0200
Subject: [SciPy-User] book on scipy
In-Reply-To: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
Message-ID: <20090731080522.GB9207@phare.normalesup.org>

On Fri, Jul 31, 2009 at 01:30:45PM +0530, Bala subramanian wrote:
>    Someone please suggest me some good books on scipy/numpy?for a beginner.

Hans Petter Langtangen: Python Scripting for computational science: by
the third edition, or download the 2nd:

http://www.tinydl.com/ebooks/15335-hans-petter-langtangen-python-scripting-for.html

The 3rd is much better than the 2nd, IMHO.

Ga?l


From sccolbert at gmail.com  Fri Jul 31 04:47:34 2009
From: sccolbert at gmail.com (Chris Colbert)
Date: Fri, 31 Jul 2009 04:47:34 -0400
Subject: [SciPy-User] book on scipy
In-Reply-To: <20090731080522.GB9207@phare.normalesup.org>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
	<20090731080522.GB9207@phare.normalesup.org>
Message-ID: <7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>

Gael,

That link is just full of internet trash, popups, and the like. I'm
glad i didnt click on it with a windows box.

Do you have a copy of the book? I can host it on my server for a while
and put up a link for the list.

Cheers!

Chris

On Fri, Jul 31, 2009 at 4:05 AM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Fri, Jul 31, 2009 at 01:30:45PM +0530, Bala subramanian wrote:
>> ? ?Someone please suggest me some good books on scipy/numpy?for a beginner.
>
> Hans Petter Langtangen: Python Scripting for computational science: by
> the third edition, or download the 2nd:
>
> http://www.tinydl.com/ebooks/15335-hans-petter-langtangen-python-scripting-for.html
>
> The 3rd is much better than the 2nd, IMHO.
>
> Ga?l
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


From gael.varoquaux at normalesup.org  Fri Jul 31 04:49:52 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 31 Jul 2009 10:49:52 +0200
Subject: [SciPy-User] book on scipy
In-Reply-To: <7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
	<20090731080522.GB9207@phare.normalesup.org>
	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>
Message-ID: <20090731084952.GC9207@phare.normalesup.org>

On Fri, Jul 31, 2009 at 04:47:34AM -0400, Chris Colbert wrote:
> Gael,

> That link is just full of internet trash, popups, and the like. I'm
> glad i didnt click on it with a windows box.

Yes, it was the first hit on google, but I should have checked it.

The good link is:

http://vefur.simula.no/~hpl/scripting/index.html

Ga?l


From cohen at lpta.in2p3.fr  Fri Jul 31 07:23:11 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Fri, 31 Jul 2009 13:23:11 +0200
Subject: [SciPy-User] book on scipy
In-Reply-To: <20090731084952.GC9207@phare.normalesup.org>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>	<20090731080522.GB9207@phare.normalesup.org>	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>
	<20090731084952.GC9207@phare.normalesup.org>
Message-ID: <4A72D41F.9050703@lpta.in2p3.fr>

and this link seems to have the 3rd edition, actually, ev en its second 
printing!
thanks Gael,
Johann

Gael Varoquaux wrote:
> On Fri, Jul 31, 2009 at 04:47:34AM -0400, Chris Colbert wrote:
>   
>> Gael,
>>     
>
>   
>> That link is just full of internet trash, popups, and the like. I'm
>> glad i didnt click on it with a windows box.
>>     
>
> Yes, it was the first hit on google, but I should have checked it.
>
> The good link is:
>
> http://vefur.simula.no/~hpl/scripting/index.html
>
> Ga?l
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From cohen at lpta.in2p3.fr  Fri Jul 31 07:24:44 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Fri, 31 Jul 2009 13:24:44 +0200
Subject: [SciPy-User] book on scipy
In-Reply-To: <20090731084952.GC9207@phare.normalesup.org>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>	<20090731080522.GB9207@phare.normalesup.org>	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>
	<20090731084952.GC9207@phare.normalesup.org>
Message-ID: <4A72D47C.1060002@lpta.in2p3.fr>

well... actually no! The download is blocked by the need to have a 
printed edition at hand to check first words on (I presume) random 
pages.....
Johann

Gael Varoquaux wrote:
> On Fri, Jul 31, 2009 at 04:47:34AM -0400, Chris Colbert wrote:
>   
>> Gael,
>>     
>
>   
>> That link is just full of internet trash, popups, and the like. I'm
>> glad i didnt click on it with a windows box.
>>     
>
> Yes, it was the first hit on google, but I should have checked it.
>
> The good link is:
>
> http://vefur.simula.no/~hpl/scripting/index.html
>
> Ga?l
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   


From aisaac at american.edu  Fri Jul 31 09:22:14 2009
From: aisaac at american.edu (Alan G Isaac)
Date: Fri, 31 Jul 2009 09:22:14 -0400
Subject: [SciPy-User] book on scipy
In-Reply-To: <4A72D47C.1060002@lpta.in2p3.fr>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>	<20090731080522.GB9207@phare.normalesup.org>	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>	<20090731084952.GC9207@phare.normalesup.org>
	<4A72D47C.1060002@lpta.in2p3.fr>
Message-ID: <4A72F006.5040206@american.edu>

On 7/31/2009 7:24 AM Johann Cohen-Tanugi apparently wrote:
> The download is blocked by the need to have a 
> printed edition at hand to check first words on (I presume) random 
> pages.....


It is not a free book.
It is available for $56 on Amazon and imo worth buying:
http://www.amazon.com/Python-Scripting-Computational-Science-Engineering/dp/3540739157/ref=sr_1_1?ie=UTF8&qid=1249046158&sr=8-1

But don't overlook the Cookbook:
http://www.scipy.org/Cookbook

Alan Isaac



From emmanuelle.gouillart at normalesup.org  Fri Jul 31 09:37:33 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Fri, 31 Jul 2009 15:37:33 +0200
Subject: [SciPy-User] book on scipy
In-Reply-To: <4A72F006.5040206@american.edu>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
	<20090731080522.GB9207@phare.normalesup.org>
	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>
	<20090731084952.GC9207@phare.normalesup.org>
	<4A72D47C.1060002@lpta.in2p3.fr> <4A72F006.5040206@american.edu>
Message-ID: <20090731133733.GA23447@phare.normalesup.org>

> But don't overlook the Cookbook:
> http://www.scipy.org/Cookbook

And if you want to read through a comprehensive and step-by-step
document, have a look at the scipy tutorial
http://docs.scipy.org/doc/scipy/reference/tutorial/ and the numpy
tutorial http://www.scipy.org/Tentative_NumPy_Tutorial 

Emmanuelle


From gokhansever at gmail.com  Fri Jul 31 09:39:25 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Fri, 31 Jul 2009 08:39:25 -0500
Subject: [SciPy-User] book on scipy
In-Reply-To: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
Message-ID: <49d6b3500907310639o6fcab573xd28c836f978d99d6@mail.gmail.com>

On Fri, Jul 31, 2009 at 3:00 AM, Bala subramanian <bala.biophysics at gmail.com
> wrote:

> Friends,
>
> Someone please suggest me some good books on scipy/numpy for a beginner.
>
> Thanks,
> Bala
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
Although not a book resource, I highly recommend you to look at these to
links:

https://cirl.berkeley.edu/fperez/py4science/starter_kit.html
Scientific Computing with Python <http://nanohub.org/resources/99>

The next two is extras :) Probably you will need them once you step into
PyScience world.

A Demonstration of the 'IPython' Interactive
Shell<http://video.google.com/videoplay?docid=2025411588241625522>
Matplotlib by John D. Hunter <http://videolectures.net/mloss08_hunter_mat/>

I have learnt great amount of information just by watching these lectures
and as well as get highly motivated to integrate my scientific research with
Python and its scientific extensions.

Good luck,

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090731/892868d2/attachment.html>

From ldl08 at gmx.net  Fri Jul 31 10:50:05 2009
From: ldl08 at gmx.net (David)
Date: Fri, 31 Jul 2009 22:50:05 +0800
Subject: [SciPy-User] book on scipy
In-Reply-To: <49d6b3500907310639o6fcab573xd28c836f978d99d6@mail.gmail.com>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
	<49d6b3500907310639o6fcab573xd28c836f978d99d6@mail.gmail.com>
Message-ID: <4A73049D.4070909@gmx.net>

Hello,

G?khan Sever wrote:
> Matplotlib by John D. Hunter
> <http://videolectures.net/mloss08_hunter_mat/>
Is there actually a video to be found there?  I, for my part, can't find
it for the life of me. I merely can download the slides.
Has the video maybe been taken down?

David


From jdh2358 at gmail.com  Fri Jul 31 10:59:41 2009
From: jdh2358 at gmail.com (John Hunter)
Date: Fri, 31 Jul 2009 09:59:41 -0500
Subject: [SciPy-User] book on scipy
In-Reply-To: <4A73049D.4070909@gmx.net>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
	<49d6b3500907310639o6fcab573xd28c836f978d99d6@mail.gmail.com>
	<4A73049D.4070909@gmx.net>
Message-ID: <88e473830907310759r56f5e9bp31b39635d8072d01@mail.gmail.com>

On Fri, Jul 31, 2009 at 9:50 AM, David<ldl08 at gmx.net> wrote:
> Hello,
>
> G?khan Sever wrote:
>> Matplotlib by John D. Hunter
>> <http://videolectures.net/mloss08_hunter_mat/>
> Is there actually a video to be found there? ?I, for my part, can't find
> it for the life of me. I merely can download the slides.
> Has the video maybe been taken down?

It looks like it has been taken down, as have other videos from the
same conference.  I emailed support at video lectures and asked them
if they could report the videos.

JDH


From josef.pktd at gmail.com  Fri Jul 31 12:01:02 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 31 Jul 2009 12:01:02 -0400
Subject: [SciPy-User] expectation function for stats.distributions
Message-ID: <1cd32cbb0907310901oe3e7ca4r74586845222acdbc@mail.gmail.com>

I just needed to verify an expectation for fixing of the stats.models code.

A while ago a wrote a function that can be attached as method to the
distributions classes. It's pretty simple, but comes in handy when an
expectation or conditional expectation has to be checked.

Is there an interest in adding this as a new method to the distributions?

explanations are here
http://jpktd.blogspot.com/2009/04/having-fun-with-expectations.html

copy of file below

Josef

'''
copy from webpage, where is the original?

Author jpktd

'''


import numpy as np
from scipy import stats, integrate

def expectedfunc(self, fn=None, args=(), lb=None, ub=None, conditional=False):
    '''calculate expected value of a function with respect to the distribution

    only for standard version of distribution,
    location and scale not tested

    Parameters
    ----------
        all parameters are keyword parameters
        fn : function (default: identity mapping)
           Function for which integral is calculated. Takes only one argument.
        args : tuple
           argument (parameters) of the distribution
        lb, ub : numbers
           lower and upper bound for integration, default is set to the support
           of the distribution
        conditional : boolean (False)
           If true then the integral is corrected by the conditional probability
           of the integration interval. The return value is the expectation
           of the function, conditional on being in the given interval.

    Returns
    -------
        expected value : float
    '''
    if fn is None:
        def fun(x, *args):
            return x*self.pdf(x, *args)
    else:
        def fun(x, *args):
            return fn(x)*self.pdf(x, *args)
    if lb is None:
        lb = self.a
    if ub is None:
        ub = self.b
    if conditional:
        invfac = self.sf(lb,*args) - self.sf(ub,*args)
    else:
        invfac = 1.0
    return integrate.quad(fun, lb, ub,
                                args=args)[0]/invfac


print expectedfunc(stats.norm, lambda(x): (x)**4)
print expectedfunc(stats.norm, lambda(x): min((x)**2,1.5**2))

#Jonathans version
from scipy.stats import norm as Gaussian
c = 1.5 # our "cutoff" point
c = 0.5 # try another value
tmp = 2 * Gaussian.cdf(c) - 1
gamma = tmp + c**2 * (1 - tmp) - 2 * c * Gaussian.pdf(c)
print gamma

print expectedfunc(stats.norm, lambda(x): min((x)**2,c**2))


From rex at nosyntax.net  Fri Jul 31 11:46:13 2009
From: rex at nosyntax.net (rex)
Date: Fri, 31 Jul 2009 08:46:13 -0700
Subject: [SciPy-User] book on scipy
In-Reply-To: <20090731133733.GA23447@phare.normalesup.org>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com>
	<20090731080522.GB9207@phare.normalesup.org>
	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com>
	<20090731084952.GC9207@phare.normalesup.org>
	<4A72D47C.1060002@lpta.in2p3.fr> <4A72F006.5040206@american.edu>
	<20090731133733.GA23447@phare.normalesup.org>
Message-ID: <20090731154612.GN10466@nosyntax.net>

Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org> [2009-07-31 06:37]:
>> But don't overlook the Cookbook:
>> http://www.scipy.org/Cookbook
>
>And if you want to read through a comprehensive and step-by-step
>document, have a look at the scipy tutorial
>http://docs.scipy.org/doc/scipy/reference/tutorial/ and the numpy
>tutorial http://www.scipy.org/Tentative_NumPy_Tutorial 

There's also:

http://docs.scipy.org/doc/numpy/reference/

Speaking of which, this example below does not work correctly with
matplotlib-0.98.1-1+lenny4 (the most recent in the Debian repository).

http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.lognormal.html?highlight=histogram

mu, sigma = 0.05, 0.25 # mean and standard deviation 
s = np.random.lognormal(mu, sigma, 100000)

import matplotlib.pyplot as plt
count, bins, ignored = plt.hist(s, 100, normed=True, align='center')
x = np.linspace(min(bins), max(bins), 10000)
pdf = (np.exp(-(np.log(x) - mu)**2 / (2 * sigma**2))/(x * sigma * np.sqrt(2 * np.pi)))
plt.plot(x, pdf, linewidth=2, color='r')
plt.axis('tight')
plt.show()


It's visually apparent that the histogram is offset to the left
from where it should be.

>>help(plt.hist) reveals that:

*align*: ['left' | 'mid' | 'right' ]

With the 'mid' keyword, the plot is correctly centered on the pdf.

'left, center, right' is certainly easier to remember than 'left',
'mid',  'right'.  

In any case, shouldn't it throw an error rather than silently
defaulting to 'left' when the 'align' keyword isn't one of the magic 3?

-rex


From jgomezdans at gmail.com  Fri Jul 31 12:59:51 2009
From: jgomezdans at gmail.com (Jose Gomez-Dans)
Date: Fri, 31 Jul 2009 17:59:51 +0100
Subject: [SciPy-User] Filling a function with values
Message-ID: <91d218430907310959q5590b5d7kd29a97689a27a5e@mail.gmail.com>

Hi,
This is a really quick and stupid question. Let's say I have a function and
I want to calculate its value for a set of points. What's the best way of
quickly doing this avoiding loops?

def MyFunction ( p1, p2, p3, t, y)
  return *some_value*

t = numpy.array ( ..... )
y = numpy.array ( ....)
a1,a2,a3 = numpy.mgrid [ 0:100, 0:1:100j, 0:100 ]

evaluated_function = MyFunction (a1, a2, a3, t, y)

evaluated_function.shape would be (100,100,100), and evaluated_function [m,
n, k] should be equal to
MyFunction ( a1[m], a2[n], a3[k], t, y )

I think mgrid (ogrid) are the right tools, but I don't really understand how
I can get it to do what I want. I have read <
http://docs.scipy.org/doc/scipy/reference/tutorial/basic.html#id5>, but that
doesn't clear things up. I guess that what I want to do is just mentioned at
the end of this Section of the tutorial, but can anyone maybe give an
example?

many thanks!
J
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090731/531d7a0f/attachment.html>

From jh at physics.ucf.edu  Fri Jul 31 13:06:37 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Fri, 31 Jul 2009 13:06:37 -0400
Subject: [SciPy-User] SciPy Foundation
Message-ID: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>

About sixteen months ago, I launched the SciPy Documentation Project
and its Marathon.  Dozens pitched in and now numpy docs are rapidly
approaching a professional level.  The "pink wave" ("Needs Review"
status) is at 56% today!  There is consensus among doc writers that
much of the rest can be labeled in the "unimportant" category, so
we're close to starting the review push (hold your fire, there is a
web site mod to be done first).

We're also nearing the end of the summer, and it's time to look ahead.
The path for docs is clear, but the path for SciPy is not.  I think
our weakest area right now is organization of the project.  There is
no consensus-based plan for improvement of the whole toward a stated
goal, no centralized coordination of work, and no funded work focused
on many of our weaknesses, notwithstanding my doc effort and what
Enthought does for code.

I define success as popular adoption in preference to commercial
packages.  I believe in vote-with-your-feet: this goal will not be
reached until all aspects of the package and its presentation to the
world exceed those of our commercial competition.  Scipy is now a
grass roots effort, but that takes it only so far.  Other projects,
such as OpenOffice and Sage, don't follow this model and do produce
quality products that compete with commercial offerings, at least on
open-source platforms.  Before we can even hope for that, we have to
do the following:

- Docs
  - Rest of numpy reference pages reviewed and proofed or marked unimportant
  - Scipy reference pages
  - User manual for the whole toolstack
  - Multiple commercial books
- Packaging
  - Personal Package Archive or equivalent for every release of every
    OS for the full toolstack (There are tools that do this but we
    don't use them.  NSF requires Metronome - http://nmi.cs.wisc.edu/
    - for funding most development grants, so right now we're not even
    on NSF's radar.)
  - Track record of having the whole toolstack installation "just
    work" in a few command lines or clicks for *everyone*
  - Regular, scheduled releases of numpy and scipy
  - Coordinated releases of numpy, scipy, and stable scikits into PPA system
- Public communication
  - A real marketing plan
  - Executing on that plan
  - Web site geared toward multiple audiences, run by experts at that
    kind of communication
  - More webinars, conference booths, training, aimed at all levels
  - Demos, testimonials, topical forums, all showcased
- Code
  - A full design review for numpy 2.0
    - No more inconsistencies like median(), lacking "out", degrees
      option for angle functions?
    - Trimming of financial functions, maybe others, from numpy?
    - Package structure review (eliminate "fromnumeric"?)
    - Goal that this be the last breakage for numpy API (the real 1.0)
  - Scipy
    - Is it maintainable?  should it be broken up?
    - Clear code addition path (or decide never to add more)
    - Docs (see above)  
  - Add-on packages
    - Both existence of and good indexing/integration/support for
      field-specific packages
    - Clearer development path for new packages
    - Central hosting system for packages (svn, mailing lists, web,
      build integration, etc.)
    - Simultaneous releases of stable packages along with numpy/scipy

I posted a basic improvement plan some years back.  The core ideas
have not changed; it is linked from the bottom of
http://scipy.org/Developer_Zone.  I chose our major weakness to begin
with and started the doc project, using some money I could justify
spending simply for the utility of docs for my own research.  I funded
the work of two doc coordinators, one each this summer and last.
Looking at http://docs.scipy.org/numpy/stats/, you can see that when a
doc coordinator was being paid (summers), work got done.  When not,
then not.  Without publicly announcing what these guys made, I'll be
the first to admit that it wasn't a lot.  Yet, those small sums bought
a huge contribution to numpy through the work of several dozen
volunteers and the major contributions of a few.

My conclusion is that active and constant coordination is central to
motivating volunteer work, and that without a salary we cannot depend
on coordination remaining active.  On the other hand, I have heard
Enthought's leaders bemoan the high cost of devoting employee time to
this project, and the low returns available from selling support to
universities and non-profit research institutes.  Their leadership has
moved us forward, particularly in the area of code, but has not
provided the momentum necessary to carry us forward on all fronts.  It
is time for the public and education sectors to kick in some resources
and organizational leadership.  We are, after all, benefitting
immensely.

Since the cost of employee time is not so high for us in the public
and education sectors, I propose to continue hiring people like Stefan
and David as UCF employees or contractors, and to expand to hiring
others in areas like packaging and marketing, provided that funding
for those hires can be found.  However, my grant situation is no
longer as rich as it has been the past two years, and the needs going
forward are greater than in the past if we're now to tackle all the
points above.  So, I will not be hiring another doc guru from my
research grants next year.

I am confident that others are willing to pitch in financially, but
few will pitch in a full FTE, and we need several.  We can (and will)
set up a donations site, but donation sites tend to receive pizza
money unless a sugar daddy comes along.  Those benefitting most from
the software, notably education, non-profit research, and government
institutions, are *forbidden* from making donations by the terms of
their grants.  NSF doesn't give you money so you can give it away.

We need to provide services they can buy on subcontract and a means
for handling payments from them.  Selling support does not solve the
problem, as that requires spending most of the income on servicing
that particular client.  Rather, we need to sell a chunk of
documentation or the packaging of a particular release, and then
provide the product not just to that client but to everyone.

We can also propose directly for federal and corporate grant funds.  I
have spoken with several NASA and NSF program managers and with
Google's Federal Accounts Representative, and the possibilities for
funding are good.  But, I am not going to do this alone.  We need a
strong proposal team to be credible.

So, I am seeking a group that is willing to work with me to put up the
infrastructure of a funded project, to write grant proposals, and to
coordinate a financial effort.  Members of this group must have a
track record of funded grants, business success, foundation support,
etc.  We might call it the SciPy Foundation.  It could be based at
UCF, which has a low overhead rate and has infrastructure (like an HR
staff), or it might be independent if we can find a good director
willing to devote significant time for relatively low pay compared to
what they can likely make elsewhere.  I would envision hiring
permanent coordinators for docs, packaging, and marketing
communications.  Enthought appears to have code covered by virtue of
having hired Travis, Robert, etc.; how to integrate that with this
effort is an open question but not a difficult one, I think, as code
is our strongest asset at this point.

I invite discussion of this approach and the task list above on the
scipy-dev at scipy.org mailing list.  If you are seeing this post
elsewhere, please reply only on scipy-dev at scipy.org.

If you are eligible to lead funding proposals and are interested in
participating in grant writing and management activities related to
work in our weak areas, please contact me directly.

Thanks,

--jh--
Prof. Joseph Harrington
Planetary Sciences Group
Department of Physics
MAP 414
4000 Central Florida Blvd.
University of Central Florida
Orlando, FL 32816-2385
jh at physics.ucf.edu
planets.ucf.edu


From wing1127aishi at gmail.com  Fri Jul 31 14:55:01 2009
From: wing1127aishi at gmail.com (Leon Sit)
Date: Fri, 31 Jul 2009 14:55:01 -0400
Subject: [SciPy-User] How do I add a data point into scikits.timeseries
Message-ID: <3af1998d0907311155w20c1fdb9o799664ae16134148@mail.gmail.com>

Hi everyone:

I am trying to append a tuple of data, (datetime, data), into a
scikits.timeserieses. I do not know the size in advance so it must be
appended. Is there a way to do it efficiently as this will be a primary
operation that in done in order of million times.

Thanks,

Leon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090731/568e18be/attachment.html>

From vanforeest at gmail.com  Fri Jul 31 15:06:18 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Fri, 31 Jul 2009 21:06:18 +0200
Subject: [SciPy-User] Solve an arbitrary function
In-Reply-To: <20090731080332.GA9207@phare.normalesup.org>
References: <a850cb31-9d79-41f3-8c26-e21164e37fb9@d32g2000yqh.googlegroups.com>
	<3d375d730907302117n78a97481w80f9742233ffdba9@mail.gmail.com>
	<fa510ff80907310019u1075e274uf128844999db1c79@mail.gmail.com>
	<20090731080332.GA9207@phare.normalesup.org>
Message-ID: <fa510ff80907311206o3d13f42ap19a7dafaab256bb1@mail.gmail.com>

2009/7/31 Gael Varoquaux <gael.varoquaux at normalesup.org>:
> On Fri, Jul 31, 2009 at 09:19:21AM +0200, nicky van foreest wrote:
>> What would be the best place to document examples such as Robert's
>> below? I am afraid such examples get burried in the mailing list while
>> they are great examples of how to use fsolve, for instance.
>
> That's a tough question. You can of course edit the Scipy Cookbook
> (www.scipy.org/Cookbook), but the problem with this approach is that it
> grows messy and may be hard to find.

I completely agree. On the other hand, having to search through the
mailing list is messier still. Adding such examples to the
documentation of a function is perhaps also not the best way. Often
people are searching for a function that is unknown to them, so they
will not find the relevant documentation. A cookbook would then be a
natural place to start looking. I, for instance, find the python
cookbook by Martelli very useful.

Hence, I decided to add this particular  example to the cookbook.

Nicky


From vanforeest at gmail.com  Fri Jul 31 15:10:46 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Fri, 31 Jul 2009 21:10:46 +0200
Subject: [SciPy-User] expectation function for stats.distributions
In-Reply-To: <1cd32cbb0907310901oe3e7ca4r74586845222acdbc@mail.gmail.com>
References: <1cd32cbb0907310901oe3e7ca4r74586845222acdbc@mail.gmail.com>
Message-ID: <fa510ff80907311210k7f0cd0a9y798e1bbeeeba471e@mail.gmail.com>

Hi Josef,

I would like such a function. As a matter of fact, just today I needed
the expection of min(X,k), where X is a Poisson RV and k a number. Of
course, this is not particularly difficult, but a function that does
the work for me is better yet. BTW: will integrate.quad also work for
discrete distributions?

Nicky

2009/7/31  <josef.pktd at gmail.com>:
> I just needed to verify an expectation for fixing of the stats.models code.
>
> A while ago a wrote a function that can be attached as method to the
> distributions classes. It's pretty simple, but comes in handy when an
> expectation or conditional expectation has to be checked.
>
> Is there an interest in adding this as a new method to the distributions?
>
> explanations are here
> http://jpktd.blogspot.com/2009/04/having-fun-with-expectations.html
>
> copy of file below
>
> Josef
>
> '''
> copy from webpage, where is the original?
>
> Author jpktd
>
> '''
>
>
> import numpy as np
> from scipy import stats, integrate
>
> def expectedfunc(self, fn=None, args=(), lb=None, ub=None, conditional=False):
> ? ?'''calculate expected value of a function with respect to the distribution
>
> ? ?only for standard version of distribution,
> ? ?location and scale not tested
>
> ? ?Parameters
> ? ?----------
> ? ? ? ?all parameters are keyword parameters
> ? ? ? ?fn : function (default: identity mapping)
> ? ? ? ? ? Function for which integral is calculated. Takes only one argument.
> ? ? ? ?args : tuple
> ? ? ? ? ? argument (parameters) of the distribution
> ? ? ? ?lb, ub : numbers
> ? ? ? ? ? lower and upper bound for integration, default is set to the support
> ? ? ? ? ? of the distribution
> ? ? ? ?conditional : boolean (False)
> ? ? ? ? ? If true then the integral is corrected by the conditional probability
> ? ? ? ? ? of the integration interval. The return value is the expectation
> ? ? ? ? ? of the function, conditional on being in the given interval.
>
> ? ?Returns
> ? ?-------
> ? ? ? ?expected value : float
> ? ?'''
> ? ?if fn is None:
> ? ? ? ?def fun(x, *args):
> ? ? ? ? ? ?return x*self.pdf(x, *args)
> ? ?else:
> ? ? ? ?def fun(x, *args):
> ? ? ? ? ? ?return fn(x)*self.pdf(x, *args)
> ? ?if lb is None:
> ? ? ? ?lb = self.a
> ? ?if ub is None:
> ? ? ? ?ub = self.b
> ? ?if conditional:
> ? ? ? ?invfac = self.sf(lb,*args) - self.sf(ub,*args)
> ? ?else:
> ? ? ? ?invfac = 1.0
> ? ?return integrate.quad(fun, lb, ub,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?args=args)[0]/invfac
>
>
> print expectedfunc(stats.norm, lambda(x): (x)**4)
> print expectedfunc(stats.norm, lambda(x): min((x)**2,1.5**2))
>
> #Jonathans version
> from scipy.stats import norm as Gaussian
> c = 1.5 # our "cutoff" point
> c = 0.5 # try another value
> tmp = 2 * Gaussian.cdf(c) - 1
> gamma = tmp + c**2 * (1 - tmp) - 2 * c * Gaussian.pdf(c)
> print gamma
>
> print expectedfunc(stats.norm, lambda(x): min((x)**2,c**2))
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From robert.kern at gmail.com  Fri Jul 31 15:27:14 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 31 Jul 2009 14:27:14 -0500
Subject: [SciPy-User] SciPy Foundation
In-Reply-To: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
Message-ID: <3d375d730907311227l52282e91mb244f00f4e6eab7a@mail.gmail.com>

On Fri, Jul 31, 2009 at 12:06, Joe Harrington<jh at physics.ucf.edu> wrote:
>?Enthought appears to have code covered by virtue of
> having hired Travis, Robert, etc.;

Eh, what? We work on numpy and scipy in our spare time, just like
everyone else. There are rare occasions when a client wants to fund a
particular feature, or we need to fix a bug in the course of our work,
but that's a far cry from having "code covered".

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From josef.pktd at gmail.com  Fri Jul 31 16:10:07 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 31 Jul 2009 16:10:07 -0400
Subject: [SciPy-User] expectation function for stats.distributions
In-Reply-To: <fa510ff80907311210k7f0cd0a9y798e1bbeeeba471e@mail.gmail.com>
References: <1cd32cbb0907310901oe3e7ca4r74586845222acdbc@mail.gmail.com>
	<fa510ff80907311210k7f0cd0a9y798e1bbeeeba471e@mail.gmail.com>
Message-ID: <1cd32cbb0907311310g59c4f2a5uf9f328acba0bad07@mail.gmail.com>

On Fri, Jul 31, 2009 at 3:10 PM, nicky van foreest<vanforeest at gmail.com> wrote:
> Hi Josef,
>
> I would like such a function. As a matter of fact, just today I needed
> the expection of min(X,k), where X is a Poisson RV and k a number. Of
> course, this is not particularly difficult, but a function that does
> the work for me is better yet. BTW: will integrate.quad also work for
> discrete distributions?

Thanks for the interest.

No, integrate.quad only work for continuous random variables. But
there are the corresponding generic functions also for the discrete
distributions in stats.distribution.

An expectation function could be based on the generic calculation of
the uncentered moments of the discrete distributions. I fixed the
functions last year, but haven't looked at them in a while. I think
the main point was to decide which and how many points to use in the
calculation, otherwise it is just a vector inner product.


To the name: I'm up for ideas
I don't like integral so much because it is not obvious that it
calculates an expectation and not just the integral of the pdf, as in
stats.kde and my wrapper for the multivariate normal cdf.
Additionally, my function also calculates conditional expectations,
which are not just integrals.

I like expectation, but it is a noun and might refer to the
expectation of the random variable itself, i.e. the mean. That's why I
ended up with a variation on expectation_function.

Josef


>
> Nicky
>
> 2009/7/31 ?<josef.pktd at gmail.com>:
>> I just needed to verify an expectation for fixing of the stats.models code.
>>
>> A while ago a wrote a function that can be attached as method to the
>> distributions classes. It's pretty simple, but comes in handy when an
>> expectation or conditional expectation has to be checked.
>>
>> Is there an interest in adding this as a new method to the distributions?
>>
>> explanations are here
>> http://jpktd.blogspot.com/2009/04/having-fun-with-expectations.html
>>
>> copy of file below
>>
>> Josef
>>
>> '''
>> copy from webpage, where is the original?
>>
>> Author jpktd
>>
>> '''
>>
>>
>> import numpy as np
>> from scipy import stats, integrate
>>
>> def expectedfunc(self, fn=None, args=(), lb=None, ub=None, conditional=False):
>> ? ?'''calculate expected value of a function with respect to the distribution
>>
>> ? ?only for standard version of distribution,
>> ? ?location and scale not tested
>>
>> ? ?Parameters
>> ? ?----------
>> ? ? ? ?all parameters are keyword parameters
>> ? ? ? ?fn : function (default: identity mapping)
>> ? ? ? ? ? Function for which integral is calculated. Takes only one argument.
>> ? ? ? ?args : tuple
>> ? ? ? ? ? argument (parameters) of the distribution
>> ? ? ? ?lb, ub : numbers
>> ? ? ? ? ? lower and upper bound for integration, default is set to the support
>> ? ? ? ? ? of the distribution
>> ? ? ? ?conditional : boolean (False)
>> ? ? ? ? ? If true then the integral is corrected by the conditional probability
>> ? ? ? ? ? of the integration interval. The return value is the expectation
>> ? ? ? ? ? of the function, conditional on being in the given interval.
>>
>> ? ?Returns
>> ? ?-------
>> ? ? ? ?expected value : float
>> ? ?'''
>> ? ?if fn is None:
>> ? ? ? ?def fun(x, *args):
>> ? ? ? ? ? ?return x*self.pdf(x, *args)
>> ? ?else:
>> ? ? ? ?def fun(x, *args):
>> ? ? ? ? ? ?return fn(x)*self.pdf(x, *args)
>> ? ?if lb is None:
>> ? ? ? ?lb = self.a
>> ? ?if ub is None:
>> ? ? ? ?ub = self.b
>> ? ?if conditional:
>> ? ? ? ?invfac = self.sf(lb,*args) - self.sf(ub,*args)
>> ? ?else:
>> ? ? ? ?invfac = 1.0
>> ? ?return integrate.quad(fun, lb, ub,
>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?args=args)[0]/invfac
>>
>>
>> print expectedfunc(stats.norm, lambda(x): (x)**4)
>> print expectedfunc(stats.norm, lambda(x): min((x)**2,1.5**2))
>>
>> #Jonathans version
>> from scipy.stats import norm as Gaussian
>> c = 1.5 # our "cutoff" point
>> c = 0.5 # try another value
>> tmp = 2 * Gaussian.cdf(c) - 1
>> gamma = tmp + c**2 * (1 - tmp) - 2 * c * Gaussian.pdf(c)
>> print gamma
>>
>> print expectedfunc(stats.norm, lambda(x): min((x)**2,c**2))
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From pgmdevlist at gmail.com  Fri Jul 31 16:49:33 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Fri, 31 Jul 2009 16:49:33 -0400
Subject: [SciPy-User] How do I add a data point into scikits.timeseries
In-Reply-To: <3af1998d0907311155w20c1fdb9o799664ae16134148@mail.gmail.com>
References: <3af1998d0907311155w20c1fdb9o799664ae16134148@mail.gmail.com>
Message-ID: <F148F11E-7361-4231-AB6C-829ED040945C@gmail.com>


On Jul 31, 2009, at 2:55 PM, Leon Sit wrote:

> Hi everyone:
>
> I am trying to append a tuple of data, (datetime, data), into a  
> scikits.timeserieses. I do not know the size in advance so it must  
> be appended. Is there a way to do it efficiently as this will be a  
> primary operation that in done in order of million times.


Just like with standard arrays, appending to a TimeSeries is not very  
efficient. Instead, you may want to consider creating a list of tuples  
and appending as it goes. Then, create a TimeSeries out of it at the  
very end.

 >>>  test=[(dt.datetime(2009,01,d),f) for (d,f) in  
zip(range(1,16),np.random.rand(15))]
 >>> # Make a tmp array
 >>>  tmp=np.array(test,dtype=[('dates',object),('f',float)])
 >>> ts.time_series(tmp['f'],dates=tmp['dates'])

Note that you'd have to define a frequency or you'd end up with an  
'undefined' frequency...




From jh at physics.ucf.edu  Fri Jul 31 17:04:46 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Fri, 31 Jul 2009 17:04:46 -0400
Subject: [SciPy-User] SciPy Foundation
In-Reply-To: <3d375d730907311227l52282e91mb244f00f4e6eab7a@mail.gmail.com>
	(message from Robert Kern on Fri, 31 Jul 2009 14:27:14 -0500)
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<3d375d730907311227l52282e91mb244f00f4e6eab7a@mail.gmail.com>
Message-ID: <wl8ws5o4l8x.fsf@glup.physics.ucf.edu>

Robert wrote:
> On Fri, Jul 31, 2009 at 12:06, Joe Harrington<jh at physics.ucf.edu> wrote:
> >?Enthought appears to have code covered by virtue of
> > having hired Travis, Robert, etc.;

> Eh, what? We work on numpy and scipy in our spare time, just like
> everyone else. There are rare occasions when a client wants to fund a
> particular feature, or we need to fix a bug in the course of our work,
> but that's a far cry from having "code covered".

Then please accept my profusest apologies!  Eric mentioned to me that
Enthough had paid significantly for scipy development and I thought
that meant a portion of developers' time.  Perhaps this was just in
the past.

--jh--


From robert.kern at gmail.com  Fri Jul 31 17:13:48 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 31 Jul 2009 16:13:48 -0500
Subject: [SciPy-User] SciPy Foundation
In-Reply-To: <wl8ws5o4l8x.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<3d375d730907311227l52282e91mb244f00f4e6eab7a@mail.gmail.com> 
	<wl8ws5o4l8x.fsf@glup.physics.ucf.edu>
Message-ID: <3d375d730907311413s3bdf7021n4a3056b3edc989b@mail.gmail.com>

On Fri, Jul 31, 2009 at 16:04, Joe Harrington<jh at physics.ucf.edu> wrote:
> Robert wrote:
>> On Fri, Jul 31, 2009 at 12:06, Joe Harrington<jh at physics.ucf.edu> wrote:
>> >?Enthought appears to have code covered by virtue of
>> > having hired Travis, Robert, etc.;
>
>> Eh, what? We work on numpy and scipy in our spare time, just like
>> everyone else. There are rare occasions when a client wants to fund a
>> particular feature, or we need to fix a bug in the course of our work,
>> but that's a far cry from having "code covered".
>
> Then please accept my profusest apologies! ?Eric mentioned to me that
> Enthough had paid significantly for scipy development and I thought
> that meant a portion of developers' time. ?Perhaps this was just in
> the past.

Still do; it's just not part of our daily duties and is usually
focused on what we need, not general maintenance. Not to mention the
infrastructure support.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From stefan at sun.ac.za  Fri Jul 31 17:48:01 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 31 Jul 2009 23:48:01 +0200
Subject: [SciPy-User] book on scipy
In-Reply-To: <20090731154612.GN10466@nosyntax.net>
References: <288df32a0907310100p354d5e81hafd366618a219870@mail.gmail.com> 
	<20090731080522.GB9207@phare.normalesup.org>
	<7f014ea60907310147k780dbc43xa6acd045ed6cae19@mail.gmail.com> 
	<20090731084952.GC9207@phare.normalesup.org>
	<4A72D47C.1060002@lpta.in2p3.fr> <4A72F006.5040206@american.edu>
	<20090731133733.GA23447@phare.normalesup.org> 
	<20090731154612.GN10466@nosyntax.net>
Message-ID: <9457e7c80907311448med24475leebdfccef15e8f9c@mail.gmail.com>

Hi Rex

2009/7/31 rex <rex at nosyntax.net>:
> Speaking of which, this example below does not work correctly with
> matplotlib-0.98.1-1+lenny4 (the most recent in the Debian repository).

I'm glad you spotted this problem.  Would you mind creating an account
on docs.scipy.org and fixing it?

Thanks!
St?fan


From eadrogue at gmx.net  Mon Jul 27 15:07:27 2009
From: eadrogue at gmx.net (Ernest =?iso-8859-1?Q?Adrogu=E9?=)
Date: Mon, 27 Jul 2009 21:07:27 +0200
Subject: [SciPy-User] [SciPy-user] optimization using fmin_bfgs with
	gradient	information
In-Reply-To: <ec9f80fa0907230142q2b8c103co497ecca8ad9255a5@mail.gmail.com>
References: <20090718123544.GA3122@doriath.local>
	<ec9f80fa0907180936p2e0c2ffdr57bb2f15f8841f01@mail.gmail.com>
	<20090719110543.GA7138@doriath.local>
	<ec9f80fa0907190841s75f9088dxfecb1b284f614169@mail.gmail.com>
	<20090719183923.GA21105@doriath.local>
	<ec9f80fa0907200143j6a6748dx62d605991341d74d@mail.gmail.com>
	<20090722163332.GA3515@doriath.local>
	<ec9f80fa0907230142q2b8c103co497ecca8ad9255a5@mail.gmail.com>
Message-ID: <20090727190727.GA32763@doriath.local>

23/07/09 @ 10:42 (+0200), thus spake Sebastian Walter:
> 1)
> approx_fprime uses probably finite differences to approximate the
> gradient, so at x = 2
> it computes  (f(2+epsilon) - f(2))/epsilon =  (2+epsilon)**2/epsilon
> which can be very large
> 
> 2) When using AD to evaluate derivatives, only one path through the
> control flow graph (which is defined by the if statement in your
> function) is taken.
> I.e. if x<2, AD will not know that for x>=2 it would have taken
> another way through the control flow graph of your algorithm.
> 
> I.e. it  would compute
> f'(x<2) = 0 and f'(x>=2) = x**2  without realizing that the function
> is nondifferentiable at that point.
> 
> 3)
> If you have such an objective function as you have shown above, then
> your optimization problem is not well-formed because it does not have
> a real minimizer x_*.
> I'd assume that your objective function looks more like
> f(x) = abs(x)
> which is non-differentiable at x=0.
> This kind of objective function destroys the convergence properties of
> algorithms that assume continuously differentiable objective
> functions, i.e. the convergence to the minimizer can be very slow.
> For such problems, special purpose algorithms, e.g.  the so-called
> "bundle methods" can be used which converge superlinearly, as far as I
> know.

Nice explanation. I have made some tests with a simplified
model, and have found fmin_slsqp to be the fastest method by far.
It takes around 1 minute and a half to estimate a model with 128
parameters.

p	time
16	0.36s
32	4.56s
64	32.74s
128	3.66min

Second comes fsolve, but its performance decreases rapidly as
the number of parameters increases.

So I probably will stick to slsqp, unless I find a better solution.
These "bundle methods" sound interesting. I have found an
interesting paper with one such algorithm described in detail,
which might come handy if I finally decide to try it out. In case
someone is interested, is here:
http://www.mathematik.uni-trier.de/~huebner/software/bundle_method_documentation.pdf

Ernest



From Dharhas.Pothina at twdb.state.tx.us  Mon Jul 27 16:24:19 2009
From: Dharhas.Pothina at twdb.state.tx.us (Dharhas Pothina)
Date: Mon, 27 Jul 2009 15:24:19 -0500
Subject: [SciPy-User] Ubuntu vs Fedora for scientific work?
Message-ID: <4A6DC6A2.63BA.009B.0@twdb.state.tx.us>

Hi All,

This is slightly off topic but I felt that this lists membership would have good input on this question. I am presently running fedora 8 on my main workstation and it is getting a bit long in the tooth. Back when I set this machine up, RHEL was too much of a pain to use because scientific packages were always extremely outdated and difficult to install. Ubuntu was nice to use at home but installing scientific packages like the Intel Fortran compiler etc was complicated (not impossible, just more work than I wanted). So I went with Fedora which has worked pretty well so far. 

Recently, I've noticed that a lot of scientific packages now have ubuntu repositories and even the intel compiler has an ubuntu option. So I'm trying to decide whether to go with Fedora 11 or Ubuntu Jaunty. 

I'm not trying to start a flame war but I'm interested in what people's experience has been. 

thanks,

- dharhas



From josef.pktd at gmail.com  Mon Jul 27 16:45:37 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 27 Jul 2009 16:45:37 -0400
Subject: [SciPy-User] Scipy and statistics: probability density function
In-Reply-To: <3d375d730907270859w42327684gb726549eea094147@mail.gmail.com>
References: <C1C4F27B-636C-4A00-A169-A50440DEF3E4@imperial.ac.uk>
	<3d375d730907270859w42327684gb726549eea094147@mail.gmail.com>
Message-ID: <1cd32cbb0907271345v7ce27a5frfd0513ce3a62c20a@mail.gmail.com>

On Mon, Jul 27, 2009 at 11:59 AM, Robert Kern<robert.kern at gmail.com> wrote:
> On Mon, Jul 27, 2009 at 02:33, Daniel J
> Farrell<daniel.farrell at imperial.ac.uk> wrote:
>> Dear list,
>>
>> I am looking for some of the functionality provided by the GNU
>> Scientific Library histogram module (http://www.gnu.org/software/gsl/manual/html_node/Histograms.html
>> ).
>>
>> In particular, a need to be able to create a probability density
>> function from my histogram of data. This will allow inverse look-ups
>> to be performed, i.e. for a random number (0-->1) find the associated
>> probability (http://www.gnu.org/software/gsl/manual/html_node/The-histogram-probability-distribution-struct.html
>> ). This allows the distribution and for samples to be returned
>> weighted by the probability of the distribution -- which is a common
>> task!
>
> It looks like you want a CDF (or rather it's inverse, the PPF) rather
> than a PDF. Anyways, this is straightforward. Compute the histogram
> using normed=False. Find the cumulative sum and divide by the sum to
> get the (smoothed) empirical CDF. Prepend a 0.0 to this, and then this
> will align with the edges array that is also returned. Then you can
> use linear interpolation to do lookups. If you use the edges array as
> "X" and the empirical CDF as "Y", then this is a CDF. If you use the
> empircal CDF array as "X" and the edges as "Y", then this is a PPF.
>
>
> In [12]: x = np.random.uniform(0, 10, size=1000)
>
> In [13]: hist, edges = np.histogram(x)
>
> In [15]: hist
> Out[15]: array([100, 101, 104, 108, ?80, 111, ?96, ?88, 108, 104])
>
> In [16]: edges
> Out[16]:
> array([ ?3.53879571e-04, ? 1.00026423e+00, ? 2.00017458e+00,
> ? ? ? ? 3.00008493e+00, ? 3.99999528e+00, ? 4.99990563e+00,
> ? ? ? ? 5.99981598e+00, ? 6.99972633e+00, ? 7.99963668e+00,
> ? ? ? ? 8.99954704e+00, ? 9.99945739e+00])
>
> In [18]: ecdf = np.hstack([0.0, hist.cumsum() / float(hist.sum())])
>
> In [19]: ecdf
> Out[19]:
> array([ 0. ? , ?0.1 ?, ?0.201, ?0.305, ?0.413, ?0.493, ?0.604, ?0.7 ?,
> ? ? ? ?0.788, ?0.896, ?1. ? ])
>
> In [20]: np.interp(np.linspace(0, 10), edges, ecdf)
> Out[20]:
> array([ 0. ? ? ? ?, ?0.0203746 , ?0.04078459, ?0.06119459, ?0.08160458,
> ? ? ? ?0.10203472, ?0.12264881, ?0.14326291, ?0.163877 ?, ?0.18449109,
> ? ? ? ?0.20522712, ?0.22645351, ?0.24767991, ?0.2689063 , ?0.29013269,
> ? ? ? ?0.31160366, ?0.33364646, ?0.35568925, ?0.37773204, ?0.39977483,
> ? ? ? ?0.41953158, ?0.43585957, ?0.45218756, ?0.46851556, ?0.48484355,
> ? ? ? ?0.50433802, ?0.52699311, ?0.54964821, ?0.5723033 , ?0.59495839,
> ? ? ? ?0.61577382, ?0.63536742, ?0.65496101, ?0.6745546 , ?0.6941482 ,
> ? ? ? ?0.71259664, ?0.73055743, ?0.74851823, ?0.76647902, ?0.78443982,
> ? ? ? ?0.80567348, ?0.82771627, ?0.84975906, ?0.87180185, ?0.89384465,
> ? ? ? ?0.91515087, ?0.93637726, ?0.95760365, ?0.97883004, ?1. ? ? ? ?])
>
> In [21]: np.interp(np.linspace(0, 1), ecdf, edges)
> Out[21]:
> array([ ?3.53879571e-04, ? 2.04417216e-01, ? 4.08480553e-01,
> ? ? ? ? 6.12543890e-01, ? 8.16607227e-01, ? 1.02046852e+00,
> ? ? ? ? 1.22251143e+00, ? 1.42455434e+00, ? 1.62659724e+00,
> ? ? ? ? 1.82864015e+00, ? 2.02980301e+00, ? 2.22601775e+00,
> ? ? ? ? 2.42223250e+00, ? 2.61844725e+00, ? 2.81466200e+00,
> ? ? ? ? 3.01047705e+00, ? 3.19942458e+00, ? 3.38837211e+00,
> ? ? ? ? 3.57731965e+00, ? 3.76626718e+00, ? 3.95521472e+00,
> ? ? ? ? 4.19462069e+00, ? 4.44969986e+00, ? 4.70477903e+00,
> ? ? ? ? 4.95985820e+00, ? 5.15488346e+00, ? 5.33872431e+00,
> ? ? ? ? 5.52256515e+00, ? 5.70640600e+00, ? 5.89024684e+00,
> ? ? ? ? 6.08569264e+00, ? 6.29825861e+00, ? 6.51082459e+00,
> ? ? ? ? 6.72339057e+00, ? 6.93595654e+00, ? 7.16204944e+00,
> ? ? ? ? 7.39393960e+00, ? 7.62582975e+00, ? 7.85771991e+00,
> ? ? ? ? 8.07294833e+00, ? 8.26189586e+00, ? 8.45084340e+00,
> ? ? ? ? 8.63979093e+00, ? 8.82873846e+00, ? 9.01838365e+00,
> ? ? ? ? 9.21459840e+00, ? 9.41081315e+00, ? 9.60702789e+00,
> ? ? ? ? 9.80324264e+00, ? 9.99945739e+00])
>


This, together with the answer last week to a piecewise constant pdf,
would make most of the elements for a distribution class, similar to
the generic discrete distribution only for the continuous case.

It could take as constructor either a data array (and use histogram as
in this example, with theoretically optimal bin size) or a set of
points and pdf values as in last weeks example.

Does quad work correctly for integration of a piecewise function?
If yes, then we could just use the generic methods by subclassing rv_continuous.

Is it worth collecting these results?

Josef


> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


From s.mientki at ru.nl  Mon Jul 27 18:29:07 2009
From: s.mientki at ru.nl (Stef Mientki)
Date: Tue, 28 Jul 2009 00:29:07 +0200
Subject: [SciPy-User] optical ray tracing
In-Reply-To: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
References: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
Message-ID: <4A6E2A33.6030202@ru.nl>



Elliot Hallmark wrote:
> howdy,
>
> I'm looking to do some raytracing for a nonimaging design and there
> are no opensource packages that address the geometry needed.  Scipy
> seems like a good package to become familiar with anyway (I miss
> mathematica now that i'm no longer a university student), but i was
> wondering if anyone has already done some optical ray tracing work in
> scipy that i could build off of.
>
> Wouldn't be to hard to do from scratch, but my focus is on the design
> and the fewer excursions into building new tools the better.
>
>   
This might be interesting
  http://www-ee.eng.hawaii.edu/~zqyun/caevp.html

I'ld love to integrate this in PyLab_Works,
so the optical system can be manipulated both through interacting with 
the graphical window and by code.
I tried to connect the guy, but he doesn't respond.

cheers,
Stef


> thanks,
>
> elliot
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
>   


From yosefmel at post.tau.ac.il  Tue Jul 28 02:42:04 2009
From: yosefmel at post.tau.ac.il (Yosef Meller)
Date: Tue, 28 Jul 2009 09:42:04 +0300
Subject: [SciPy-User] optical ray tracing
In-Reply-To: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
References: <a07a0f1c0907270934w4cd87d5eufb78f8efe8c0d057@mail.gmail.com>
Message-ID: <200907280942.05360.yosefmel@post.tau.ac.il>

On Monday 27 July 2009 19:34:37 Elliot Hallmark wrote:
> I'm looking to do some raytracing for a nonimaging design and there
> are no opensource packages that address the geometry needed.  Scipy
> seems like a good package to become familiar with anyway (I miss
> mathematica now that i'm no longer a university student), but i was
> wondering if anyone has already done some optical ray tracing work in
> scipy that i could build off of.
>
> Wouldn't be to hard to do from scratch, but my focus is on the design
> and the fewer excursions into building new tools the better.

I have an intern writing a ray-tracer for optical ray tracing right now. In 
about a month we expect to have the basic framework in releasable shape, but 
if you're impatient you can have a look at her git tree:

http://github.com/jasminef/tracer/tree/optic_tests

Corrently some of the tests fail, but we have the following:
* Construction of assemblies of objects
* surfaces: flat and spherical (parabolic on the way)
* specular reflection and refraction

We are focusing on modular design and programmability, not speed. It should, 
then, be not too hard to add a new surface geometry.

Again, this is still not in releasable shape, but maybe it'll help.



From fperez.net at gmail.com  Tue Jul 28 13:39:44 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 28 Jul 2009 10:39:44 -0700
Subject: [SciPy-User] [SciPy-user] Numpy/Scipy and the Python African
	Tour
In-Reply-To: <20090723141737.GF10321@phare.normalesup.org>
References: <20090723141737.GF10321@phare.normalesup.org>
Message-ID: <db6b5ecc0907281039q7adfe9d6ic3ac652239633072@mail.gmail.com>

Hi Emanuelle,

On Thu, Jul 23, 2009 at 7:17 AM, Emmanuelle
Gouillart<emmanuelle.gouillart at normalesup.org> wrote:
> Dear users of Numpy and Scipy,
>
> here is an informal report on the last event of the Python African Tour,
> which took place in Dakar (Senegal) on July 6-10th. It might interest

Many thanks for this great post, I am very happy to see this kind of
effort sprouting organically in different places.  I had the chance to
do something similar with Stefan, his advisor and colleagues a few
years ago in Stellenbosch, South Africa, and it was a phenomenal
experience.  The audience was extremely engaged and the facilities
first-rate, and I enjoyed it enormously.

I'm glad to hear that you've set up a mailing list for your students
to continue growing, but I'd also encourage them to subscribe to the
'normal' numpy/scipy lists.  Even if they initially feel a bit shy
about posting, simply 'lurking' on the lists for a while and reading
the flow of information can be very useful both to learn technical
ideas and to get familiar with the discussion patterns of a community.
 This will ease them into naturally engaging the upstream projects
when they feel comfortable, which I think in the long term is both
useful and necessary.

> Speaking about difficulties, one may say that there was some disparity in
> the level of experience inside the group, some being very well at ease
> with array programming while others were obviously discovering
> programming. It was therefore difficult to meet everybody's needs, but
> well, I hope everybody learned something!

Audience heterogeneity: that's always the hardest part in these
things, and we face it everywhere.  It's the main source of concern
even at the main Scipy conference intro tutorial, since the audience
tends to be all over the place in terms of background and experience.

Mmh, I just had a thought.  A number of us have by now taught
reasonably organized workshops of this kind in different places and
institutions. I wonder if it wouldn't be a good idea to have a page on
the scipy site collecting this information, with a short paragraph
about the event and a link to any additional external materials that
may exist.  This could encourage others to create similar things in
their own communities, whether with the help of one of the previous
presenters or fully independently.  Does that sound useful?

In any case, thanks again for the great post!

Best,

f