From schut at sarvision.nl  Fri Jul  1 03:59:51 2011
From: schut at sarvision.nl (Vincent Schut)
Date: Fri, 01 Jul 2011 09:59:51 +0200
Subject: [SciPy-Dev] Many errors/failures running scipy.test() using
 latest numpy and scipy
In-Reply-To: <iuilv2$na6$1@dough.gmane.org>
References: <BANLkTi=Um09tEjqh4-4itub5zZd_M65CQg@mail.gmail.com>
	<itqg2b$abc$1@dough.gmane.org> <iuhm65$854$1@dough.gmane.org>
	<iuilv2$na6$1@dough.gmane.org>
Message-ID: <iujupn$udb$1@dough.gmane.org>

On 06/30/2011 10:22 PM, Pauli Virtanen wrote:
> On Thu, 30 Jun 2011 13:20:37 +0200, Vincent Schut wrote:
> [clip]
>> anything I could do to diagnose this further?
>
> This is a bit difficult, or at least time-consuming. One thing that
> can be done is to diff the assembler output from Gfortran 4.6 and 4.4
> of the iterative solvers, find the points where it differs, and then
> check if the Fortran code does something suspicious at the
> corresponding point.
>
> If someone else wants to spend some quality time with assembler
> listings before I find time etc. for this, please go ahead :)
>
>> I'd like to have a working scipy again...
>
> These failures have always been there, it's just that now
> we have better test coverage so that we actually catch them...
>
> 	Pauli

Hmm, my assembler skills are a bit rusty... :-)
I'll just keep gfortran-4.4 installed and as the default fortran 
compiler than, for the time being. Just to be sure: I suppose there is 
no problem mixing gfortran-4.4 for scipy's fortran code and gcc-4.6 for 
the c code parts? As a precaution, I have redone my atlas/blas/lapack 
with gfortran-4.4 also, so the entire scientific python stack is 
gfortran-4.4 based, now.

Off to the testing failures of scikits-learn, then.
Thanks,
Vincent.



From drwells at vt.edu  Wed Jul  6 14:21:42 2011
From: drwells at vt.edu (Dave Wells)
Date: Wed, 6 Jul 2011 14:21:42 -0400
Subject: [SciPy-Dev] Desire to volunteer
Message-ID: <CALa6-KomtUZwP1rQx3GWd0Xgrc1KTwCEWhG2Rchk+h0ZNxPjkw@mail.gmail.com>

Hello SciPy!

This is my first time posting to the list so please redirect me if this is
off-topic or would be better suited somewhere else. I am a graduate student
in math who made the switch from MATLAB so SciPy over the last six months
for class work and I couldn't be happier with the project.

I would like to get involved in development, or writing documentation, or
licking stamps (really anything) to help. In particular I like linear
algebra and finite elements. How can I get involved with SciPy?

Thanks,
Dave Wells
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110706/79ad0e36/attachment.html>

From charlesr.harris at gmail.com  Wed Jul  6 17:27:09 2011
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 6 Jul 2011 15:27:09 -0600
Subject: [SciPy-Dev] Desire to volunteer
In-Reply-To: <CALa6-KomtUZwP1rQx3GWd0Xgrc1KTwCEWhG2Rchk+h0ZNxPjkw@mail.gmail.com>
References: <CALa6-KomtUZwP1rQx3GWd0Xgrc1KTwCEWhG2Rchk+h0ZNxPjkw@mail.gmail.com>
Message-ID: <CAB6mnxJOta1dv-NY5YmvhC8nK+Bhq8MKV22hGiJPrFMDRBmaDA@mail.gmail.com>

On Wed, Jul 6, 2011 at 12:21 PM, Dave Wells <drwells at vt.edu> wrote:

> Hello SciPy!
>
> This is my first time posting to the list so please redirect me if this is
> off-topic or would be better suited somewhere else. I am a graduate student
> in math who made the switch from MATLAB so SciPy over the last six months
> for class work and I couldn't be happier with the project.
>
> I would like to get involved in development, or writing documentation, or
> licking stamps (really anything) to help. In particular I like linear
> algebra and finite elements. How can I get involved with SciPy?
>
>
Depends on what your skills and interests are. If you are comfortable in
other programming languages trying to fix some of the open
tickets<http://projects.scipy.org/scipy>would be both useful and
educational. Adding and cleaning up
documentation <http://docs.scipy.org/scipy/Front%20Page/> would be another
area to explore, but you will need to get permissions for that; ask on the
list. Beyond that, much depends on your own interests. We can always use
help.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110706/adb41232/attachment.html>

From ilanschnell at gmail.com  Fri Jul  8 01:47:01 2011
From: ilanschnell at gmail.com (Ilan Schnell)
Date: Fri, 8 Jul 2011 00:47:01 -0500
Subject: [SciPy-Dev] ANN: EPD 7.1 released
Message-ID: <CAHxB1U1R7VaRrfH2kN_wZC1PWK_+RAyXcH2XMgWg170S=06zFw@mail.gmail.com>

Hello,

I am pleased to announce that EPD (Enthought Python Distribution)
version 7.1 has been released.  The most significant change is the
addition of an "EPD Free" version, which has its own very liberal
license, and can be downloaded and used free of any charge by
anyone (not only academics).  "EPD Free" includes a subset of the
packages included in the full EPD.  The highlights of this subset are:
numpy, scipy, matplotlib, traits and chaco.  To see which libraries
are included in the free vs. full version, please see:

        http://www.enthought.com/products/epdlibraries.php

In addition we have opened our PyPI build mirror for everyone.
This means that one can type "enpkg xyz" for 10,000+ packages.
However, there are still benefits to becoming an EPD subscriber.

        http://www.enthought.com/products/getepd.php

Apart from the addition of "EPD Free", this release includes updates
to over 30 packages, including numpy, scipy, ipython and ETS.
We have also added PySide, Qt and MDP to this release.  Please find the
complete list of additions, updates and bug fixes in the change log:

        http://www.enthought.com/products/changelog.php


About EPD
---------
The Enthought Python Distribution (EPD) is a "kitchen-sink-included"
distribution of the Python programming language, including over 90
additional tools and libraries. The EPD bundle includes NumPy, SciPy,
IPython, 2D and 3D visualization, and many other tools.

EPD is currently available as a single-click installer for Windows XP,
Vista and 7, MacOS (10.5 and 10.6), RedHat 3, 4 and 5, as well as
Solaris 10 (x86 and x86_64/amd64 on all platforms).

All versions of EPD (32 and 64-bit) are free for academic use.  An
annual subscription including installation support is available for
individual and commercial use.  Additional support options, including
customization, bug fixes and training classes are also available:

        http://www.enthought.com/products/epd_sublevels.php

- Ilan


From thouis at gmail.com  Mon Jul 11 08:43:51 2011
From: thouis at gmail.com (Thouis (Ray) Jones)
Date: Mon, 11 Jul 2011 14:43:51 +0200
Subject: [SciPy-Dev] Post-doctoral position in image & data analysis for
 image-based high-content screening
Message-ID: <CAHGWWxHvK6LcbSPnnjVWjsJbdzipc3OKUH1u22wuC3QsrQWsdA@mail.gmail.com>

Short summary:
A funded post-doctoral position in data analysis, machine-learning,
and statistics applied to biological image-based high-content
screening is available at the BioPhenics platform of the Curie
Institute (Paris, France).

The position involves development and maintenance of tools in Python,
Matlab, or R that will be used both to analyze current screens and as
a contribution to a larger community. Expertise in machine learning or
statistical analysis with large biological datasets is desirable, good
knowledge of image processing tools is an asset.

Full details at:
http://dl.dropbox.com/u/16028921/PostDocApp_MachineLearning_BFX.pdf

Thouis Jones
Institut Curie


From gael.varoquaux at normalesup.org  Sat Jul 16 17:28:49 2011
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 16 Jul 2011 23:28:49 +0200
Subject: [SciPy-Dev] Renaming scikits.learn
Message-ID: <20110716212849.GA31151@phare.normalesup.org>

Hey list,

I was having lunch with some of the scikits.image and scikits.statsmodel
people and the statsmodel guys said that they were fed up with namespace
packages and wanted to get rid of them. In other words, getting rid of
the scikits namespace. I am _very_ enthousiastic about this, because I
profoundly dislike namespace packages, and would thus like to suggest
going down that road. Now, this discussion is probably going to be fairly
long so I'll try to have a good signal to noise.

Why: the problems with namespace packages
==========================================

 a. The scikits/__init__.py file is owned by whichever installs last. As
    a result a different scikit can screw us by mistake.

 b. Debian, active-state and EPD systematically remove namespace package:
    they install as standard directories and merge the __init__.py files.

    In this case, installing as a user (non root) does not work: the scikits
    directory has a __init__.py without the right incantation to get
    namespace packages working, and thus for the user to install a new
    version, he has to strip the namespace package too (in other words,
    setup.py and easy_install do not work out of the box). This is of
    course related to a.

 c. People have a hard time figuring out where there stuff is installed.
    I have had a lot of support calls related to this one.

 d. The namespace machinery slows down a lot the imports because it
    forces the Python imports to do a lot of 'stat' calls. This is the
    main reason why EPD remove the namespace packages.

 e. Shortening import paths.

Why not
========

Two obvious reasons:

 a. Backward compatibility of the imports

 b. Branding: we are going to take a hit on the name that we have managed
    to establish.

 c. Scikits, as a brand name, is taking a hit if we all do that.

Point a is a technical one and can be addressed using a proxy package for
a little while (say 2 releases).

Proposals for the future
=========================

As you have guessed, I am very much in favor of getting rid of namespace
packages. Here are possible ways of doing this:

 a. Rename completly the package (learnpy?)

 b. Rename the package with a prefix common to scikits (sklearn?)

 c. Keep the branding name 'scikit-learn', but change the import path to
    learn

I am leaning towards c. Feedback?

Gael


From cournape at gmail.com  Sat Jul 16 21:03:16 2011
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 17 Jul 2011 10:03:16 +0900
Subject: [SciPy-Dev] Renaming scikits.learn
In-Reply-To: <20110716212849.GA31151@phare.normalesup.org>
References: <20110716212849.GA31151@phare.normalesup.org>
Message-ID: <CAGY4rcXJ==4g8JZjVh-vdvzgqo0fVV1xx5fat2uptPuV14_KiQ@mail.gmail.com>

On Sun, Jul 17, 2011 at 6:28 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> Hey list,
>
> I was having lunch with some of the scikits.image and scikits.statsmodel
> people and the statsmodel guys said that they were fed up with namespace
> packages and wanted to get rid of them. In other words, getting rid of
> the scikits namespace. I am _very_ enthousiastic about this, because I
> profoundly dislike namespace packages, and would thus like to suggest
> going down that road. Now, this discussion is probably going to be fairly
> long so I'll try to have a good signal to noise.
>
> Why: the problems with namespace packages
> ==========================================
>
> ?a. The scikits/__init__.py file is owned by whichever installs last. As
> ? ?a result a different scikit can screw us by mistake.
>
> ?b. Debian, active-state and EPD systematically remove namespace package:
> ? ?they install as standard directories and merge the __init__.py files.
>
> ? ?In this case, installing as a user (non root) does not work: the scikits
> ? ?directory has a __init__.py without the right incantation to get
> ? ?namespace packages working, and thus for the user to install a new
> ? ?version, he has to strip the namespace package too (in other words,
> ? ?setup.py and easy_install do not work out of the box). This is of
> ? ?course related to a.
>
> ?c. People have a hard time figuring out where there stuff is installed.
> ? ?I have had a lot of support calls related to this one.
>
> ?d. The namespace machinery slows down a lot the imports because it
> ? ?forces the Python imports to do a lot of 'stat' calls. This is the
> ? ?main reason why EPD remove the namespace packages.
>
> ?e. Shortening import paths.

Are those rationales still valid for PEP 382 ? I happen to like the
namespace package idea, but don't know if its flaws are inherent to
python importing mechanism or just an issue with current
implementation. c and e are not really convincing (people have a hard
time figuring out where stuff is installed in general, and I am
doubtful people who know how to locate imported packages cannot figure
it out for namespace packages).

> Why not
> ========
>
> Two obvious reasons:
>
> ?a. Backward compatibility of the imports
>
> ?b. Branding: we are going to take a hit on the name that we have managed
> ? ?to establish.
>
> ?c. Scikits, as a brand name, is taking a hit if we all do that.
>
> Point a is a technical one and can be addressed using a proxy package for
> a little while (say 2 releases).

How did EPD developers handle it ? If this is the road to be taken, it
would be worthwhile to think about a convertion script to help
transition. Maybe something like rope can be used to that effect (I
have recently used it in a very hackish way to do the exact opposite).

cheers,

David


From olivier.grisel at ensta.org  Sun Jul 17 05:06:55 2011
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 17 Jul 2011 11:06:55 +0200
Subject: [SciPy-Dev] [Scikit-learn-general] Renaming scikits.learn
In-Reply-To: <20110716212849.GA31151@phare.normalesup.org>
References: <20110716212849.GA31151@phare.normalesup.org>
Message-ID: <CAFvE7K5shtKX28ue_w=4=+tKQMOPEP-je75zVDb_Rs9gzceVwA@mail.gmail.com>

2011/7/16 Gael Varoquaux <gael.varoquaux at normalesup.org>:
>
> Proposals for the future
> =========================
>
> As you have guessed, I am very much in favor of getting rid of namespace
> packages. Here are possible ways of doing this:
>
> ?a. Rename completly the package (learnpy?)
>
> ?b. Rename the package with a prefix common to scikits (sklearn?)
>
> ?c. Keep the branding name 'scikit-learn', but change the import path to
> ? ?learn

I don't have an opinion yet on whether or not we should abandon the
namespace package. But if we do we cannot use a package name as
generic as "learn" for the scikit-learn project IMHO. And it would be
better if it was the same as the project name to make it easier to
guess what to import.

In any case, if we decide to change the package (and the project)
names of the scikits.* projects we should definitely try to do it in a
consistent way across projects rather than act in isolation.

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel


From mathieu at mblondel.org  Sun Jul 17 11:00:35 2011
From: mathieu at mblondel.org (Mathieu Blondel)
Date: Mon, 18 Jul 2011 00:00:35 +0900
Subject: [SciPy-Dev] Renaming scikits.learn
In-Reply-To: <20110716212849.GA31151@phare.normalesup.org>
References: <20110716212849.GA31151@phare.normalesup.org>
Message-ID: <CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>

On Sun, Jul 17, 2011 at 6:28 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:

> ?a. The scikits/__init__.py file is owned by whichever installs last. As
> ? ?a result a different scikit can screw us by mistake.

That seems unlikely to happen if the scikits agree upon using the same
file for __init__.py (which they seem to do).

> ?d. The namespace machinery slows down a lot the imports because it
> ? ?forces the Python imports to do a lot of 'stat' calls. This is the
> ? ?main reason why EPD remove the namespace packages.

Any measurement? That seems negligible compared to, say, loading the
scipy.sparse machinery...

> ?b. Rename the package with a prefix common to scikits (sklearn?)

Overall, I didn't feel like the reasons you mention are strong enough
to change anything but if you want to go down that path, I also prefer
b, both for preserving the branding and because "learn" is a too
generic name.

I think there would be a lot to do on the marketing side. Currently
the scikit portal (http://scikits.appspot.com/) seems really poorly
maintained. The website should be maintained with sphinx and include
nice pictures and code examples of each scikit. The quality of the
different scikits seems to be quite heterogeneous. It seems to me that
it's because any project can become a scikit. Since scikits are
somewhat "endorsed" by the scipy community, It would be better if
existing projects applied for being a scikit. Also what about
collaboration between scikits? At least at the documentation level, it
seems like scikits could point to each other.

My 2c,
Mathieu


From fabian.pedregosa at inria.fr  Mon Jul 18 03:09:03 2011
From: fabian.pedregosa at inria.fr (Fabian Pedregosa)
Date: Mon, 18 Jul 2011 09:09:03 +0200
Subject: [SciPy-Dev] [Scikit-learn-general] Renaming scikits.learn
In-Reply-To: <1747323296.3613710.1310929812848.JavaMail.root@zmbs3.inria.fr>
References: <20110716212849.GA31151@phare.normalesup.org>
	<1747323296.3613710.1310929812848.JavaMail.root@zmbs3.inria.fr>
Message-ID: <CAH0k1yXrPfQP69i1PGNNo7RYMfkKTCEMbuzA=6aJtmr_5w39Hw@mail.gmail.com>

On Sun, Jul 17, 2011 at 9:10 PM, Vlad Niculae <vlad at vene.ro> wrote:
> I would like to point out that there was already some confusion (at
> least for me) regarding whether it's scikits.learn, scikit-learn,
> scikits-learn or scikit.learn. I'm +1 for the name 'learn' and +0.5
> for 'sklearn'.
>

I like both, although sklearn might be more search-friendly.

Fabian.

> Best,
> Vlad
>
> ------------------------------------------------------------------------------
> AppSumo Presents a FREE Video for the SourceForge Community by Eric
> Ries, the creator of the Lean Startup Methodology on "Lean Startup
> Secrets Revealed." This video shows you how to validate your ideas,
> optimize your ideas and identify your business strategy.
> http://p.sf.net/sfu/appsumosfdev2dev
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>


From cournape at gmail.com  Mon Jul 18 23:27:52 2011
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 19 Jul 2011 12:27:52 +0900
Subject: [SciPy-Dev] Renaming scikits.learn
In-Reply-To: <CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
References: <20110716212849.GA31151@phare.normalesup.org>
	<CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
Message-ID: <CAGY4rcUtcVXGztij0=P5aPKGpyb-RSsketwxGWKHgD06cnZQAg@mail.gmail.com>

On Mon, Jul 18, 2011 at 12:00 AM, Mathieu Blondel <mathieu at mblondel.org> wrote:
> On Sun, Jul 17, 2011 at 6:28 AM, Gael Varoquaux
> <gael.varoquaux at normalesup.org> wrote:
>
>> ?a. The scikits/__init__.py file is owned by whichever installs last. As
>> ? ?a result a different scikit can screw us by mistake.
>
> That seems unlikely to happen if the scikits agree upon using the same
> file for __init__.py (which they seem to do).

Actually, that's not the only way a namespace package can be screwed
up (see e.g. http://mail.python.org/pipermail/distutils-sig/2009-May/011786.html).

But I see this more as a special case of the whole python packaging
mess than something inherent to namespace package.

>
> Overall, I didn't feel like the reasons you mention are strong enough
> to change anything but if you want to go down that path, I also prefer
> b, both for preserving the branding and because "learn" is a too
> generic name.

Agreed. I tried to find an explanation about why Enthought did remove
their own namespace package, but could not find anything. Renaming
some scikits and not other (which will inevitably happen) seems even
more confusing than the current situation.

I am rather against it, but then I have not contributed for quite a while :)

David


From olivier.grisel at ensta.org  Tue Jul 19 06:04:40 2011
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Tue, 19 Jul 2011 12:04:40 +0200
Subject: [SciPy-Dev] [Scikit-learn-general]  Renaming scikits.learn
In-Reply-To: <CAJPUwMAPwDypBtO7PNx8hDYEQ5mX59BuVg0Cciq0XnwmuiBidQ@mail.gmail.com>
References: <20110716212849.GA31151@phare.normalesup.org>
	<CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
	<CAGY4rcUtcVXGztij0=P5aPKGpyb-RSsketwxGWKHgD06cnZQAg@mail.gmail.com>
	<CAJPUwMAPwDypBtO7PNx8hDYEQ5mX59BuVg0Cciq0XnwmuiBidQ@mail.gmail.com>
Message-ID: <CAFvE7K5cPn=bn=g9HskD=-KbCmrsVfE4AWaO7UmS3ih4XJ5L2A@mail.gmail.com>

2011/7/19 Wes McKinney <wesmckinn at gmail.com>:
>
> As Ga?l mentioned, we ({scikits}.statsmodels) have more or less
> decided to ditch the namespace. Of course we are a smaller group of
> developers. I personally don't see the benefit of the namespace,
> especially in larger and well-established libraries. I understand the
> original intent of scikits was to create a scipy-like package but
> allow everyone to have their own release cycle. For a library like
> scikit-learn or statsmodels (which are already becoming quite large,
> not as big as scipy but maybe someday!) it seems like mainly an
> annoyance with few redeeming qualities.

Do you plan to ditch the "scikits" name from the project branding as
well or just from the python package?

It would be great if other scikits.* developers could express their
own opinion on the issue.

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel


From josef.pktd at gmail.com  Tue Jul 19 08:02:15 2011
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 19 Jul 2011 14:02:15 +0200
Subject: [SciPy-Dev] [Scikit-learn-general] Renaming scikits.learn
In-Reply-To: <CAFvE7K5cPn=bn=g9HskD=-KbCmrsVfE4AWaO7UmS3ih4XJ5L2A@mail.gmail.com>
References: <20110716212849.GA31151@phare.normalesup.org>
	<CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
	<CAGY4rcUtcVXGztij0=P5aPKGpyb-RSsketwxGWKHgD06cnZQAg@mail.gmail.com>
	<CAJPUwMAPwDypBtO7PNx8hDYEQ5mX59BuVg0Cciq0XnwmuiBidQ@mail.gmail.com>
	<CAFvE7K5cPn=bn=g9HskD=-KbCmrsVfE4AWaO7UmS3ih4XJ5L2A@mail.gmail.com>
Message-ID: <CAMMTP+BFUgFRVKASsWu=0dX7qzLgXsX5fxyqj0MvSQo5fau5Gg@mail.gmail.com>

On Tue, Jul 19, 2011 at 12:04 PM, Olivier Grisel
<olivier.grisel at ensta.org> wrote:
> 2011/7/19 Wes McKinney <wesmckinn at gmail.com>:
>>
>> As Ga?l mentioned, we ({scikits}.statsmodels) have more or less
>> decided to ditch the namespace. Of course we are a smaller group of
>> developers. I personally don't see the benefit of the namespace,
>> especially in larger and well-established libraries. I understand the
>> original intent of scikits was to create a scipy-like package but
>> allow everyone to have their own release cycle. For a library like
>> scikit-learn or statsmodels (which are already becoming quite large,
>> not as big as scipy but maybe someday!) it seems like mainly an
>> annoyance with few redeeming qualities.
>
> Do you plan to ditch the "scikits" name from the project branding as
> well or just from the python package?
>
> It would be great if other scikits.* developers could express their
> own opinion on the issue.

I'm pretty much indifferent either way.
I really like the name recognition and family relationship that the
scikits name includes, both across scikits and with scipy. On Windows
without any distribution, we had some problems early on with namespace
installation, but I didn't hear of any problems in a long time.
(Problems with EPD enstaller seems to be more a problem of EPD than of
the scikits.)

On the other hand we picked the name statsmodels from the beginning
for it's usefulness as a standalone name that is easily searchable,
and most times we only refer to scikits.statsmodels as statmodels.

So except for the grouping together of the scikits packages and scipy
in terms of association by name (e.g. we used it for GSOC
applications), I don't see any disadvantage for statsmodels, and not
having a namespace package still makes life a bit easier.

Josef

>
> --
> Olivier
> http://twitter.com/ogrisel - http://github.com/ogrisel
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From ben.root at ou.edu  Tue Jul 19 14:21:17 2011
From: ben.root at ou.edu (Benjamin Root)
Date: Tue, 19 Jul 2011 13:21:17 -0500
Subject: [SciPy-Dev] Documentation error for scipy.ndimage?
Message-ID: <CANNq6FmETv2mr=f-dqbyhH40LU-R27iR3QqjsgH_T155U26V5g@mail.gmail.com>

Hello,

I have been using the binary morphology functions in ndimage in a project.
I noticed that the docs for binary_opening() (and others) stated that it
accepts an integer or float for the iterations kwarg.  Also, that values
less than 1 would result in the binary_opening iteration to occur
successively until there were no more changes in the image.  However, if I
set iterations to 0.5, I get an error saying that it was expecting an
integer, and setting iterations to zero just obliterates the image.

>>> import numpy as np
>>> import scipy.ndimage as ndimg
>>> a = np.zeros((10, 10), dtype=bool)
>>> a[3:5, 2:5] = True
>>> a[5:7, 5:7] = True
>>> a[7:10, 2:8] = True
>>> a
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0]])
>>> ndimg.binary_opening(a, iterations=0).astype(int)
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])
>>> ndimg.binary_opening(a, iterations=0.5).astype(int)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bvr/Programs/scipy/scipy/ndimage/morphology.py", line 654, in
binary_opening
    origin)
  File "/home/bvr/Programs/scipy/scipy/ndimage/morphology.py", line 402, in
binary_erosion
    output, border_value, origin, 0, brute_force)
  File "/home/bvr/Programs/scipy/scipy/ndimage/morphology.py", line 278, in
_binary_erosion
    origin, invert, coordinate_list)
TypeError: integer argument expected, got float


Manually performing iterations reveals that the final result shouldn't be an
obliterated image:

>>> ndimg.binary_opening(a).astype(int)
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 1, 1, 1, 1, 0, 0, 0]])
>>> ndimg.binary_opening(ndimg.binary_opening(a))
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 0, 0, 1, 1, 0, 0, 0],
       [0, 0, 0, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 1, 1, 1, 1, 0, 0, 0]])


Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110719/b2ec1c3d/attachment.html>

From collinstocks at gmail.com  Tue Jul 19 23:16:26 2011
From: collinstocks at gmail.com (Collin Stocks)
Date: Tue, 19 Jul 2011 23:16:26 -0400
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
	factorization)
Message-ID: <1311131786.3630.15.camel@SietchTabr>

SciPy Development Team,

I would like to bring your attention to a patch I have submitted to
SciPy involving the scipy.linalg.qr() function. The relevant ticket
number is #1473, found at http://projects.scipy.org/scipy/ticket/1473.

I have added a boolean keyword argument called 'pivoted' which causes
qr() to perform QR decomposition with pivoting (rank revealing QR
factorization). Ultimately, 'mode' should probably be used to flag
whether or not this sort of decomposition should be done, but for
backwards-compatibility sake, I have added the functionality through the
use of a new keyword.

I have also created a bunch of new tests to test this functionality of
qr(). I believe that these tests are as comprehensive as the existing
tests for the same function.

I hope that this is able to make it into the next release of SciPy. The
patch I have created should be sufficient to add the requested feature.

Rationale:
QR decomposition with pivoting is a necessary feature in order to write
a numpy implementation of the MatLab stepwisefit() function from the
statistics toolbox which matches the original as closely as possible.
``scikits.statsmodels'' has expressed interest in such a function, and I
have implemented one using cvxopt.lapack.geqp3() for the qr
decomposition with pivoting. However, this is slow due to copying
between numpy arrays and cvxopt matrices, and for some reason leads to a
massive memory leak. In some circumstances, it even seems to lead to a
segmentation fault. This is clearly unacceptable. A better solution
would seem to be to add this functionality directly to SciPy.

Sincerely,
Collin Stocks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110719/c33aea00/attachment.sig>

From martin.teichmann at mbi-berlin.de  Wed Jul 20 03:32:36 2011
From: martin.teichmann at mbi-berlin.de (Martin Teichmann)
Date: Wed, 20 Jul 2011 09:32:36 +0200
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
	factorization)
In-Reply-To: <1311131786.3630.15.camel@SietchTabr>
References: <1311131786.3630.15.camel@SietchTabr>
Message-ID: <CAO9LD_CF=oY-v9UWi1o83-ZdfmVLWmG4q_EuJJTNmtt3rqXWqA@mail.gmail.com>

Hi List,
Hi Collin,

coincidentally, I've been working on the same problem,
and so I also want to share my two cents:

> I have added a boolean keyword argument called 'pivoted' which causes
> qr() to perform QR decomposition with pivoting (rank revealing QR

I would prefer the keyword "pivoting", since this is how it is usually
called.

> factorization). Ultimately, 'mode' should probably be used to flag
> whether or not this sort of decomposition should be done,

I think a keyword is a much better choice. Mode decides whether
the second step of finding Q is done or not, which, in principle,
has nothing to do with pivoting or not.

On the other hand, I opt for a new mode, which I would call "reflectors",
which returns the reflectors of Q calculated in the first step of
the factorization, some people out there (me) need that result.

Greetings

Martin


From gael.varoquaux at normalesup.org  Wed Jul 20 08:23:01 2011
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 20 Jul 2011 14:23:01 +0200
Subject: [SciPy-Dev] [Scikit-learn-general]  Renaming scikits.learn
In-Reply-To: <CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
References: <20110716212849.GA31151@phare.normalesup.org>
	<CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
Message-ID: <20110720122301.GB23430@phare.normalesup.org>

Thanks for everybody's input. Here is a wrap up.

I am replying to Mathieu, but condensing several points raised by
different people:

On Mon, Jul 18, 2011 at 12:00:35AM +0900, Mathieu Blondel wrote:
> > ?d. The namespace machinery slows down a lot the imports because it
> > ? ?forces the Python imports to do a lot of 'stat' calls. This is the
> > ? ?main reason why EPD remove the namespace packages.

> Any measurement?

No. 2 to 3 years ago, when setuptools bashing was a trending thing,
people made measurement on the ETS.

> That seems negligible compared to, say, loading the scipy.sparse
> machinery...

Depends on the platform and the disk setup. On windows a 'stat' is slow,
and namespace packages induce non-negligeable overhead in startup times.
When EPD got systematicaly rid on namespace packages, there was a speedup
of several folds for application startup time on windows.

Another environment in which it ends up being slow is on NFS drives,
which are still fairly comon in research labs.

The slowness was the major driver between EPD working around them by
instaling them in a flat way.

> Overall, I didn't feel like the reasons you mention are strong enough
> to change anything

My major driver is my point b:

> > b. Debian, active-state and EPD systematically remove namespace package:
> >    they install as standard directories and merge the __init__.py files.
> >
> >    In this case, installing as a user (non root) does not work: the
> >    scikits directory has a __init__.py without the right incantation to 
> >    get namespace packages working, and thus for the user to install a new
> >    version, he has to strip the namespace package too (in other words,
> >    setup.py and easy_install do not work out of the box). This is of
> >    course related to a.

this is a major problem. More and more people are moving to
distribution-controled package installation, whether it is with Debian or
EPD. For these people, upgrading the scikit-learn with a local install is
pretty much hell. I have been experiencing this with mayavi on my Ubuntu
box at work: it's almost impossible to run a different version than the
systems. So far, I hadn't any problems with the scikit, as it wasn't
installed by the sysadmins. 

On Sun, Jul 17, 2011 at 10:03:16AM +0900, David Cournapeau wrote:
> Are those rationales still valid for PEP 382 ?

Possibly not, but it's going to take years before it is implemented and
available on all the platform we support. 

In addition, it seems to me that there is a really simply solution to our
technical problem (change the import path to not have a '.' in the
middle) and a really hard one (get import mechanism to route accross
modules). I like to go for the simply one.

> How did EPD developers handle it ?

Simply install everything as --signle-version-externally-managed.

> I tried to find an explanation about why Enthought did remove
> their own namespace package, but could not find anything.

I was talking to them last week and it simply boils down to them being
tired of dealing with the side effects mentioned above for no gain.

On Tue, Jul 19, 2011 at 11:32:48PM -0400, Wes McKinney wrote:
> I think project branding and package name are slightly separate
> topics, no? For example, I don't think that PyTables has suffered from
> it being "import tables". I actually prefer it to the alternative
> "import pytables", maybe just from typing it so many times.

That's a good point. There are other examples (wxpython for instance, and
I had others on my mind, but they are gone...).

__________________________________________________

To sumarize the general fealing, it seems to me that people would prefer:

  - Keep the branding name 'scikit-learn'
  - Change the import path to 'sklearn'.

Am I giving a good summary?

Gael



From mathieu at mblondel.org  Wed Jul 20 11:50:16 2011
From: mathieu at mblondel.org (Mathieu Blondel)
Date: Thu, 21 Jul 2011 00:50:16 +0900
Subject: [SciPy-Dev] [Scikit-learn-general]  Renaming scikits.learn
In-Reply-To: <CADeotZog3V7P_hZO6zMg8edtvKA39od13Y+qs9=DKR5Sv+JaAg@mail.gmail.com>
References: <20110716212849.GA31151@phare.normalesup.org>
	<CAOKSrLwUArJ7BErfq3-Q_HhGfaoKZcHfhZmo6_BKN5GShpX2tA@mail.gmail.com>
	<20110720122301.GB23430@phare.normalesup.org>
	<CADeotZog3V7P_hZO6zMg8edtvKA39od13Y+qs9=DKR5Sv+JaAg@mail.gmail.com>
Message-ID: <CAOKSrLzF3E=5rX+4C515BieUJGheO0PjhUTn9eG0ccKrU0X2aQ@mail.gmail.com>

On Thu, Jul 21, 2011 at 12:02 AM, Alexandre Gramfort
<alexandre.gramfort at inria.fr> wrote:

> even I was not convinced at first like Mathieu and David,
> after advocating your suggestion so strongly I feel like I have to agree :)

To summarize my point of view: I'm ok with fixing the technical issues
related to namespaces but I don't want to give up on the scikits idea
yet, I think there are nice things to do.

Transition from scikits.learn to sklearn should be quite painless as
long as the submodule relative path doesn't change.

> This suggests a few things:
> - http://scikits.appspot.com/ should be kept up to date or removed.

I think the website really doesn't need to be dynamic (appspot). I
suggest scikit people maintain it in github and upload it to
scikits.scipy.org once in a while (if it's ok with scipy people).

> - scikits should share best practices regarding documentation and
> should link to each other
> - scikits should share best practices regarding test and coverage

> I fully agree with this. I feel that being a scikit should be a
> guarantee of quality.

[...]

> - projects should be reviewed by other scikits before becoming one.

Yes, "scikit" should be used as a branding. Otherwise, there isn't
much difference with a standalone project. So new projects should get
in after a vote and existing projects should be split into "mature"
(or graduated) and "work in progress" projects. The idea is quite
similar to the Apache foundation projects.

Mathieu


From collinstocks at gmail.com  Wed Jul 20 16:19:20 2011
From: collinstocks at gmail.com (Collin Stocks)
Date: Wed, 20 Jul 2011 16:19:20 -0400
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
 factorization)
In-Reply-To: <mailman.3.1311181202.30790.scipy-dev@scipy.org>
References: <mailman.3.1311181202.30790.scipy-dev@scipy.org>
Message-ID: <1311193160.3630.63.camel@SietchTabr>

Hi Martin,

I can change the keyword to 'pivoting' without any trouble. I don't
happen to know anything about reflectors, so I can't really help you
there at the moment, but perhaps if you gave me more information, I
could incorporate that into my patch (I'm not certain if doing so would
be a good idea or not -- please comment).

I'm glad to hear that you think it should be a keyword, as that's what
I've already done! What do you think about the 'economic' mode, though?
That is also independent of whether or not finding Q is done.

-- Collin

> Hi List,
> Hi Collin,
> 
> coincidentally, I've been working on the same problem,
> and so I also want to share my two cents:
> 
> > I have added a boolean keyword argument called 'pivoted' which
> causes
> > qr() to perform QR decomposition with pivoting (rank revealing QR
> 
> I would prefer the keyword "pivoting", since this is how it is usually
> called.
> 
> > factorization). Ultimately, 'mode' should probably be used to flag
> > whether or not this sort of decomposition should be done,
> 
> I think a keyword is a much better choice. Mode decides whether
> the second step of finding Q is done or not, which, in principle,
> has nothing to do with pivoting or not.
> 
> On the other hand, I opt for a new mode, which I would call
> "reflectors",
> which returns the reflectors of Q calculated in the first step of
> the factorization, some people out there (me) need that result.
> 
> Greetings
> 
> Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110720/c0a21f52/attachment.sig>

From collinstocks at gmail.com  Wed Jul 20 16:27:34 2011
From: collinstocks at gmail.com (Collin Stocks)
Date: Wed, 20 Jul 2011 16:27:34 -0400
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
 factorization)
In-Reply-To: <1311131786.3630.15.camel@SietchTabr>
References: <1311131786.3630.15.camel@SietchTabr>
Message-ID: <1311193654.3630.67.camel@SietchTabr>

Hi list,

I'm really sorry for not replying directly to the thread before. I
thought if I changed the subject, that it would behave properly and
attach my post to the existing thread. Obviously, I was wrong. I have
reproduced the post here. This is the one that should be replied to, not
the other.

---------------------------------------------------------------------

Hi Martin,

I can change the keyword to 'pivoting' without any trouble. I don't
happen to know anything about reflectors, so I can't really help you
there at the moment, but perhaps if you gave me more information, I
could incorporate that into my patch (I'm not certain if doing so would
be a good idea or not -- please comment).

I'm glad to hear that you think it should be a keyword, as that's what
I've already done! What do you think about the 'economic' mode, though?
That is also independent of whether or not finding Q is done.

-- Collin

> Hi List,
> Hi Collin,
> 
> coincidentally, I've been working on the same problem,
> and so I also want to share my two cents:
> 
> > I have added a boolean keyword argument called 'pivoted' which
> causes
> > qr() to perform QR decomposition with pivoting (rank revealing QR
> 
> I would prefer the keyword "pivoting", since this is how it is usually
> called.
> 
> > factorization). Ultimately, 'mode' should probably be used to flag
> > whether or not this sort of decomposition should be done,
> 
> I think a keyword is a much better choice. Mode decides whether
> the second step of finding Q is done or not, which, in principle,
> has nothing to do with pivoting or not.
> 
> On the other hand, I opt for a new mode, which I would call
> "reflectors",
> which returns the reflectors of Q calculated in the first step of
> the factorization, some people out there (me) need that result.
> 
> Greetings
> 
> Martin
-------------- next part --------------
An embedded message was scrubbed...
From: Collin Stocks <collinstocks at gmail.com>
Subject: QR decomposition with pivoting (rank revealing QR factorization)
Date: Tue, 19 Jul 2011 23:16:26 -0400
Size: 2650
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110720/2fcfa6e6/attachment.mht>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110720/2fcfa6e6/attachment.sig>

From collinstocks at gmail.com  Wed Jul 20 18:35:12 2011
From: collinstocks at gmail.com (Collin Stocks)
Date: Wed, 20 Jul 2011 18:35:12 -0400
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
 factorization)
In-Reply-To: <1311193654.3630.67.camel@SietchTabr>
References: <1311131786.3630.15.camel@SietchTabr>
	<1311193654.3630.67.camel@SietchTabr>
Message-ID: <1311201312.3630.73.camel@SietchTabr>

List,

FYI, my diff can be visualized quite nicely here:
https://github.com/collinstocks/scipy/compare/master...qr-with-pivoting

Please let me know if there is anything else I should do, or please
comment on the changes I have made. Once I have some more feedback, I
will file a pull request.

Thanks,
Collin
-------------- next part --------------
An embedded message was scrubbed...
From: Collin Stocks <collinstocks at gmail.com>
Subject: Re: QR decomposition with pivoting (rank revealing QR factorization)
Date: Wed, 20 Jul 2011 16:27:34 -0400
Size: 5900
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110720/e80d73f8/attachment.mht>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110720/e80d73f8/attachment.sig>

From kgdunn at gmail.com  Thu Jul 21 01:17:28 2011
From: kgdunn at gmail.com (Kevin Dunn)
Date: Thu, 21 Jul 2011 01:17:28 -0400
Subject: [SciPy-Dev] SciPy Central: a file and link sharing site - now active
Message-ID: <CAGD7iRYoyrsy0tYnj6GORPh+8GxgcE_LQEwOGKqzhz=pms38sQ@mail.gmail.com>

Hi

The SciPy Central website is now active and waiting for high-quality
code snippets and links to scientific resources of interest to the
SciPy community.

The website is at http://scipy-central.org and you can read some
background about the site's history at http://scipy-central.org/about

While we are officially in beta-mode, the site should be pretty
stable. If you detect any bugs, or have any suggestions for
improvements, please post them at
https://github.com/kgdunn/SciPyCentral/issues/

Thanks,
Kevin


From ralf.gommers at googlemail.com  Thu Jul 21 02:07:28 2011
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Thu, 21 Jul 2011 08:07:28 +0200
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
	factorization)
In-Reply-To: <1311201312.3630.73.camel@SietchTabr>
References: <1311131786.3630.15.camel@SietchTabr>
	<1311193654.3630.67.camel@SietchTabr>
	<1311201312.3630.73.camel@SietchTabr>
Message-ID: <CABL7CQhQ=bznSo3EjpHYjVMGM6vgDHCOOb17jOzT4JvW09MHUg@mail.gmail.com>

On Thu, Jul 21, 2011 at 12:35 AM, Collin Stocks <collinstocks at gmail.com>wrote:

> List,
>
> FYI, my diff can be visualized quite nicely here:
> https://github.com/collinstocks/scipy/compare/master...qr-with-pivoting
>
> Please let me know if there is anything else I should do, or please
> comment on the changes I have made. Once I have some more feedback, I
> will file a pull request.
>
> I made one comment on github, other than that the qr changes look pretty
good to me from a quick glance.

Cheers,
Ralf




> ---------- Forwarded message ----------
> From: Collin Stocks <collinstocks at gmail.com>
> To: scipy-dev at scipy.org
> Date: Wed, 20 Jul 2011 16:27:34 -0400
> Subject: Re: QR decomposition with pivoting (rank revealing QR
> factorization)
> Hi list,
>
> I'm really sorry for not replying directly to the thread before. I
> thought if I changed the subject, that it would behave properly and
> attach my post to the existing thread. Obviously, I was wrong. I have
> reproduced the post here. This is the one that should be replied to, not
> the other.
>
> ---------------------------------------------------------------------
>
> Hi Martin,
>
> I can change the keyword to 'pivoting' without any trouble. I don't
> happen to know anything about reflectors, so I can't really help you
> there at the moment, but perhaps if you gave me more information, I
> could incorporate that into my patch (I'm not certain if doing so would
> be a good idea or not -- please comment).
>
> I'm glad to hear that you think it should be a keyword, as that's what
> I've already done! What do you think about the 'economic' mode, though?
> That is also independent of whether or not finding Q is done.
>
> -- Collin
>
> > Hi List,
> > Hi Collin,
> >
> > coincidentally, I've been working on the same problem,
> > and so I also want to share my two cents:
> >
> > > I have added a boolean keyword argument called 'pivoted' which
> > causes
> > > qr() to perform QR decomposition with pivoting (rank revealing QR
> >
> > I would prefer the keyword "pivoting", since this is how it is usually
> > called.
> >
> > > factorization). Ultimately, 'mode' should probably be used to flag
> > > whether or not this sort of decomposition should be done,
> >
> > I think a keyword is a much better choice. Mode decides whether
> > the second step of finding Q is done or not, which, in principle,
> > has nothing to do with pivoting or not.
> >
> > On the other hand, I opt for a new mode, which I would call
> > "reflectors",
> > which returns the reflectors of Q calculated in the first step of
> > the factorization, some people out there (me) need that result.
> >
> > Greetings
> >
> > Martin
>
>
> ---------- Forwarded message ----------
> From: Collin Stocks <collinstocks at gmail.com>
> To: scipy-dev at scipy.org
> Date: Tue, 19 Jul 2011 23:16:26 -0400
> Subject: QR decomposition with pivoting (rank revealing QR factorization)
> SciPy Development Team,
>
> I would like to bring your attention to a patch I have submitted to
> SciPy involving the scipy.linalg.qr() function. The relevant ticket
> number is #1473, found at http://projects.scipy.org/scipy/ticket/1473.
>
> I have added a boolean keyword argument called 'pivoted' which causes
> qr() to perform QR decomposition with pivoting (rank revealing QR
> factorization). Ultimately, 'mode' should probably be used to flag
> whether or not this sort of decomposition should be done, but for
> backwards-compatibility sake, I have added the functionality through the
> use of a new keyword.
>
> I have also created a bunch of new tests to test this functionality of
> qr(). I believe that these tests are as comprehensive as the existing
> tests for the same function.
>
> I hope that this is able to make it into the next release of SciPy. The
> patch I have created should be sufficient to add the requested feature.
>
> Rationale:
> QR decomposition with pivoting is a necessary feature in order to write
> a numpy implementation of the MatLab stepwisefit() function from the
> statistics toolbox which matches the original as closely as possible.
> ``scikits.statsmodels'' has expressed interest in such a function, and I
> have implemented one using cvxopt.lapack.geqp3() for the qr
> decomposition with pivoting. However, this is slow due to copying
> between numpy arrays and cvxopt matrices, and for some reason leads to a
> massive memory leak. In some circumstances, it even seems to lead to a
> segmentation fault. This is clearly unacceptable. A better solution
> would seem to be to add this functionality directly to SciPy.
>
> Sincerely,
> Collin Stocks
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110721/5a04bd19/attachment.html>

From j.j.green at gmx.fr  Thu Jul 21 08:47:37 2011
From: j.j.green at gmx.fr (J.J. Green)
Date: Thu, 21 Jul 2011 14:47:37 +0200
Subject: [SciPy-Dev] scipy.interpolate.Rbf default epsilon
Message-ID: <1311252457.2283.72.camel@razi>

Hi all,

I believe that there is a problem with the default value
of the "epsilon" parameter in the scipy.interpolate.Rbf
initialisation function. 

In the original documentation for the library from which
this package is derived (Matlab code by Alex Chirokov)
the author states that epsilon should be "the average
distance between nodes", but means "the average distance
between consecutive nodes" (slide 10 of the ppt included
with the matlab package, available here

http://www.mathworks.com/matlabcentral/fileexchange/10056-scattered-data-interpolation-and-approximation-using-radial-base-functions
).   

The distinction is not clear as the author deals with the
one dimensional case in his example.

The version in scipy.interpolate.Rbf takes this advice literally
and calculates the mean distance between nodes, so uses a
value of epsilon which is *much* too large in general. 

I noticed this when getting bad results interpolating from 
300 data-points on a region 4,500 square, the mean distance 
between *nearest* data-points is around 330, but the default 
epsilon used by the package was 2,500. 

To fix this one could use the method from the original package
(matlab)

  (prod(max(x')-min(x'))/nXCount)^(1/nXDim);

ie, take the volume of the bounding cube, divide by the number 
of points and take the n-th root (n = number of dimensions).

This approach would give 260 for my example above.

Cheers

Jim
-- 
J.J. Green <j.j.green at gmx.fr>



From collinstocks at gmail.com  Thu Jul 21 09:28:05 2011
From: collinstocks at gmail.com (Collin Stocks)
Date: Thu, 21 Jul 2011 09:28:05 -0400
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
 factorization)
In-Reply-To: <CABL7CQhQ=bznSo3EjpHYjVMGM6vgDHCOOb17jOzT4JvW09MHUg@mail.gmail.com>
References: <1311131786.3630.15.camel@SietchTabr>
	<1311193654.3630.67.camel@SietchTabr>
	<1311201312.3630.73.camel@SietchTabr>
	<CABL7CQhQ=bznSo3EjpHYjVMGM6vgDHCOOb17jOzT4JvW09MHUg@mail.gmail.com>
Message-ID: <1311254885.3630.78.camel@SietchTabr>

Thanks for the comment, Ralf. I've made the change you suggested. Also,
there is another change that I made since the revision you commented on
which fixes a bug I had inadvertently caused, whereby trying to use
linalg.qr() with pivoting on a complex matrix would cause a segmentation
fault. This was due to differences in the prototypes of the <s,d>geqp3
and the <c,z>geqp3 LAPACK functions.

I also added tests for this functionality.

I'll just wait another day or two before filing a pull request.

The link, again, for convenience's sake:
https://github.com/collinstocks/scipy/compare/master...qr-with-pivoting

-- Collin
-------------- next part --------------
An embedded message was scrubbed...
From: Ralf Gommers <ralf.gommers at googlemail.com>
Subject: Re: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR	factorization)
Date: Thu, 21 Jul 2011 08:07:28 +0200
Size: 13211
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110721/e62f6417/attachment.mht>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110721/e62f6417/attachment.sig>

From pav at iki.fi  Thu Jul 21 13:43:26 2011
From: pav at iki.fi (Pauli Virtanen)
Date: Thu, 21 Jul 2011 17:43:26 +0000 (UTC)
Subject: [SciPy-Dev] SciPy Central: a file and link sharing site - now
	active
References: <CAGD7iRYoyrsy0tYnj6GORPh+8GxgcE_LQEwOGKqzhz=pms38sQ@mail.gmail.com>
Message-ID: <j09oft$h8h$1@dough.gmane.org>

Hi,

On Thu, 21 Jul 2011 01:17:28 -0400, Kevin Dunn wrote:
> The SciPy Central website is now active and waiting for high-quality
> code snippets and links to scientific resources of interest to the SciPy
> community.

This looks pretty good! Thanks for taking the plunge and getting the site
up to working order, I'm sure this will be an useful resource.

It looks like this could already be a better replacement for the Cookbook
section of the scipy.org site. When it's stabilized a bit, I believe it
would be useful to add links to it from docs.scipy.org and scipy.org.

	Pauli



From collinstocks at gmail.com  Thu Jul 21 21:53:22 2011
From: collinstocks at gmail.com (Collin Stocks)
Date: Thu, 21 Jul 2011 21:53:22 -0400
Subject: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR
 factorization)
In-Reply-To: <1311254885.3630.78.camel@SietchTabr>
References: <1311131786.3630.15.camel@SietchTabr>
	<1311193654.3630.67.camel@SietchTabr>
	<1311201312.3630.73.camel@SietchTabr>
	<CABL7CQhQ=bznSo3EjpHYjVMGM6vgDHCOOb17jOzT4JvW09MHUg@mail.gmail.com>
	<1311254885.3630.78.camel@SietchTabr>
Message-ID: <1311299602.3630.90.camel@SietchTabr>

List,

I have now made a pull request, which can be seen here:
https://github.com/scipy/scipy/pull/44

If you haven't already commented (or even if you have) and you think you
might have some more suggestions, please go there and let me know!

Otherwise, if someone could review the code, that would be particularly
helpful. Specifically, I am not particularly confident in my wrapper
code around <s,d>geqp3 and <c,z>geqp3. I think I got it right, but I am
somewhat less familiar with that format than should be expected of
someone who is editing it!

Thanks,
Collin
-------------- next part --------------
An embedded message was scrubbed...
From: Collin Stocks <collinstocks at gmail.com>
Subject: Re: [SciPy-Dev] QR decomposition with pivoting (rank revealing QR	factorization)
Date: Thu, 21 Jul 2011 09:28:05 -0400
Size: 15511
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110721/44bca9b6/attachment.mht>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110721/44bca9b6/attachment.sig>

From cjordan1 at uw.edu  Fri Jul 22 12:34:06 2011
From: cjordan1 at uw.edu (Christopher Jordan-Squire)
Date: Fri, 22 Jul 2011 11:34:06 -0500
Subject: [SciPy-Dev] Pull Request Review: scipy.optimize doc additions
Message-ID: <CAEJxiFoNJ2dWgZv3LF3ROsDiUAxBk1rDbB1JQfzaQOSVPFXq7w@mail.gmail.com>

Hi--I just submitted a pull request adding notes and references for several
scipy.optimize methods. This is my first scipy pull request, so I'd
appreciate any feedback or additions/modification suggestions.

https://github.com/scipy/scipy/pull/45

-Chris Jordan-Squire
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110722/2b3bfc4d/attachment.html>

From kgdunn at gmail.com  Fri Jul 22 15:11:02 2011
From: kgdunn at gmail.com (Kevin Dunn)
Date: Fri, 22 Jul 2011 15:11:02 -0400
Subject: [SciPy-Dev] SciPy Central: a file and link sharing site - now
	active
Message-ID: <CAGD7iRajXcg=ehAVFqOCu2S-O9NYFneN37xj-cFL=QPEDiPz5A@mail.gmail.com>

> Hi,
>
> On Thu, 21 Jul 2011 01:17:28 -0400, Kevin Dunn wrote:
>> The SciPy Central website is now active and waiting for high-quality
>> code snippets and links to scientific resources of interest to the SciPy
>> community.
>
> This looks pretty good! Thanks for taking the plunge and getting the site
> up to working order, I'm sure this will be an useful resource.

Thanks Pauli.  The Django code that yourself and Jason Grout had up on
GitHub was a good start for me.

> It looks like this could already be a better replacement for the Cookbook
> section of the scipy.org site. When it's stabilized a bit, I believe it
> would be useful to add links to it from docs.scipy.org and scipy.org.

That's the plan. The docs.scipy.org links seem stable, so people
should feel free to start adding them to SciPy Central so other users
searching for terms can be directed to the official docs, in addition
to whatever other results they find.

Maybe someone would like to create a "SciPy Docs" username, so that no
one in particular gets credit for them? If someone takes ownership for
this, I will set the search engine settings to bump those links to the
top of the search results.

In my mind the site is stable enough to start making these additions.
The database will be backed up automatically to an off-site location
every 6 hours (from tonight onwards).

Also, what is the general consensus of transferring the Cookbook items
to SciPy Central. That will also be a lot of work. I started with one
entry (http://scpyce.org/5/), but don't have time right now to look at
the others.

Kevin


From collinstocks at gmail.com  Mon Jul 25 15:04:05 2011
From: collinstocks at gmail.com (collinstocks at gmail.com)
Date: Mon, 25 Jul 2011 15:04:05 -0400
Subject: [SciPy-Dev] f2py
Message-ID: <CAMBdst65dTyz2rjJBshTLxwLPJWJpB3L5J3nz=VxsKzNPg6mow@mail.gmail.com>

Lists (scipy-dev and scipy-user),

If anybody here is at least somewhat familiar with f2py, could you please
review and comment on that section of my pull request: https://github.com/
scipy/scipy/pull/44

Specifically, please take a look at
https://github.com/scipy/scipy/pull/44/files#diff-1

Thanks in advance.

Collin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110725/dfc67bf2/attachment.html>

From cjordan1 at uw.edu  Mon Jul 25 18:16:43 2011
From: cjordan1 at uw.edu (Christopher Jordan-Squire)
Date: Mon, 25 Jul 2011 17:16:43 -0500
Subject: [SciPy-Dev] scipy.linalg asarray_chkfinite
Message-ID: <CAEJxiFrFqTwvPgn3mwOkjfcHLAFp9J8070T8bEz+cigMg6sTBA@mail.gmail.com>

Hi,

I've been trying to speed up a linear least squares solve in scipy and
running into some problems. I was hoping someone could help.

I've been using scipy.linalg.lstsq. But it's slower than I want. Profiling
showed A LOT of time in lstsq was spent just checking that the entries of my
input matrices were finite using asarray_chkfinite. (Typically around 1/3 of
the lstsq function call seems to be checking that.) But I know that the
matrices I'm passing in have finite values.

Also, lstsq uses an svd-based solution method instead of the faster qr
solution method. I had hoped I could write my own qr-based least squares
solver to circumvent both problems, but I found the asarray_chkfinite calls
seem to be in most scipy.linalg functions.

Is there a way around these time consuming checks when calling scipy.linalg
routines?

Thanks,
Chris Jordan-Squire
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110725/0e849120/attachment.html>

From oliphant at enthought.com  Mon Jul 25 18:20:36 2011
From: oliphant at enthought.com (Travis Oliphant)
Date: Mon, 25 Jul 2011 16:20:36 -0600
Subject: [SciPy-Dev] scipy.linalg asarray_chkfinite
In-Reply-To: <CAEJxiFrFqTwvPgn3mwOkjfcHLAFp9J8070T8bEz+cigMg6sTBA@mail.gmail.com>
References: <CAEJxiFrFqTwvPgn3mwOkjfcHLAFp9J8070T8bEz+cigMg6sTBA@mail.gmail.com>
Message-ID: <58AE4B39-2BC4-4874-9B60-9A70CCF8EE69@enthought.com>

Probably not,  but perhaps their should be.  If I recall correctly, non finite values give some bad behavior --- sometimes aborting the process to most of the routines and so the default is to check.

Travis 

--
(mobile phone of)
Travis Oliphant
Enthought, Inc.
http://www.enthought.com

On Jul 25, 2011, at 4:16 PM, Christopher Jordan-Squire <cjordan1 at uw.edu> wrote:

> Hi,
> 
> I've been trying to speed up a linear least squares solve in scipy and running into some problems. I was hoping someone could help. 
> 
> I've been using scipy.linalg.lstsq. But it's slower than I want. Profiling showed A LOT of time in lstsq was spent just checking that the entries of my input matrices were finite using asarray_chkfinite. (Typically around 1/3 of the lstsq function call seems to be checking that.) But I know that the matrices I'm passing in have finite values.
> 
> Also, lstsq uses an svd-based solution method instead of the faster qr solution method. I had hoped I could write my own qr-based least squares solver to circumvent both problems, but I found the asarray_chkfinite calls seem to be in most scipy.linalg functions. 
> 
> Is there a way around these time consuming checks when calling scipy.linalg routines? 
> 
> Thanks,
> Chris Jordan-Squire 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From howarth at bromo.med.uc.edu  Tue Jul 26 10:11:13 2011
From: howarth at bromo.med.uc.edu (Jack Howarth)
Date: Tue, 26 Jul 2011 10:11:13 -0400
Subject: [SciPy-Dev] supported suitesparse versions?
Message-ID: <20110726141113.GA29748@bromo.med.uc.edu>

  We are in the process of updating the fink 10.7 suitesparse package and are
running into a build issue with scipy-0.90 when built against suitesparse-3.6.1...

building 'scipy.sparse.linalg.dsolve.umfpack.__umfpack' extension
compiling C sources
C compiler: gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/include

creating build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve
creating build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack
compile options: '-DNO_ATLAS_INFO=3 -I/sw/lib/python2.6/site-packages/numpy/core/include -I/sw/include/python2.6 -c'
extra options: '-msse3 -I/System/Library/Frameworks/vecLib.framework/Headers'
gcc: build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c
build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c:2351:23: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign]
                  res = SWIG_AddCast(res);
                  ~~~ ^              ~~~
build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c:2354:23: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign]
                  res = SWIG_AddCast(res);                  
                  ~~~ ^              ~~~
build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c:6941:14: warning: explicitly assigning a variable of type 'void *' to itself [-Wself-assign]
  clientdata = clientdata;
  ~~~~~~~~~~ ^ ~~~~~~~~~~
3 warnings generated.
gcc -L/sw/lib -bundle -L/sw/lib/python2.6/config -lpython2.6 -I/sw/include build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.o -L/opt/local/lib -Lbuild/temp.macosx-10.7-x86_64-2.6 -lumfpack -lamd -o build/lib.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/__umfpack.so -Wl,-framework -Wl,Accelerate
Undefined symbols for architecture x86_64:
  "_cholmod_start", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_transpose", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_analyze", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_free_sparse", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_free_factor", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_print_common", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_finish", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_l_start", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_transpose", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_analyze", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_free_sparse", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_free_factor", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_print_common", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_finish", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Undefined symbols for architecture x86_64:
  "_cholmod_start", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_transpose", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_analyze", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_free_sparse", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_free_factor", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_print_common", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_finish", referenced from:
      _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o)
  "_cholmod_l_start", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_transpose", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_analyze", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_free_sparse", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_free_factor", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_print_common", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
  "_cholmod_l_finish", referenced from:
      _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: Command "gcc -L/sw/lib -bundle -L/sw/lib/python2.6/config -lpython2.6 -I/sw/include build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.o -L/opt/local/lib -Lbuild/temp.macosx-10.7-x86_64-2.6 -lumfpack -lamd -o build/lib.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/__umfpack.so -Wl,-framework -Wl,Accelerate" failed with exit status 1
### execution of /var/tmp/tmp.1.p2OWF9 failed, exit code 1

FYI, the build patch we are using for suitesparse is...

diff -uNr SuiteSparse/KLU/Include/klu_internal.h SuiteSparse.patched/KLU/Include/klu_internal.h
--- SuiteSparse/KLU/Include/klu_internal.h      2009-03-24 14:55:04.000000000 -0400
+++ SuiteSparse.patched/KLU/Include/klu_internal.h      2011-07-24 09:50:47.000000000 -0400
@@ -9,7 +9,6 @@
 
 #include "klu.h" 
 #include "btf.h"
-#include "klu_version.h"
 
 /* ========================================================================== */
 /* make sure debugging and printing is turned off */
@@ -36,6 +35,8 @@
 #include <limits.h>
 #include <stdlib.h>
 #include <math.h>
+/* moved klu_version.h after math.h to circumvent compile error on ppc */
+#include "klu_version.h"
 
 #undef ASSERT
 #ifndef NDEBUG
diff -uNr SuiteSparse/UFconfig/UFconfig.mk SuiteSparse.patched/UFconfig/UFconfig.mk
--- SuiteSparse/UFconfig/UFconfig.mk    2011-05-10 16:47:15.000000000 -0400
+++ SuiteSparse.patched/UFconfig/UFconfig.mk    2011-07-24 09:53:37.000000000 -0400
@@ -54,7 +54,7 @@
 F77LIB =
 
 # C and Fortran libraries
-LIB = -lm
+# LIB = -lm
 
 # For compiling MATLAB mexFunctions (MATLAB 7.5 or later)
 MEX = mex -O -largeArrayDims -lmwlapack -lmwblas
@@ -91,8 +91,8 @@
 # naming the BLAS and LAPACK library (*.a or *.so) files.
 
 # This is probably slow ... it might connect to the Standard Reference BLAS:
-BLAS = -lblas -lgfortran
-LAPACK = -llapack
+# BLAS = -lblas -lgfortran
+# LAPACK = -llapack
 
 # NOTE: this next option for the "Goto BLAS" has nothing to do with a "goto"
 # statement.  Rather, the Goto BLAS is written by Dr. Kazushige Goto.
@@ -132,13 +132,13 @@
 # The path is relative to where it is used, in CHOLMOD/Lib, CHOLMOD/MATLAB, etc.
 # You may wish to use an absolute path.  METIS is optional.  Compile
 # CHOLMOD with -DNPARTITION if you do not wish to use METIS.
-METIS_PATH = ../../metis-4.0
-METIS = ../../metis-4.0/libmetis.a
+# METIS_PATH = ../../metis-4.0
+# METIS = ../../metis-4.0/libmetis.a
 
 # If you use CHOLMOD_CONFIG = -DNPARTITION then you must use the following
 # options:
-# METIS_PATH =
-# METIS =
+METIS_PATH =
+METIS =
 
 #------------------------------------------------------------------------------
 # UMFPACK configuration:
@@ -194,7 +194,7 @@
 # -DNSUNPERF       for Solaris only.  If defined, do not use the Sun
 #                      Performance Library
 
-CHOLMOD_CONFIG =
+CHOLMOD_CONFIG = -DNPARTITION
 
 #------------------------------------------------------------------------------
 # SuiteSparseQR configuration:
@@ -208,7 +208,7 @@
 # -DHAVE_TBB        enable the use of Intel's Threading Building Blocks (TBB)
 
 # default, without timing, without TBB:
-SPQR_CONFIG =
+SPQR_CONFIG = -DNPARTITION
 # with timing and TBB:
 # SPQR_CONFIG = -DTIMING -DHAVE_TBB
 # with timing
@@ -328,11 +328,11 @@
 # Macintosh
 #------------------------------------------------------------------------------
 
-# CC = gcc
-# CFLAGS = -O3 -fno-common -no-cpp-precomp -fexceptions
-# LIB = -lstdc++
-# BLAS = -framework Accelerate
-# LAPACK = -framework Accelerate
+CC = gcc
+CFLAGS = -O3 -fno-common -no-cpp-precomp -fexceptions
+LIB = -lstdc++
+BLAS = -framework Accelerate
+LAPACK = -framework Accelerate
 
 #------------------------------------------------------------------------------
 # IBM RS 6000

What is the known versions of suitesparse that are compatible with scipy-0.90?
There seems to be a range of different suitesparse releases in use for Linux
these days but no distribution appears to be using the 3.6.x releases.
              Jack
ps On fink, scipy-0.90 builds fine against suitesparse-3.1.0. MacPorts is
building against suitesparse-3.4.0 which is the latest version that I see
any vendor shipping.


From bsouthey at gmail.com  Wed Jul 27 10:04:27 2011
From: bsouthey at gmail.com (Bruce Southey)
Date: Wed, 27 Jul 2011 09:04:27 -0500
Subject: [SciPy-Dev] scipy.linalg asarray_chkfinite
In-Reply-To: <58AE4B39-2BC4-4874-9B60-9A70CCF8EE69@enthought.com>
References: <CAEJxiFrFqTwvPgn3mwOkjfcHLAFp9J8070T8bEz+cigMg6sTBA@mail.gmail.com>
	<58AE4B39-2BC4-4874-9B60-9A70CCF8EE69@enthought.com>
Message-ID: <4E301AEB.9070401@gmail.com>

On 07/25/2011 05:20 PM, Travis Oliphant wrote:
> Probably not,  but perhaps their should be.  If I recall correctly, non finite values give some bad behavior --- sometimes aborting the process to most of the routines and so the default is to check.
>
> Travis
>
> --
> (mobile phone of)
> Travis Oliphant
> Enthought, Inc.
> http://www.enthought.com
>
> On Jul 25, 2011, at 4:16 PM, Christopher Jordan-Squire<cjordan1 at uw.edu>  wrote:
>
>> Hi,
>>
>> I've been trying to speed up a linear least squares solve in scipy and running into some problems. I was hoping someone could help.
>>
>> I've been using scipy.linalg.lstsq. But it's slower than I want. Profiling showed A LOT of time in lstsq was spent just checking that the entries of my input matrices were finite using asarray_chkfinite. (Typically around 1/3 of the lstsq function call seems to be checking that.) But I know that the matrices I'm passing in have finite values.
>>
>> Also, lstsq uses an svd-based solution method instead of the faster qr solution method. I had hoped I could write my own qr-based least squares solver to circumvent both problems, but I found the asarray_chkfinite calls seem to be in most scipy.linalg functions.
>>
>> Is there a way around these time consuming checks when calling scipy.linalg routines?
>>
>> Thanks,
>> Chris Jordan-Squire
Without knowing what you actually need to solve, have you tried numpy's 
version of lstsq (in numpy.linalg)?
Alternatively, since you know your inputs, create a stripped down 
version of scipy's lstsq function as there are very few lines you would 
need from it.

There is a thread on QR:
http://mail.scipy.org/pipermail/scipy-dev/2011-July/016366.html

The more general issue is that scipy can not automatically assume that 
the input is correct. Note that the scipy.linalg.lstsqr docstring is not 
completely correct since it accept array-like inputs not just arrays 
(due to the fact that numpy.asarray_chkfinite accepts array-like 
inputs). So if you can find a faster way to do this then please file a 
ticket so the community can benefit from it and it not get lost in the list.

Bruce


From cjordan1 at uw.edu  Wed Jul 27 10:54:11 2011
From: cjordan1 at uw.edu (Christopher Jordan-Squire)
Date: Wed, 27 Jul 2011 09:54:11 -0500
Subject: [SciPy-Dev] scipy.linalg asarray_chkfinite
In-Reply-To: <4E301AEB.9070401@gmail.com>
References: <CAEJxiFrFqTwvPgn3mwOkjfcHLAFp9J8070T8bEz+cigMg6sTBA@mail.gmail.com>
	<58AE4B39-2BC4-4874-9B60-9A70CCF8EE69@enthought.com>
	<4E301AEB.9070401@gmail.com>
Message-ID: <CAEJxiFo53mp2uWDH+_CmLPVLvHO6mkmuAwjBeDKps3bV+craww@mail.gmail.com>

On Wed, Jul 27, 2011 at 9:04 AM, Bruce Southey <bsouthey at gmail.com> wrote:

> On 07/25/2011 05:20 PM, Travis Oliphant wrote:
> > Probably not,  but perhaps their should be.  If I recall correctly, non
> finite values give some bad behavior --- sometimes aborting the process to
> most of the routines and so the default is to check.
> >
> > Travis
> >
> > --
> > (mobile phone of)
> > Travis Oliphant
> > Enthought, Inc.
> > http://www.enthought.com
> >
> > On Jul 25, 2011, at 4:16 PM, Christopher Jordan-Squire<cjordan1 at uw.edu>
>  wrote:
> >
> >> Hi,
> >>
> >> I've been trying to speed up a linear least squares solve in scipy and
> running into some problems. I was hoping someone could help.
> >>
> >> I've been using scipy.linalg.lstsq. But it's slower than I want.
> Profiling showed A LOT of time in lstsq was spent just checking that the
> entries of my input matrices were finite using asarray_chkfinite. (Typically
> around 1/3 of the lstsq function call seems to be checking that.) But I know
> that the matrices I'm passing in have finite values.
> >>
> >> Also, lstsq uses an svd-based solution method instead of the faster qr
> solution method. I had hoped I could write my own qr-based least squares
> solver to circumvent both problems, but I found the asarray_chkfinite calls
> seem to be in most scipy.linalg functions.
> >>
> >> Is there a way around these time consuming checks when calling
> scipy.linalg routines?
> >>
> >> Thanks,
> >> Chris Jordan-Squire
> Without knowing what you actually need to solve, have you tried numpy's
> version of lstsq (in numpy.linalg)?
> Alternatively, since you know your inputs, create a stripped down
> version of scipy's lstsq function as there are very few lines you would
> need from it.
>
> There is a thread on QR:
> http://mail.scipy.org/pipermail/scipy-dev/2011-July/016366.html
>
> The more general issue is that scipy can not automatically assume that
> the input is correct. Note that the scipy.linalg.lstsqr docstring is not
> completely correct since it accept array-like inputs not just arrays
> (due to the fact that numpy.asarray_chkfinite accepts array-like
> inputs). So if you can find a faster way to do this then please file a
> ticket so the community can benefit from it and it not get lost in the
> list.
>
>
Thanks for the link. It's instructive, but it looks like they're mostly
talking about a different qr algorithm.

I hadn't thought of using the numpy version, but I prefer to use the scipy
version to make sure the functions are using the correct lapack.

Later this week I'm going to add a flag to several of the scipy.linalg
functions to allow the caller to disable the checking if they desire. This
will replace the asarray_chkfinite with asarray, which will also take
array-likes, but will pass pre-existing numpy arrays straight through. This
should speed things up a lot if you're passing in a numpy array that's
already been verified as having finite entries.

I experimented a bit yesterday, and it appears that the following is true.
If you pass a numpy array with a NaN in it to lstsq (after removing the
chkfinite command), then it fails and gives back arrays full of nan's. But
if you pass a numpy array with inf's in it, then it hangs. With qr, if you
pass a NaN you again get an array of NaN's back, but if you pass in an inf
you get something back (i.e. no error), but it has NaN's and Inf's in the
matrix. So the user can check for and catch that error.

-Chris Jordan-Squire










> Bruce
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110727/3e3dbe74/attachment.html>

From fperez.net at gmail.com  Sun Jul 31 13:19:51 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 31 Jul 2011 12:19:51 -0500
Subject: [SciPy-Dev] [ANN] IPython 0.11 is officially out
Message-ID: <CAHAreOpeS1yWz_dg7+HYwyCLvL12DZ3JwoWb_6nHOq4eMNGd+w@mail.gmail.com>

Hi all,

on behalf of the IPython development team, I'm thrilled to announce,
after more than two years of development work, the official release of
IPython 0.11.

This release brings a long list of improvements and new features
(along with hopefully few new bugs).  We have completely refactored
IPython, making it a much more friendly project to participate in by
having better separated and organized internals.  We hope you will not
only use the new tools and libraries, but also join us with new ideas
and development.

After this very long development effort, we hope to make a few
stabilization releases at a quicker pace, where we iron out the kinks
in the new APIs and complete some remaining internal cleanup work.  We
will then make a (long awaited) IPython 1.0 release with these stable
APIs.

*Downloads*
Download links and instructions are at: http://ipython.org/download.html
And IPython is also on PyPI: http://pypi.python.org/pypi/ipython

Those contain a built version of the HTML docs; if you want pure
source downloads with no docs, those are available on github:

Tarball: https://github.com/ipython/ipython/tarball/rel-0.11
Zipball: https://github.com/ipython/ipython/zipball/rel-0.11

* Features *
Here is a quick listing of the major new features:

- Standalone Qt console
- High-level parallel computing with ZeroMQ
- New model for GUI/plotting support in the terminal
- A two-process architecture
- Fully refactored internal project structure
- Vim integration
- Integration into Microsoft Visual Studio
- Improved unicode support
- Python 3 support
- New profile model
- SQLite storage for history
- New configuration system
- Pasting of code with prompts

And many more...

We closed over 500 tickets, merged over 200 pull requests, and more
than 60 people contributed over 2200 commits for the final release.
Please see our release notes for the full details on everything about
this release: https://github.com/ipython/ipython/zipball/rel-0.11

As usual, if you find any problem, please file a ticket --or even
better, a pull request fixing it-- on our github issues site
(https://github.com/ipython/ipython/issues/).

Many thanks to all who contributed!

Fernando, on behalf of the IPython development team.

http://ipython.org


From gustavo.goretkin at gmail.com  Sun Jul 31 20:16:10 2011
From: gustavo.goretkin at gmail.com (Gustavo Goretkin)
Date: Sun, 31 Jul 2011 20:16:10 -0400
Subject: [SciPy-Dev] setup.py develop
Message-ID: <CAD+Uu3P5NXz-1bzhiEBqjEsXZ5bGXaSkNokr9ST_X9qxOHB7Aw@mail.gmail.com>

Hey Devs,

I'm making some tiny local changes to SciPy and I'd like to install it in
virtualenv using setup.py develop (as opposed to setup.py install) as I've
done for other projects. What's the way to do this for SciPy?

Thanks,
Gustavo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110731/a04be962/attachment.html>