[SciPy-user] OLS matrix-f(x) = 0 problem (Was: linear regression)

[Gael Varoquaux]

On Wed, May 27, 2009 at 06:27:29PM -0400, josef.pktd at gmail.com wrote:
> Sounds like a recursive system of linear (simultaneous) equations with
> linear restrictions to me. If you want an unbiased estimator, then
> going row by row, and solving each linear OLS, linalg.lstsq, would be
> the standard way to go. Substuting the previous estimates of the Y's
> into the next step.

Oups, I realise I forgot to answer.

You are right, this is a way to interpret it, and I was solving the
system as you suggest. What didn't like is that the solution I was
getting was dependant on the order of the variables, but I had forgotten
that the lower triangular matrix was an approximation. The 
non-permutation-invariance came from this approximation, not the way I
was solving the system.

Unfortunately, it seems that the solution to the complete problem is
still an open research question (FYI the problem is to find the OLS
solution to "M X = X + e", with M definite positive, and with a given
support.

X's dimension are everywhere between (50, 50) to (300, 500), including
the bad situation (300, 50).

This is related sparse covariance matrix estimation. I don't think there
is (yet) an easy answer.

Thanks for your answer, it brought me back to Earth, making me realize
that I was already doing the right thing, and look for the problem
elsewhere.

Gaël