[SciPy-User] Generalized least square on large dataset

[josef.pktd at gmail.com]

On Wed, Mar 7, 2012 at 9:39 PM, Peter Cimermančič
<peter.cimermancic at gmail.com> wrote:
> Hi,
>
> I'd like to linearly fit the data that were NOT sampled independently. I
> came across generalized least square method:
>
> b=(X'*V^(-1)*X)^(-1)*X'*V^(-1)*Y
>
>
> X and Y are coordinates of the data points, and V is a "variance matrix".
>
> The equation is Matlab format - I've tried solving problem there too, bit it
> didn't work - but eventually I'd like to be able to solve problems like that
> in python. The problem is that due to its size (1000 rows and columns), the
> V matrix becomes singular, thus un-invertable. Any suggestions for how to
> get around this problem? Maybe using a way of solving generalized linear
> regression problem other than GLS?

V is (nobs,nobs) has nobs*(nobs-1)/2 parameter (or something like
this) (nobs number of observations rows)

I don't think there is a general solution without imposing a lot of
structure on V that reduces the effective number of parameters.
(singular matrix in itself is not necessarily a problem using pinv or
a small Ridge penalty)

Most of the solutions I'm looking for for GLS is for cases where V is
a kronecker product (or block matrix) or where
V^{-0.5} (the matrix to transform the X) has a nice solution. I only
looked at a few special cases so far.

There are some papers that claim that they have an efficient cholesky
of the V inverse for mixed effects models for example but I never
looked at the details.

General rule: if you manage to get it to work in matlab, then it
should be possible to get it to work in numpython (unless ... which
shouldn't apply in this case)

I think the main question is: What kind of V matrix do you have? What
kind of violation if independent sampling?

I'm also still struggling with this question, and with which linear
algebra to use, so I'm also interested in any solutions.

Josef

>
> Regards,
>
> Peter
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>