[SciPy-User] timeseries forwardfill

[Christopher Clarke]

Hi
Something seems to have gone wrong with my initial reply!!
Anyway, I often encounter the "initial values" use case when I am creating
business day time series out of RDBMS tables using a subset of the
observations in the table.  For example i have a SQL query fragment like
WHERE symbol='SFC' and dateix BETWEEN '2009-01-01' AND '2009-09-01'
Now suppose that 2009-01-02 and 2009-01-05 are missing (trading is sparse on
many of the exchanges i'm dealing with) i am supposed to forward_fill using
the last traded value for SFC which may or may not be 2008-12-31.  Hence i
have a query that find the last traded values and i use these as the
"initial values".

Anyway here is by forward_fill wrapper. Its not very efficient as i'm
copying and forward_fill is copying etc but..
 I'm actually starting to have reservation about the usefulness forward_fill
on 2d as opposed to  the individual the individual series arrays as i am
finding that i've often got to do loads of transformations and checking on
the individuals arrays before i can combine them into  a single ma array for
filling anyway


def forward_fill2(marr,maxgap=None,init_vals=None):
  """
  init_vals a list with  the same no. of elements as marr.shape[1]
  """
    arr=ma.array(marr,copy=True
    if arr.ndim == 1:
        if init_vals:
            if arr.mask.any() and arr.mask[0]:
                if init_vals:
                    arr[0] = init_vals[0]

        return forward_fill(arr,maxgap)
    else:
        n = arr.shape[1]
        if init_vals:
            mask=ma.getmask(arr)
            if len(init_vals) != n:
                raise ValueError, 'Initial Values sequence does no match
number of columns'
            for c in range(len(init_vals)):
                if arr.mask.any() and mask[0,c]:
                        if init_vals[c]:
                            arr[0,c]=init_vals[c]
        arr = ma.hsplit(arr, n)[0]
        return ma.column_stack([forward_fill(np.squeeze(a),maxgap) for a in
arr])



On Thu, Nov 19, 2009 at 5:07 AM, Chris Clarke <cclarke at chrisdev.com> wrote:

> Hi
> The "initial value"
>
> On Nov 18, 2009, at 9:17 PM, Pierre GM wrote:
>
>
> On Nov 18, 2009, at 5:18 PM, Chris Clarke wrote:
>
> Sorry for the later reply.  Yes forward_fill is still there and it
>
> works!!!
>
>
> Good
>
> But it seemed to have some more capability (initial values, 2d arrays)
>
> when  it was in the sandbox??
>
> I may be wrong and mixing up with some other library.
>
>
> That does sound familiar, but i don't think it was part of
> scikits.timeseries...
> A patch for 2D would be welcome, I'm not quite sure what you mean by
> initial value, though
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20091123/095fb31e/attachment.html>