[SciPy-dev] time series implementation approach

[David Huard]

For the record:
I use time series to
1. Convert daily data (precip, ET, disharges) to aggregated monthly data.
2. Extract the longest continuous series from a gappy series (sometimes
there are dates but missing values (-99 for instance), and sometimes the
dates themselves are missing).

Wich list: To be able to apply a method on logical chunks of data. For
instance, from a daily series, compute the monthly mean and variance. Or is
that already implemented ?

David Huard

2006/12/13, Pierre GM <pgmdevlist at gmail.com>:
>
> David,
> Merci beaucoup pour les sources.
> I guess a lot of people in this list have their own implementation of
> timeseries...
>
> Storing a date along data gives indeed the possibility to deal with gapped
> series. However, things becomes quite messy when you have to combine
> several
> series: how do you deal with the gaps? Your approach (for the glance I
> got)
> assumes either a step or a linear interpolation, which should work nicely
> in
> your case, but is doubtfully applicable to other cases.
>
> In fact, you just gave me an idea. In maskedarrays, the mask can be nomask
> (viz, no masked data, surprise), which greatly simplifies most operations.
> Here, we could have a frequency of nofreq, which would indicate that the
> time
> step is not constant: one simple option is then to cast it to the smallest
> reasonable timestep, and set the missing  values to "masked". (By
> reasonable,
> I mean average: if most of your data is roughly monthly but for a couple
> of
> daily ones, stick to monthly. Unless you really want to go daily. Dammit,
> we
> gonna have to leave this possibility).
>
> So yeah, we may eventually have to consider varying frequencies. Which
> would
> mean that the shifting array approach will have to be reexamined. Maybe
> not:
> if freq is nofreq, then it'll panic, for sure. But if freq is set, then it
> remains a quite viable option.
>
> OK, now I stop thinking aloud. David, could you tell us more about what
> you
> need ? What kind of data do you have to work with ? Are missing dates
> something you have to deal with on a very regular basis ?
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20061213/e96d989b/attachment.html>