[SciPy-dev] timeseries tdates.py

[Matt Knox]

> I am pretty excited about this package as i see it replacing FAME  
> TimeIQ and my own hodgepodge stuff

I am in the same situation here. I rely on FAME extensively for financial data,
and would like to decrease that reliance. There are two hurdles to overcome for
that... I need something I can manipulate time series with (and the work for
that is well under way in the timeseries module Pierre and I have been working
on), and I need something to store the TimeSeries objects in.

I have actually developed a python wrapper to a bunch of the FAME c-api stuff
which reads/writes to/from FAME databases using the TimeSeries objects in the
module we are writing. I'm still polishing some things up with that currently,
but I plan to release it at some point (perhaps as part of an IO submodule in
the timeseries module).

> I am interested in the datetime as i want to store the "raw" data in  
> a generic RDBMS (OK PostgreSQL/MS SQlServer  for now)

I'm not a fan of the idea of storing time series data in a relational db.
Performance is gonna suck unless you are storing TimeSeries as binary objects...
but that has a whole host of problems too. I think FAME is still the best
storage solution (that I am aware of) for this kind of data currently.

That being said... I think it would be feasible (although not entirely trivial)
to write a module specifically for storing TimeSeries in pytables. If that was
done, this could potentially trump FAME both in performance, and flexibility. As
a first step, something would have to be written to work with masked arrays in
pytables. It's probably worth waiting for pytables 2.0 (with numpy at its core)
before any work is done on this though. I think a good python timeseries module,
in combination with pytables support, would be a very compelling piece of
software capable of going toe-to-toe with some expensive commericial offerings
out there (ie. FAME) (feature-wise, if not reliability and support-wise).

> I am new to your timeseries library so i may be completely off base  
> here.
> But rrule allows you to configure new/strange  "frequencies" (at  
> least the dates that should be in the calendar)
> For example many of the emerging stock markets trading only on  
> tuesday, wednesday and friday
> How do i go about defining new frequencies? do i need to do so in C?

The tricky part with new frequencies is supporting converting series at one
frequency to another frequency. If you never need to convert from one frequency
to another, it would be very easy to add different frequencies. I don't *think*
rrule would be of any help in this regard, although I am a relative simpleton
and I often miss obvious things :). Pierre and I had talked about adding some
flexibility for defining custom frequencies, but nothing concrete has been
hammered out yet.

> PS. Have you seen the scipy 02  paper by Reggie Duggard  ???

I have not seen it, but would be interested in taking a look if you have a copy
available.

Also, as Pierre mentioned, we are planning to make an announcement regarding
availability of an alpha stage version of the timeseries module in the near
future. The items pushing that back right now are updating tseries.py to reflect
the major overhaul of maskedarray that Pierre did the other day (it actually
won't work right now if you try to use the latest version of maskedarray from
the sandbox in svn), and reviewing some axis labelling behaviour with the
plotlib sub-module and adding some plotting examples to the wiki. There are
other things to do beyond that obviously, but those are the main things holding
up the alpha release (correct me if you disagree here Pierre).

- Matt