[Tutor] [OT] ETL Tools

Stephen Nelson-Smith sanelson at gmail.com
Fri Mar 30 09:27:54 CEST 2007


Hello all,

Does anyone know of any ETL (Extraction, Transformation, Loading)
tools in Python (or at any rate, !Java)?

I have lots (and lots) of raw data in the form of log files which I
need to process and aggregate and then do a whole bunch of group-by
operations, before dumping them into text/relational database for a
search engine to access.

At present we have a bunch of scripts in perl and ruby, and a berkley
and mysql database for the grouping operations.  This is proving to be
a little slow with the amount of data we now have, so I am looking
into alternatives.

Does anyone have any experience of this sort of  thing?  Or know
someone who does, that I could talk to?

Best regards,

S.


More information about the Tutor mailing list