[Chicago] Easy Reports?

Carl Karsten carl at personnelware.com
Wed Aug 28 18:01:20 CEST 2013


On Tue, Aug 27, 2013 at 10:50 PM, Don Drake <don at drakeconsulting.com> wrote:
> Pandas is an option for a python approach, but you will probably run out of
> memory if you need analyze anything close to your 200GB of data on a single
> machine.

No clue if this will work for 200G.. but here a talk about it:

http://youtu.be/D1x_O36HZwo?t=17m42s

https://github.com/rduplain/expression_file

Lazy Pandas
An experiment in lazily loading data into a Python pandas v0.12.0 DataFrame.

Load the data into pandas.
... without aggressively consuming memory.
Too large to fit into memory?

http://slid.es/rduplain/lazy-pandas

-- 
Carl K


More information about the Chicago mailing list