aggregation for a nested dict

chris ozric at web.de
Thu Dec 2 14:01:05 EST 2010


Hi,

i would like to parse many thousand files and aggregate the counts for
the field entries related to every id.

extract_field grep the identifier for the fields with regex.

result = [ { extract_field("id", line) : [extract_field("field1",
line),extract_field("field2", line)]}  for line  in FILE ]

result gives me.
{'a: ['0', '84']},
{'a': ['0', '84']},
{'b': ['1000', '83']},
{'b': ['0', '84']},

i like to aggregate them for every line or maybe file and get after
the complete parsing procedure
the possibility to count the amount of ids  having > 0 entries in
'83'.

{'a: {'0':2, '84':2}}
{'b': {'1000':1,'83':1,'84':1} }

My current solution with mysql is really slow.

Many thanks for advance.
Christian








More information about the Python-list mailing list