Python object overhead?

[Bruno Desthuilliers]

Matt Garman a écrit :
(snip)
> Also, many folks have suggested operating on only one line at a time
> (i.e. not storing the whole data set).  Unfortunately, I'm constantly
> "looking" forward and backward in the record set while I process the
> data (i.e., to process any particular record, I sometimes need to know
> the whole contents of the file).  (This is purchased proprietary
> vendor data that needs to be converted into our own internal format.)

Don't know if this could solve your problem, but have considered using 
an intermediate (preferably embedded) SQL database (something like 
SQLite) ?