Can you do it faster? (parsing text file)
Andrew Bennetts
andrew-pythonlist at puzzling.org
Tue Jan 28 06:26:03 EST 2003
On Tue, Jan 28, 2003 at 11:16:04AM +0000, Marcus Stojek wrote:
> Hi,
>
> I have to parse large txt-files (15Mb and more, 200000 lines).
> The file consits of comma separated keywords and alphanumerical data.
While I see you have some quirky requirements, I'd recommend:
- Use the CSV to parse the fields:
http://www.object-craft.com.au/projects/csv/
- Read the file with "for line in myFile:" (you're using Python 2.2, I
presume?)
I suspect with those you could get a nice speed boost. You could also try
mucking around with Psyco and/or the Python 2.3 alpha (Python 2.3 is faster
than older versions).
There's various other micro-optimisations that could be made, but they're
probably not worth it.
-Andrew.
More information about the Python-list
mailing list