Too many open files

AMD amdescombes at gmail.com
Mon Feb 4 07:57:39 EST 2008


Hello,

I need to split a very big file (10 gigabytes) into several thousand 
smaller files according to a hash algorithm, I do this one line at a 
time. The problem I have is that opening a file using append, writing 
the line and closing the file is very time consuming. I'd rather have 
the files all open for the duration, do all writes and then close them 
all at the end.
The problem I have under windows is that as soon as I get to 500 files I 
get the Too many open files message. I tried the same thing in Delphi 
and I can get to 3000 files. How can I increase the number of open files 
in Python?

Thanks in advance for any answers!

Andre M. Descombes



More information about the Python-list mailing list