Too many open files

[Larry Bates]

AMD wrote:
> Hello,
> 
> I need to split a very big file (10 gigabytes) into several thousand 
> smaller files according to a hash algorithm, I do this one line at a 
> time. The problem I have is that opening a file using append, writing 
> the line and closing the file is very time consuming. I'd rather have 
> the files all open for the duration, do all writes and then close them 
> all at the end.
> The problem I have under windows is that as soon as I get to 500 files I 
> get the Too many open files message. I tried the same thing in Delphi 
> and I can get to 3000 files. How can I increase the number of open files 
> in Python?
> 
> Thanks in advance for any answers!
> 
> Andre M. Descombes

Not quite sure what you mean by "a hash algorithm" but if you sort the file 
(with external sort program) on what you want to split on, then you only have to 
have 1 file at a time open.

-Larry