I ran into a similar situation with a massive directory of PIL generated images (around 10k). No problems on the filesystem/Python side of things but other tools (most noteably 'ls') don't cope very well. As it happens my data has natural groups so I broke the big dir into subdirs to sidestep the problem.