Parallel/Multiprocessing script design question

[Paddy]

Hi Amit,
Why not create a list of those 800+ files and a sccript that when run,
looks for an environment variable that will be a number from 1 to 800,
selectss the file at that line number and processes it fully.

For the process control install a job scheduler such as LSF or the Sun
grid Engine http://en.wikipedia.org/wiki/Sun_Grid_Engine and use JOB
ARRAYS:
http://docs.sun.com/app/docs/doc/820-0699/6nce0ht7s?l=en&a=view#i999498

- Paddy.