regd efficient methods to manipulate *large* files

[Paddy]

I take it that you have a binary file that takes a file name and
proceses the file contents.
Sometimes Unix binaries are written so that a file name of '-', (just a
dash), causes it to take input from stdin so that the piping mentioned
in a previous reply could work.
On some of our unix systems /tmp is set up as a 'virtual disk' It
quacks like a normal disk filesystem but is actually implimented in
RAM/virtual memory, and is faster than normal disk access.
(Unfortunately we are not allowed to save multi-gigabyte files there as
it affects other aspects of the OS).

Maybe you can mount a similar filesystem if you have the RAM.

-- Pad.