Creating huge data in very less time.

[Tim Chase]

venutaurus539 at gmail.com wrote:
> On Mar 31, 1:15 pm, Steven D'Aprano
>> The fastest HDDs can reach about 125 MB per second under
>> ideal circumstances, so that will take at least 8 seconds
>> per 1GB file or 8000 seconds in total.
> 
> That time is reasonable. 

You did catch the bit about "the *fastest* HDDs" (my emphasis). 
Unless you've got some massive RAID or Gig-E/Fiberchanel SAN, you 
likely don't have these ideal conditions.  Additionally, I've 
seen "125MB/sec" as the read speeds -- sustained write speeds are 
often lower.  Under more real-world testing, you'll likely get 
throughput closer to 30-70MB/sec.  Call that roughly half the 
throughput, and you're up to 16,000 seconds, or about 4.5hr.  A 
far cry from the "few minutes of time" you first mentioned...

And this doesn't take into consideration the OS overhead of the 
filesystem type.  Some filesystem types are optimized for large 
sequential access, while others work better with smaller files. 
You then have things like OS permission overhead, directory-path 
overhead, and other disk I/O going on at the same time.

-tkc