Very Slow Disk Writes when Writing Large Data Blocks

Irmen de Jong irmen.NOSPAM at xs4all.nl
Fri Jun 2 15:29:10 EDT 2017


On 2-6-2017 20:14, remmm wrote:

> These write speeds are in the range of 18 to 25 MBytes per second for spinning disks and about 50 Mbytes/sec for SSDs.  Keep in mind these numbers should be more like 120 MBytes/sec for spinning disks and 300 MBytes/sec for SSDs.  

You'll only reach those numbers in the ideal situation. Is there just one program doing
this disk i/o, sequentially, from a single thread?
If it is not, you are probably suffering disk i/o trashing once you filled up the
drive's cache buffers.

For example using Crystal Disk Mark on one of my HDD drives it reports max 60 MBytes/sec
write speed sequentially in the ideal case (ok, it is not a very fast drive...) but only
0.7 (!) in the random 4k block case.

Apparently Linux deals with this better than Windows, for your situation.

Other than that the only other thing I can think of is interference of other programs on
the system, such as malware protection or anti-virus tooling that is trying to scan your
big files at the same time.  That should be visible in Window's resource monitor tool
however, I think.


> I've since written test code using just python "write" 

Post it somewhere?

Irmen



More information about the Python-list mailing list