Iterating through a file significantly slower when file has big buffer

Roger Binns rogerb at rogerbinns.com
Tue Jan 27 05:25:48 EST 2009


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

python at bdurham.com wrote:
> The following tests were run on a Windows XP system using Python 2.6.1

Unless you changed the defaults, the Windows XP system cache size is
10MB.  When you use a larger read size, chances are it is blowing out
that cache and causes metadata (file block locations) to have to be
reread on your next read.

You are also funnelling all the data through your CPU cache with a
similar effect although it will be less noticeable.

To change XP cache sizes, see:

 http://marc.info/?l=sqlite-users&m=116743785223905&w=2
 http://support.microsoft.com/kb/895932
 http://www.techspot.com/tweaks/memory-winxp/

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkl+4SkACgkQmOOfHg372QRgcACfVOdUWQGyj8xtNvHob/CtcM8g
JsEAoKt/xI36iR5RuQOfZDMz2ze4L3Ia
=DrDw
-----END PGP SIGNATURE-----




More information about the Python-list mailing list