Fatest standard way to sum bytes (and their squares)?

Erik Max Francis max at alcyone.com
Sun Aug 12 05:26:59 EDT 2007


For a file hashing system (finding similar files, rather than identical 
ones), I need to be able to efficiently and quickly sum the ordinals of 
the bytes of a file and their squares.  Because of the nature of the 
application, it's a requirement that I do it in Python, or only with 
standard library modules (if such facilities exist) that might assist.

So far the fastest way I've found is using the `sum` builtin and 
generators::

	ordinalSum = sum(ord(x) for x in data)
	ordinalSumSquared = sum(ord(x)**2 for x in data)

This is about twice as fast as an explicit loop, but since it's going to 
be processing massive amounts of data, the faster the better.  Are there 
any tricks I'm not thinking of, or perhaps helper functions in other 
modules that I'm not thinking of?

-- 
Erik Max Francis && max at alcyone.com && http://www.alcyone.com/max/
  San Jose, CA, USA && 37 20 N 121 53 W && AIM, Y!M erikmaxfrancis
   Whoever named it necking was a poor judge of anatomy.
    -- Groucho Marx



More information about the Python-list mailing list