[Numpy-discussion] incremental histogram

[Robert Kern]

On Wed, May 7, 2014 at 3:22 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> I needed a histogram that is built incrementally.  My need is for 1D only.
>
> The idea is to not require storage of all the data (assume it could be too
> large).
>
> This is a naive implementation, perhaps someone could suggest something better.
>
> ,----[ /home/nbecker/sigproc.ndarray/histogram3.py ]
> | import numpy as np
> |
> | class histogram (object):
> |     def __init__ (self, nbins):
> |         self.nbins = nbins
> |         self.centers = []
> |         self.counts = []
> |     def __iadd__ (self, x):
> |         self.counts, edges = np.histogram (
> |             np.concatenate ((x, self.centers)),
> |             weights = np.concatenate ((np.ones (len(x)), self.counts)),
> |             bins=self.nbins)

That's just begging for subtle aliasing issues as the data range
increases and the bins shift around. Instead, consider keeping the bin
width and origin fixed and append or prepend bins as needed if the new
data falls outside of the original bin edges.

-- 
Robert Kern