Answer by nwsteg for Incremental averaging

Here is a solution similar to @Michal Wolodzko but with a constant batch size $w$. I developed this because I need to take an average of a large sequence of high resolution images. With this method, I can load $w$ images at a time and update the running average of the images. This is useful for computing the a "background" state which can be used for further image analysis. The following derivation is for a sequence of numbers, but you can easily generalize it to a sequence of images with pixel intensities stored in $m \times n$ arrays.

Consider a sequence of numbers $x_0,...,x_{N-1}$.

Define the following:

• $N$ length of the sequence
• $w$ width or batch size
• $N_b = \text{floor }(N/w)$ number of batches
• $N_r = \mod(N/w)$ remaining elements that don't fit evenly into a batch
• $\overline{S}_k = \frac{1}{w}\sum_{i=kw}^{(k+1)w-1} x_i$ average over batch $k$ where $0\leq k \leq N_b-1$
• $\overline{S}_{r} = \frac{1}{N_r}\sum_{i=Nw}^{N-1} x_i$ average over the remaining elements
• $\overline{S}_{0:k} = \frac{1}{k}\sum_{i=0}^{k-1} \overline{S}_k$ average of the batch averages
• $\overline{S} = \frac{1}{N}\sum_{i=0}^{N-1}x_i$ the average over all the elements

Note that$$\begin{align*}\overline{S}_{0:k} &= \frac{1}{k} \sum_{i=0}^{k-1} \overline{S}_k\\&= \frac{1}{kw}\left(\overline{S}_0w + \overline{S_1}w + \cdots + \overline{S}_{k-1}w\right)\\&= \text{average of $\{x_i\}$ up to element $kw$}\end{align*}$$so we can get the running average $\overline{S}_{0:k}$ by just averaging the batch averages.

Now, we don't want to store all the batch averages. Instead, we want to update the running average with each newly computed batch average. Consider$$\begin{align*}\overline{S}_{0:k+1}&=\frac{1}{k+1}\sum_{i=0}^k \overline{S}_i\\&= \frac{1}{k+1}\left(\sum_{i=0}^{k-1} \overline{S_i} + \overline{S_k}\right)\\&= \frac{1}{k+1}\left(\overline{S}_{0:k}k + \overline{S_k}\right)\\&= \frac{k}{k+1}\overline{S}_{0:k} + \frac{1}{k+1}\overline{S_k}\end{align*}$$Thus we can just update the running average by taking a weighted average of what we've computed so far. The last step is to compute update the average with the awkward number of remaining elements $N_r$.$$\begin{align*}\overline{S} &= \frac{1}{N}\left(\overline{S}_{0:N_b-1}N_bw + \overline{S}_rN_r\right)\\&= \frac{N_bw}{N}\overline{S}_{0:N_b-1} + \frac{N_r}{N}\overline{S}_r\end{align*}$$Overall, this method is nice because we have no risk of overflow (computing an excessively large sum) and no need to load all the numbers (or images) into memory at once.

Here's some python code demonstrating the method.

import numpy as npimport math# replace this with a sequence of images# only need to load (w) at a timex=np.random.rand(37)w=5 # batch sizeN=len(x) # length of sequenceNb=math.floor(N/w) # number of batch averagesNr=N%w # remaining elements to averageSrun=0 # running averagefor k in range(0,Nb):    Sk=np.mean(x[k*w:(k+1)*w]) # batch average    Srun=Srun*(k/(k+1))+Sk/(k+1) # updated running averageSr = np.mean(x[Nb*w:]) # average over the remaining elementsSrun=Srun*(Nb*w/N)+Sr*(Nr/N) # final running average# compare the running averageprint('Numpy: ', x.mean())print('Running: ', Srun)

with output

Numpy:  0.4537805990791784Running:  0.45378059907917856