Weight

Many OnlineStats are parameterized by a Weight that controls the influence of new observations. If the OnlineStat is capable of calculating the same result as a corresponding offline estimator, it will have a keyword argument weight. If the OnlineStat uses stochastic approximation, it will have a keyword argument rate.

Consider how weights affect the influence of the next observation on an online mean $\theta^{(t)}$, as many OnlineStats use updates of this form. A larger weight $\gamma_t$ puts higher influence on the new observation $x_t$:

\[\theta^{(t)} = (1-\gamma_t)\theta^{(t-1)} + \gamma_t x_t\]

Note

The values produced by a Weight must follow two rules:

$\gamma_1 = 1$ (guarantees $\theta^{(1)} = x_1$)
$\gamma_t \in (0, 1), \quad \forall t > 1$ (guarantees $\theta^{(t)}$ stays inside a convex space)

Weight Types

OnlineStatsBase.EqualWeight — Type.

EqualWeight()

Equally weighted observations.

$γ(t) = 1 / t$

source

OnlineStatsBase.ExponentialWeight — Type.

ExponentialWeight(λ::Float64)
ExponentialWeight(lookback::Int)

Exponentially weighted observations. The first weight is 1.0 and all else are λ = 2 / (lookback + 1).

$γ(t) = λ$

source

OnlineStatsBase.LearningRate — Type.

LearningRate(r = .6)

Slowly decreasing weight. Satisfies the standard stochastic approximation assumption $∑ γ(t) = ∞, ∑ γ(t)^2 < ∞$ if $r ∈ (.5, 1]$.

$γ(t) = inv(t ^ r)$

source

OnlineStatsBase.HarmonicWeight — Type.

HarmonicWeight(a = 10.0)

Weight determined by harmonic series.

$γ(t) = a / (a + t - 1)$

source

OnlineStatsBase.McclainWeight — Type.

McclainWeight(α = .1)

Weight which decreases into a constant.

$γ(t) = γ(t-1) / (1 + γ(t-1) - α)$

source

Weight wrappers

OnlineStatsBase.Bounded — Type.

Bounded(w::Weight, λ::Float64)

Bound the weight by a constant.

$γ′(t) = max(γ(t), λ)$

source

OnlineStatsBase.Scaled — Type.

Scaled(w::Weight, λ::Float64)

Scale a weight by a constant.

$γ′(t) = λ * γ(t)$

source

Custom Weighting

The Weight can be any callable object that receives the number of observations as its argument. For example:

weight = inv will have the same result as weight = EqualWeight().
weight = x -> x == 1 ? 1.0 : .01 will have the same result as weight = ExponentialWeight(.01)


julia> y = randn(100);

julia> fit!(Mean(weight = EqualWeight()), y)
Mean: n=100 | value=-0.165585

julia> fit!(Mean(weight = inv), y)
Mean: n=100 | value=-0.165585

julia> fit!(Mean(weight = ExponentialWeight(.01)), y)
Mean: n=100 | value=-0.416768

julia> fit!(Mean(weight = x -> x == 1 ? 1.0 : .01), y)
Mean: n=100 | value=-0.416768