# Weights

Many `OnlineStat`

s are parameterized by a `Weight`

that controls the influence of new observations. If the `OnlineStat`

is capable of calculating the same result as a corresponding offline estimator, it will have a keyword argument `weight`

. If the `OnlineStat`

uses stochastic approximation, it will have a keyword argument `rate`

.

Consider how weights affect the influence of the next observation on an online mean $\theta^{(t)}$, as many `OnlineStat`

s use updates of this form. A larger weight $\gamma_t$ puts higher influence on the new observation $x_t$:

The values produced by a `Weight`

must follow two rules:

- $\gamma_1 = 1$ (guarantees $\theta^{(1)} = x_1$)
- $\gamma_t \in (0, 1), \quad \forall t > 1$ (guarantees $\theta^{(t)}$ stays inside a convex space)

## Weight Types

`OnlineStatsBase.EqualWeight`

— Type.`EqualWeight()`

Equally weighted observations.

$γ(t) = 1 / t$

```
ExponentialWeight(λ::Float64)
ExponentialWeight(lookback::Int)
```

Exponentially weighted observations. The first weight is 1.0 and all else are `λ = 2 / (lookback + 1)`

.

$γ(t) = λ$

`OnlineStatsBase.LearningRate`

— Type.`LearningRate(r = .6)`

Slowly decreasing weight. Satisfies the standard stochastic approximation assumption $∑ γ(t) = ∞, ∑ γ(t)^2 < ∞$ if $r ∈ (.5, 1]$.

$γ(t) = inv(t ^ r)$

`OnlineStatsBase.HarmonicWeight`

— Type.`HarmonicWeight(a = 10.0)`

Weight determined by harmonic series.

$γ(t) = a / (a + t - 1)$

`OnlineStatsBase.McclainWeight`

— Type.`McclainWeight(α = .1)`

Weight which decreases into a constant.

$γ(t) = γ(t-1) / (1 + γ(t-1) - α)$

## Weight wrappers

`OnlineStatsBase.Bounded`

— Type.`Bounded(w::Weight, λ::Float64)`

Bound the weight by a constant.

$γ′(t) = max(γ(t), λ)$

`OnlineStatsBase.Scaled`

— Type.`Scaled(w::Weight, λ::Float64)`

Scale a weight by a constant.

$γ′(t) = λ * γ(t)$

## Custom Weighting

The `Weight`

can be any callable object that receives the number of observations as its argument. For example:

`weight = inv`

will have the same result as`weight = EqualWeight()`

.`weight = x -> x == 1 ? 1.0 : .01`

will have the same result as`weight = ExponentialWeight(.01)`

```
julia> y = randn(100);
julia> fit!(Mean(weight = EqualWeight()), y)
Mean: n=100 | value=-0.0442521
julia> fit!(Mean(weight = inv), y)
Mean: n=100 | value=-0.0442521
julia> fit!(Mean(weight = ExponentialWeight(.01)), y)
Mean: n=100 | value=0.0454786
julia> fit!(Mean(weight = x -> x == 1 ? 1.0 : .01), y)
Mean: n=100 | value=0.0454786
```