Diffusion models and calibration

The notion of "diffusion" and "calibration" relates to the prior part of the model.

Background: The prior

We model the ODE solution $y(t)$ with a Gauss–Markov prior. More precisely, let

\[\begin{aligned} Y(t) = \left[ Y^{(0)}(t), Y^{(1)}(t), \dots Y^{(q)}(t) \right], \end{aligned}\]

be the solution to the SDE

\[\begin{aligned} \text{d} Y^{(i)}(t) &= Y^{(i+1)}(t) \ \text{d}t, \qquad i = 0, \dots, q-1, \\ \text{d} Y^{(q)}(t) &= \textcolor{#389826}{A} Y(t) \ \text{d}t + \textcolor{#4063D8}{\Gamma} \ \text{d}W(t), \\ Y(0) &\sim \textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }. \end{aligned}\]

Then $Y^{(i)}(t)$ models the $i$-th derivative of $y(t)$. In this section, we consider choices relating to the "diffusion" $\textcolor{#4063D8}{\Gamma}$. If you're more interested in the drift matrix $\textcolor{#389826}{A}$ check out the Priors section, and for info on the initial distribution $\textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }$ check out the Initialization section.

Diffusion and calibration

We call $\textcolor{#4063D8}{\Gamma}$ the "diffusion" parameter. Since it is typically not known we need to estimate it; this is called "calibration".

ProbNumDiffEq.jl provides a few different choices for how to model and estimate $\textcolor{#4063D8}{\Gamma}$:

FixedDiffusion assumes an isotropic, time-fixed $\textcolor{#4063D8}{\Gamma} = \sigma \cdot I_d$,
DynamicDiffusion assumes an isotropic, time-varying $\textcolor{#4063D8}{\Gamma}(t) = \sigma(t) \cdot I_d$ (recommended),
FixedMVDiffusion assumes a diagonal, time-fixed $\textcolor{#4063D8}{\Gamma} = \operatorname{diag}(\sigma_1, \dots, \sigma_d)$,
DynamicMVDiffusion assumes a diagonal, time-varying $\textcolor{#4063D8}{\Gamma}(t) = \operatorname{diag}(\sigma_1(t), \dots, \sigma_d(t))$.

Or more compactly:

	Isotropic:	Diagonal (only for the `EK0`)
Time-varying	`DynamicDiffusion`	`DynamicMVDiffusion`
Time-fixed	`FixedDiffusion`	`FixedMVDiffusion`

For more details on diffusions and calibration, check out this paper [9].

API

ProbNumDiffEq.DynamicDiffusion — Type

DynamicDiffusion()

Time-varying, isotropic diffusion, which is quasi-maximum-likelihood-estimated at each step.

This is the recommended diffusion when using adaptive step-size selection, and in particular also when solving stiff systems.

source

ProbNumDiffEq.FixedDiffusion — Type

FixedDiffusion(; initial_diffusion=1.0, calibrate=true)

Time-fixed, isotropic diffusion, which is (optionally) quasi-maximum-likelihood-estimated.

This is the recommended diffusion when using fixed steps.

By default with calibrate=true, all covariances are re-scaled at the end of the solve with the MLE diffusion. Set calibrate=false to skip this step, e.g. when setting the initial_diffusion and then estimating the diffusion outside of the solver (e.g. with Fenrir.jl).

source

ProbNumDiffEq.DynamicMVDiffusion — Type

DynamicMVDiffusion()

Time-varying, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.

Only works with the EK0!

A multi-variate version of DynamicDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.

References

[9] Bosch et al, "Calibrated Adaptive Probabilistic ODE Solvers", AISTATS (2021)

source

ProbNumDiffEq.FixedMVDiffusion — Type

FixedMVDiffusion(; initial_diffusion=1.0, calibrate=true)

Time-fixed, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.

Only works with the EK0!

A multi-variate version of FixedDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.

References

[9] Bosch et al, "Calibrated Adaptive Probabilistic ODE Solvers", AISTATS (2021)

source

References

[9]: N. Bosch, P. Hennig and F. Tronarp. Calibrated Adaptive Probabilistic ODE Solvers. In: Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, Vol. 130 of Proceedings of Machine Learning Research, edited by A. Banerjee and K. Fukumizu (PMLR, 13–15 Apr 2021); pp. 3466–3474.