GitHub

Diffusion models and calibration

The notion of "diffusion" and "calibration" relates to the prior part of the model.

Background: The prior

We model the ODE solution $y(t)$ with a Gauss–Markov prior. More precisely, let

\[\begin{aligned} Y(t) = \left[ Y^{(0)}(t), Y^{(1)}(t), \dots Y^{(q)}(t) \right], \end{aligned}\]

be the solution to the SDE

\[\begin{aligned} \text{d} Y^{(i)}(t) &= Y^{(i+1)}(t) \ \text{d}t, \qquad i = 0, \dots, q-1, \\ \text{d} Y^{(q)}(t) &= \textcolor{#389826}{A} Y(t) \ \text{d}t + \textcolor{#4063D8}{\Gamma} \ \text{d}W(t), \\ Y(0) &\sim \textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }. \end{aligned}\]

Then $Y^{(i)}(t)$ models the $i$-th derivative of $y(t)$. In this section, we consider choices relating to the "diffusion" $\textcolor{#4063D8}{\Gamma}$. If you're more interested in the drift matrix $\textcolor{#389826}{A}$ check out the Priors section, and for info on the initial distribution $\textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }$ check out the Initialization section.

Diffusion and calibration

We call $\textcolor{#4063D8}{\Gamma}$ the "diffusion" parameter. Since it is typically not known we need to estimate it; this is called "calibration".

ProbNumDiffEq.jl provides a few different choices for how to model and estimate $\textcolor{#4063D8}{\Gamma}$:

  • FixedDiffusion assumes an isotropic, time-fixed $\textcolor{#4063D8}{\Gamma} = \sigma \cdot I_d$,
  • DynamicDiffusion assumes an isotropic, time-varying $\textcolor{#4063D8}{\Gamma}(t) = \sigma(t) \cdot I_d$ (recommended),
  • FixedMVDiffusion assumes a diagonal, time-fixed $\textcolor{#4063D8}{\Gamma} = \operatorname{diag}(\sigma_1, \dots, \sigma_d)$,
  • DynamicMVDiffusion assumes a diagonal, time-varying $\textcolor{#4063D8}{\Gamma}(t) = \operatorname{diag}(\sigma_1(t), \dots, \sigma_d(t))$.

Or more compactly:

Isotropic:Diagonal:
Time-varyingDynamicDiffusionDynamicMVDiffusion
Time-fixedFixedDiffusionFixedMVDiffusion

The isotropic diffusions work with every solver. The diagonal diffusions estimate one diffusion per ODE dimension, which is only supported by the EK0 with the IWP prior (the default) and by the DiagonalEK1. The exception is an uncalibrated FixedMVDiffusion(diffusion, false), which is not estimated and works with every solver.

For more details on diffusions and calibration, check out this paper [9].

API

ProbNumDiffEq.DynamicDiffusion — Type
DynamicDiffusion()

Time-varying, isotropic diffusion, which is quasi-maximum-likelihood-estimated at each step.

This is the recommended diffusion when using adaptive step-size selection, and in particular also when solving stiff systems.

source
ProbNumDiffEq.FixedDiffusion — Type
FixedDiffusion(; initial_diffusion=1.0, calibrate=true)

Time-fixed, isotropic diffusion, which is (optionally) quasi-maximum-likelihood-estimated.

This is the recommended diffusion when using fixed steps.

By default with calibrate=true, all covariances are re-scaled at the end of the solve with the MLE diffusion. Set calibrate=false to skip this step, e.g. when setting the initial_diffusion and then estimating the diffusion outside of the solver (e.g. with Fenrir.jl).

source
ProbNumDiffEq.DynamicMVDiffusion — Type
DynamicMVDiffusion()

Time-varying, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.

Supported by the EK0 with the IWP prior (the default) and by the DiagonalEK1. Other setups throw an ArgumentError.

A multi-variate version of DynamicDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.

References

source
ProbNumDiffEq.FixedMVDiffusion — Type
FixedMVDiffusion(; initial_diffusion=1.0, calibrate=true)

Time-fixed, diagonal diffusion. With calibrate=true it is quasi-maximum-likelihood-estimated from the whole solve.

With calibrate=true, supported by the EK0 with the IWP prior (the default) and by the DiagonalEK1. With calibrate=false, the given initial_diffusion is used as is, and every solver supports it. Only the EK0 with the IWP prior uses a per-dimension diffusion in the local error estimate for adaptive step size selection; all other setups use a scalar estimate there.

A multi-variate version of FixedDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.

References

source

References

[9]
N. Bosch, P. Hennig and F. Tronarp. Calibrated Adaptive Probabilistic ODE Solvers. In: Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, Vol. 130 of Proceedings of Machine Learning Research, edited by A. Banerjee and K. Fukumizu (PMLR, 13–15 Apr 2021); pp. 3466–3474.