Gaussian Process Classification

Article MQL5 articles

Summary

This article introduces Gaussian process classification for binary labels and explains why it requires approximate inference. A Gaussian process places a prior over latent functions, which are mapped through a sigmoid to class probabilities. Unlike Gaussian-likelihood regression, the classification posterior under a logit likelihood is non-Gaussian and does not have a closed-form solution.

The article outlines the two prediction steps: infer a distribution for the latent function at a new input, then average sigmoid probabilities over that distribution. It develops the Laplace approximation, which replaces the posterior near its mode with a Gaussian so that predictive mean and variance calculations become tractable. The approximation trades fidelity to the true posterior for computational convenience; MCMC is described as a more computationally demanding comparison. The installment also introduces MQL5 library components for the Gaussian process and hyperparameter optimization, while detailed interfaces, demonstrations, and trading applications are deferred to the next part.

Key ideas

A sigmoid maps Gaussian process latent function values to probabilities for binary classes.
The logit likelihood makes the classification posterior non-Gaussian and prevents exact closed-form inference.
Prediction involves estimating the latent function distribution and integrating sigmoid probabilities over it.
Laplace approximation uses a Gaussian centered at the posterior mode to make inference tractable.
The approximation may be inaccurate when the true posterior differs substantially from a Gaussian, while MCMC is more computationally expensive.

Gaussian Process Classification with Laplace Approximation

Summary

Key ideas

Tags