Gaussian Processes
In finite dimensions, the Gaussian distribution is defined by the density function
where is the covariance matrix.
Definition. Let be a set, is the mean function, is the covariance function.
A random function (can write ) is called a Gaussian Process with mean and covariance if for any choice of points , are jointly Gaussian distributed with mean and covariance matrix .
The covariance matrix has to be symmetric positive semi-definite for any .
Convercely, for any and positive definite kernel, there is a Gaussian process with the characteristics, which we denote by .
Covariance function kernel
Take . .
Set for simplicity. If we look at Gaussian, with mean zero and covariance matrix
GP Regression
are given data, the goal is to find a .
In Bayesian,