A Note on Reproducing Kernel Hilbert Spaces
Vector Spaces
In this article we introduce the reproducing kernel Hilbert space.
Photo by Kaleidico on Unsplash
These notes give an introduction to the theory of reproducing kernel Hilbert spaces. We take a somewhat algebraic view of some of his results. In later chapters we consider generalizations of this theory to the
vector-valued setting. We will use the paper of V. I. Paulson in this short note of reproducing kernel Hilbert spaces. We have tried our best to translate it in simple words so that the beginner understand some of the facts in simple language.
Reproducing kernel Hilbert spaces arise in a number of areas, including
approximation theory, statistics, machine learning theory, group representation theory, operator theory and operator algebra and various areas of complex analysis.
We will consider Hilbert spaces over either the field of real numbers, R,or of complex numbers, C. We will use F to denote either R or C, so thatwhen we wish to state a definition or result that is true for either the realor complex numbers, we will use F.Given a set X, if we equip the set of all functions from X to Fwith the usual operations of addition, (f + g)(x) = f(x) + g(x), and scalar
multiplication, (λ · f)(x) = λ · (f(x)), then it is a vector space over F.
Definition
Definition 2.1. Given a set X, we will say that H is a reproducingkernel Hilbert space(RKHS) on X over F, provided that:(i) H is a vector vector space,(ii) H is endowed with an inner product, h, i, making it into a Hilbertspace,(ii) for every y ∈ X, the linear evaluation functional, E : H → F,
defined by E(f) = f(y), is bounded.
In sequel, we will use k for K(x, y) where x and y are in X.
Definition
Let H be a reproducing kernel Hilbert space on X. We say
that H separates points provided that for x is not equal to y there exists f ∈ H such that f(x) is not equal to f(y).
We have the following very important problem which can be solved by using above definition.
The following problem shows that if H is a reproducing kernel Hilbert space(RKHS) and if there is a closed subspace of H then the closed subspace of H is also a reproducing kernel Hilbert space by using the orthogonal projection from H onto that closed subspace.
An Example of Reproducing Kernel Hilbert Space
Hardy-Hilbert space of the unit disc H²(D) is an example of reproducing kernel Hilbert space. The norm on H²(D) is given by
where a_n are the Taylor coefficients of f.
The map L : H2(D) →l² (N) defined by L(f) = (a_0, a_1,a_2 . . .) is a linear inner product preservingisomorphism and hence we see that H² (D) can be identified with the Hilbertspace, l² (N), where N= {0, 1, 2, . . .} the natural numbers with 0, and hence is itself a Hilbert space. Thus, we see that (ii) of the
above definition is met.
Next we show that every power series in H2(D), converges to define a
function on the disk. To see this note that if z ∈ D, then
Thus the evaluation functional E_z is bounded.
Thus, each power series defines a function on D and the vector space operations on formal power series, clearly agrees with their vector space operations as functions on D, and so (i) is met. Hence all the three condition of the above definition satisfies and hence H²(D) is reproducing Kernel Hilbert space.
Problem:
Sobolev space on [0,1] is reproducing kernel Hilbert space.
Complexification of Reproducing Kernel Hilbert spaces
Let H be a RKHS of real-valued functions on the set X with reproducingkernel, k. Let W = {f_1 + i f_2 : f_1, f_2 ∈ H}, which is a vector space of
complex-valued functions on X. If we set,
then it is easily checked that this defines an inner product on W, with
corresponding norm,
Hence, W is a Hilbert space and since
we have that W equipped with this inner product is a RKHS of complex valued functions on X with reproducing kernel, k.
We call W the complexification of H. Since every real-valued RKHScan be complexified in a way that still preserves the reproducing kernel, we
shall from this point on, only consider the case of complex-valued reproducing kernel Hilbert spaces.
Let X be any set and let H be a RKHS on X with kernel K. In this sectionwe will show that “k” completely determines the space H and characterize
the functions that are the kernel functions of some RKHS.
The following Proposition shows that the set of reproducing kernels is dense in H.
Problem
Generalize the above scalar version to the vector valued version.