Hilbert Spaces and Riesz Representation Theorem

I though about doing this as a regular feature precisely long enough to produce this image.

The Riesz Representation Theorem: There are a lot of theorems which go by this name, but I want to deal today with one particular version.  First, I’ll (briefly) define a Hilbert space, which has been defined here before.  It is a (1) vector space equipped with an (2) inner product which is (3) complete with respect to that inner product.

I’m going to assume some familiarity with item (2)- a vector space being a collection of vectors and scalars, so that you can add vectors and multiply by scalars according to some rules.

Also, those who have seen calculus are likely familiar with (3), an inner product, though often the words “dot product” are used.  Abstractly, an inner product is a function that acts on two vectors and returns a real (or complex, though I’ll ignore that here) number.  We often denote the inner product with brackets, so $\langle (1,2),(4,5) \rangle = 1\cdot 4 + 2 \cdot 5 = 14$ would be the familiar calculus inner product on $\mathbb{R}^2$.

An abstract inner product must satisfy symmetry, $\langle v,w \rangle = \langle w,v \rangle$ for vectors v,w; linearity, $\langle av+bw,z \rangle = a \langle v,z \rangle + b \langle w,z \rangle$ for scalars a,b and vectors v,w,z (note that by symmetry, I only need to require linearity for the first coordinate, and I automatically get it in the second); and positive definiteness, that is $\langle v,v \rangle > 0$ (or 0 if is the zero vector).

Part of an orthogonal basis for (square integrable) functions. See Leonid Kovalev's post for a discussion of a different one.

An inner product allows us to talk about the geometry of a space, in particular angles between vectors, following the famous test for orthogonality (right-angledness) in Euclidean space: and are orthogonal iff $v \cdot w = 0$.  An important inner product is on the space of square integrable (real valued) functions, denoted $L^2$.  Then we can define an inner product between two functions (the vectors in this vector space) by $\langle f,v \rangle = \int f(x)g(x)~dx.$

A Cauchy sequence

Finally, (3), being complete means that any Cauchy sequence converges, and a Cauchy sequence is one so that, roughly, eventually all the terms are close together.   The real numbers are complete, but perhaps more helpfully, the rational numbers are not.  Specifically, the sequence, 3, 3.1, 3.14, 3.141, 3.1415,… will get closer and closer to $\pi$, but since pi is not a rational number, the sequence will not converge in the rationals.

The Theorem!

Given a Hilbert space H, certainly if we fix a vector v, then the inner product against v defines a (continuous) linear function on H, $\phi_v(w) = \langle v, w \rangle$.  The surprising fact (theorem!) is that given a continuous linear function f on H, there is a unique vector v so that $f(w) = \langle v, w \rangle$ for all w in H.  Put another way: not only does the inner product on H provide a supply of examples of continuous linear functionals, it provides all of them.

A not Cauchy sequence

One great application of this is that every continuous linear functional defined on square integrable functions is an integral!  $F[u] = \int f(x)u(x)~dx$ for some function f.  It becomes a little less surprising when you realize that part of the trick in the theorem is requiring continuity, which is defined in terms of the inner product.  But hey!  Still pretty cool!