I am taking a course about Hilbert Spaces this semester. A very basic notion in a Hilbert Space is that of the norm. For a while now, I kept several questions regarding norms in the back of my mind, and as I finally got to think about them, I wanted to share my conclusions with you.

Introductory Concepts 

Lets start from the beginning (those of you familiar with Normed Spaces can skip this section). A linear space is a set of objects with two operations:

1. Addition (marked by the symbol +) which acts on two elements of the space and returns an element of the space.
2. Multiplication (marked by the symbol *) which acts on an element of the space and a real number and returns an element of the space.

The operations must follow these rules (x and y are elements of a linear space, a and b are real numbers):

1. x + y = y + x                              (+ commutativity)
2. x + (y + z) = (x + y) + z           (+ associativity)
3. Existence of a zero element (marked by the symbol 0) such that for every element of the space: 0+x = x+0 = x
4. For every element x of the space, existence of a negative element (marked by -x) such that: -x + x = 0
5. (a*b)*x = a*(b*x)                    (* associativity)
6. For every element x of the space, 1*x = x
7. a*(x+y) = a*x + a*y                (distributive law 1)
8. (a+b)*x = a*x + b*x                (distributive law 2)

A simple example of a linear space is the Euclidean space of n dimensions (for any n >= 1).

Note that we define the minus symbol, -, as follows:

x – y = x + (-1 * y)

Now, a norm is essentially a function that somehow tells us how far away an element of our space is from the neutral element (0).

We write ||x|| for the norm of x.

The norm can also tell us how far apart two elements of the space are. We say that the norm generates a distance. This is done as follows:

dist(x, y) = ||x – y||

Formally, a norm is a function from the linear space to R+ (the set of non-negative real numbers), such that (x and y are elements of the linear space, a is a real number):

1. ||a*x|| = |a| * ||x||                             (homogeneity)
2. ||x+y|| <= ||x|| + ||y||                      (triangle inequality)
3. ||x|| >= 0 and ||x|| = 0 i.f.f. x = 0

Examples 

Now lets consider the 2-dimensional euclidean plane as our linear space (call it R2). An example of a norm in R2 is the euclidean norm (where u=(x,y) is a member of R2):

||u|| = sqrt(x^2 + y^2)

We shall call this norm, the l2 norm.

For those of you not familiar with the concept of the norm, verifying that this is indeed a norm may prove insightful.

The l2 norm is very useful as it measures actual distances in a plane.

Now lets consider another norm, called the l1 norm:

||u|| = |x| + |y|

Again, it can be easily verified that this is indeed a norm. This norm obviously generates a different distance than that of the plane. Does it have a “real-world” use? You are encouraged to take a minute of reflection upon this before reading on…

 Well, yes, it does have a use. Lets say that we are standing in Midtown-Manhattan.

What is the distance between us? The aerial distance is the regular euclidean distance (neglecting the roundness of the Earth) but the walking distance is the distance generated by the l1 norm! This is caused by the fact that we can only walk along the streets (i.e. parallel to the axes!). For this reason the distance generated by the l1 norm is sometimes referred to as the Manhattan-Distance.

Now, lets consider a third norm called l∞, defined as:

||u|| = max(|x|,|y|)

Again, this is obviously a norm (check it!).

Now lets start with the fun stuff…

First, can you figure out the meaning of the names of the norms (l1, l2, l∞)?

For all p >= 1, lets define the function fp:R2->R+ as:

fp(u) = (x^p + y^p)^(1/p)

Then fp is a norm on R2 and we call it lp (I feel like I am repeating myself, but again, this can be easily checked).

The names of l1 and l2 become obvious right away, and so does the name of l∞ (after a little thought ).

If you still can’t figure it out - l∞(x) is the point-wise limit of lp(x) as p->∞ for all the elements x of R2. Actually, the functions lp converge to l∞ locally-uniformly (can you prove this?).

This brings up an interesting question: Do norms other than l1 and l2 have any “real-world” meaning? If you have any insights on this, please comment!

Unit Circles

The unit circle of a normed space is the subset of elements of the space such that their norm is equal to 1.

Can you figure out how l1, l2 and l∞ unit circles look like in R2?

As I found it difficult to visualize the unit circles for norms lp, p>2, I jotted down a few lines of C in order to create the following image:

The red, green, light-blue and blue shapes denote the unit circles for the l1, l2, l3, and l∞ norms respectively. The other colors denote the unit circles for lp norms for various values of p.

In the picture I included values of p that are not whole numbers. This should not pose any problems.

I also included values of p that are less than 1 (the shapes inside the l1 circle). These are there for illustration purposes only, as they do not constitute norms!

Note: the glowing effect was generated by my program to overcome jagged edges caused by precision issues.

Something else to think about is the volume of the unit circles. The volume of the unit circle under the l1 norm is 2, under the l2 norm Π and under the l∞ norm, 4.

Can anyone supply some more values?

That’s it for now!

I might update this article, so check the updates section!

This entry was posted on Tuesday, June 19th, 2007 at 1:57 am and is filed under Math. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.