Fréchet Derivative

If F\colon X\to Y is a function between normed linear spaces the Fréchet derivative {DF\colon X\to\mathcal{B}(X,Y)} is defined by F(x + h) = F(x) + DF(x)h + o(\|h\|) for x,h\in X where \mathcal{B}(X,Y) is the set of bounded linear operators from X to Y. The “little o” notation means \|F(x + h) - F(x) - DF(x)h\|/\|h\| \to 0 as \|h\| \to 0. The function F can be approximated near x by the linear operator DF(x).

A norm is a function \|\cdot\|\colon X\to\boldsymbol{R} satisfying \|ax\| = |a|\|x\|, \|x + y\| \le \|x\| + \|y\|, \|x\|\ge0 and \|x\| = 0 implies x = 0 for a\in\boldsymbol{R}, x,y\in X.

If X = Y = \boldsymbol{R} then DF\colon\boldsymbol{R}\to\mathcal{B}(\boldsymbol{R},\boldsymbol{R}) where DF(x)h = F'(x)h.

Define the dual of a normed linear space X by X^* = \mathcal{B}(X,\boldsymbol{R}). If X^I = \{x\colon I\to\boldsymbol{R}\} is the set of functions from the set I to \boldsymbol{R} we can define an inner product (\cdot,\cdot)\colon X\times X\to\boldsymbol{R} by (x, y) = \sum_{i\in I}x(i) y(i) if I is finite. We also write x_i for x(i).

Exercise. Show the inner product is bilinear.

Hint: Show (ax + y, z) = a(x,y) + (y,z) and (x, ay + z) = a(x,y) + (x,z) for a\in\boldsymbol{R}, x,y,z\in X using (x,y) = \sum_{i\in I}x_i y_i.

The inner product defines a norm by \|x\| = \sqrt{(x, x)}.

Exercise. Show |(x,y)| \le \|x\| \|y\| for x,y\in X.

Hint: Use 0\le\|ax + y\| for a\in\boldsymbol{R}, x,y\in X and minimize over a.

Exercise. Show if |(x,y)| = \|x\| \|y\| and x\not=0 then y = ax for some a\in\boldsymbol{R}.

Exercise. Show \|\cdot\| is a norm.

…(R^I)^*\cong\boldsymbol{R}^I…

D\|x\|^p = p\|x\|^{p-1}x^*.

Let F(x) = x^2. For T\in\mathcal{L}_n