Skip to main content

Section 2.3 Differentiability

Continuity says that \(f(\vx)\) is close to \(f(\va)\) whenever \(\vx\) is close to \(\va\text{.}\) Differentiability is stronger. It says that near \(\va\text{,}\) the change in \(f\) is well approximated by a linear function of the change in the input.

Definition 2.3.1. Differentiability at a Point.

Let \(A \subseteq \R^n\text{,}\) let \(\va \in A\) be an interior point, and let \(f \colon A \to \R^m\text{.}\) We say that \(f\) is differentiable at \(\va\) if there exists a linear map \(L \colon \R^n \to \R^m\) such that
\begin{equation*} \lim_{\vx \to \va} \frac{\norm{f(\vx)-f(\va)-L(\vx-\va)}}{\norm{\vx-\va}} = 0. \end{equation*}
In this case, \(L\) is called the derivative (or differential) of \(f\) at \(\va\text{,}\) and we denote it by \(Df(\va)\text{.}\)
Thus \(f\) is differentiable at \(\va\) exactly when \(f(\vx)-f(\va)\) can be written as a linear term in \(\vx-\va\) plus an error that is small compared to \(\norm{\vx-\va}\text{.}\)
It is often convenient to write \(\vx=\va+\vh\text{.}\) Then the definition above becomes
\begin{equation*} f(\va+\vh)=f(\va)+Df(\va)(\vh)+r(\vh), \end{equation*}
where
\begin{equation*} \lim_{\vh \to 0} \frac{\norm{r(\vh)}}{\norm{\vh}} = 0. \end{equation*}
The remainder term \(r(\vh)\) is therefore of smaller order than \(\norm{\vh}\text{.}\)
The linear map \(Df(\va)\) is the best linear approximation to \(f\) near \(\va\text{.}\) In one variable, this reduces to the familiar formula
\begin{equation*} f(a+h)=f(a)+f'(a)h+r(h), \end{equation*}
with \(r(h)/h \to 0\text{.}\)
Since \(Df(\va)\) is a linear map from \(\R^n\) to \(\R^m\text{,}\) it is represented, with respect to the standard bases of \(\R^n\) and \(\R^m\text{,}\) by an \(m \times n\) matrix. This matrix is called the Jacobian matrix of \(f\) at \(\va\text{.}\)
It is convenient to say that a function \(f \colon \R^n \to \R^m\) is differentiable on \(A\) if \(f\) is differentiable at every point \(\va \in A\text{.}\)
More generally, if \(A \subseteq \R^n\) and \(f \colon A \to \R^m\text{,}\) we say that \(f\) is differentiable on \(A\) if there exists an open set \(U\) containing \(A\) and a differentiable function \(F \colon U \to \R^m\) such that \(F|_A = f\text{.}\)

Definition 2.3.2. Tangent Plane.

Let \(f \colon \R^2 \to \R\) be differentiable at \(\va=\langle a_1,a_2 \rangle\text{.}\) The tangent plane to the graph of \(f\) at the point \(\langle a_1,a_2,f(\va) \rangle\) is the graph of the affine approximation
\begin{equation*} z = f(\va) + Df(\va)\langle x-a_1, y-a_2 \rangle. \end{equation*}
This criterion is often useful because it turns differentiability into a factorization problem. A proof of its equivalence with the usual definition appears in Appendix C.

Proof.
For (1), if \(f\) is constant, then \(f(\vx)-f(\va)=0\) for every \(\vx\text{.}\) Thus
\begin{equation*} f(\vx)-f(\va)=M(\vx)(\vx-\va) \end{equation*}
with \(M(\vx)=0\text{.}\) Since \(M\) is continuous at \(\va\text{,}\) the Caratheodory criterion shows that \(f\) is differentiable at \(\va\text{,}\) and \(Df(\va)=M(\va)=0\text{.}\)
For (2), if \(f\) is linear, then
\begin{equation*} f(\vx)-f(\va)=f(\vx-\va). \end{equation*}
Let \(M(\vx)=f\text{,}\) viewed as a constant matrix-valued function. Then
\begin{equation*} f(\vx)-f(\va)=M(\vx)(\vx-\va), \end{equation*}
and \(M\) is continuous at \(\va\text{.}\) By the Caratheodory criterion, \(f\) is differentiable at \(\va\) and \(Df(\va)=M(\va)=f\text{.}\)

Example 2.3.5.

Compute the derivative of \(p \colon \R^2 \to \R\) defined by
\begin{equation*} p(x,y)=xy. \end{equation*}
Solution.
Let \(\va=\langle a_1,a_2 \rangle\text{.}\) Then
\begin{equation*} p(x,y)-p(\va) = xy-a_1a_2 = y(x-a_1)+a_1(y-a_2). \end{equation*}
This is in Caratheodory form with
\begin{equation*} A_1(x,y)=y, \qquad A_2(x,y)=a_1. \end{equation*}
Both functions are continuous at \(\va\text{.}\) Therefore \(p\) is differentiable at \(\va\text{,}\) and
\begin{equation*} Dp(\va)\langle h,k \rangle = A_1(\va)h + A_2(\va)k = a_2h+a_1k. \end{equation*}
Thus the derivative of \(p\) at \(\va=\langle a_1,a_2 \rangle\) is the linear map \(\langle h,k \rangle \mapsto a_2h+a_1k\text{.}\)

Proof.
For the sum rule, note that the addition map \(\Sigma \colon \R^m \times \R^m \to \R^m\) given by \(\Sigma(\vu,\vv)=\vu+\vv\) is linear. By Proposition 2.3.4, its derivative is \(D\Sigma=\Sigma\text{.}\) This suggests the formula above, and the Caratheodory criterion proves it directly. Write
\begin{equation*} f(\vx)-f(\va)=M_f(\vx)(\vx-\va), \qquad g(\vx)-g(\va)=M_g(\vx)(\vx-\va), \end{equation*}
where \(M_f\) and \(M_g\) are continuous at \(\va\text{.}\) Then
\begin{equation*} (f+g)(\vx)-(f+g)(\va) = (M_f(\vx)+M_g(\vx))(\vx-\va). \end{equation*}
Since \(M_f+M_g\) is continuous at \(\va\text{,}\) the Caratheodory criterion shows that \(f+g\) is differentiable at \(\va\text{,}\) and
\begin{equation*} D(f+g)(\va)=M_f(\va)+M_g(\va)=Df(\va)+Dg(\va). \end{equation*}
For the product rule, the example above computed the derivative of the multiplication map \(p(u,v)=uv\text{.}\) Again we argue directly with the Caratheodory criterion. Write
\begin{equation*} f(\vx)-f(\va)=\sum_{i=1}^n A_i(\vx)(x_i-a_i), \qquad g(\vx)-g(\va)=\sum_{i=1}^n B_i(\vx)(x_i-a_i), \end{equation*}
where each \(A_i\) and \(B_i\) is continuous at \(\va\text{.}\) Then
\begin{align*} (fg)(\vx)-f(\va)g(\va) \amp = f(\vx)g(\vx)-f(\va)g(\va)\\ \amp = f(\vx)\bigl(g(\vx)-g(\va)\bigr)+g(\va)\bigl(f(\vx)-f(\va)\bigr)\\ \amp = \sum_{i=1}^n \bigl(f(\vx)B_i(\vx)+g(\va)A_i(\vx)\bigr)(x_i-a_i). \end{align*}
Because differentiability implies continuity, \(f(\vx)\) is continuous at \(\va\text{.}\) Hence each coefficient \(f(\vx)B_i(\vx)+g(\va)A_i(\vx)\) is continuous at \(\va\text{.}\) By the Caratheodory criterion, \(fg\) is differentiable at \(\va\text{,}\) and
\begin{equation*} D(fg)(\va)(\vh) = \sum_{i=1}^n \bigl(f(\va)B_i(\va)+g(\va)A_i(\va)\bigr)h_i = f(\va)\,Dg(\va)(\vh)+g(\va)\,Df(\va)(\vh). \end{equation*}

Example 2.3.7.

Show that \(f \colon \R^2 \to \R\) defined by
\begin{equation*} f(x,y)=x^2+xy \end{equation*}
is differentiable at \(\langle 1,2 \rangle\text{.}\)
Solution.
Let \(\va=\langle 1,2 \rangle\text{.}\) We use Theorem 2.3.3. Since \(f(\va)=3\text{,}\)
\begin{equation*} f(x,y)-f(\va) = x^2+xy-3 = (x+y+1)(x-1) + (y-2). \end{equation*}
Thus the Caratheodory functions are
\begin{equation*} A_1(x,y)=x+y+1, \qquad A_2(x,y)=1. \end{equation*}
Both are continuous at \(\va\text{.}\) Therefore \(f\) is differentiable at \(\va\text{.}\) Moreover,
\begin{equation*} Df(\va)(\vh) = A_1(\va)h + A_2(\va)k = 4h+k. \end{equation*}
So the derivative at \(\langle 1,2 \rangle\) is the linear map \(\vh=\langle h,k \rangle \mapsto 4h+k\text{.}\)
By Definition 2.3.2, the tangent plane to the graph of \(f\) at \(\langle 1,2,3 \rangle\) is
\begin{align*} z \amp = f(\va) + Df(\va)\langle x-1, y-2 \rangle\\ \amp = 3 + 4(x-1) + (y-2)\\ \amp = 4x+y-3. \end{align*}

Example 2.3.8.

Show that \(F \colon \R^2 \to \R^2\) defined by
\begin{equation*} F(x,y)=\langle x^2-y, xy \rangle \end{equation*}
is differentiable at \(\langle 1,2 \rangle\text{.}\)
Solution.
Again let \(\va=\langle 1,2 \rangle\) and \(\vh=\langle h,k \rangle\text{.}\) Then
\begin{align*} F(\va+\vh) \amp = \left\langle (1+h)^2-(2+k), (1+h)(2+k) \right\rangle\\ \amp = \langle -1+2h-k+h^2, 2+2h+k+hk \rangle. \end{align*}
Since \(F(\va)=\langle -1,2 \rangle\text{,}\)
\begin{align*} F(\va+\vh)-F(\va) \amp = \langle 2h-k, 2h+k \rangle\\ \amp\phantom{{}={}} + \langle h^2, hk \rangle. \end{align*}
Hence the linear part is
\begin{equation*} L(\vh)=\langle 2h-k, 2h+k \rangle, \end{equation*}
and the error term is \(r(\vh)=\langle h^2,hk \rangle\text{.}\) Since
\begin{equation*} \norm{r(\vh)} = \sqrt{h^4+h^2k^2} = |h|\norm{\vh}, \end{equation*}
we get
\begin{equation*} \frac{\norm{r(\vh)}}{\norm{\vh}} = |h| \to 0. \end{equation*}
Therefore \(F\) is differentiable at \(\va\text{,}\) and
\begin{equation*} DF(\va)(\vh) = \langle 2h-k, 2h+k \rangle. \end{equation*}
In particular, a vector-valued function is differentiable exactly when its components are differentiable.

Proof.
Write \(f(\va+\vh)=f(\va)+Df(\va)(\vh)+r(\vh)\text{,}\) where \(\norm{r(\vh)}/\norm{\vh}\to 0\text{.}\) Since \(Df(\va)\) is linear, there is a constant \(C\) such that \(\norm{Df(\va)(\vh)} \le C\norm{\vh}\text{.}\) Therefore
\begin{equation*} \norm{f(\va+\vh)-f(\va)} \le C\norm{\vh} + \norm{r(\vh)} \to 0 \end{equation*}
as \(\vh \to 0\text{.}\) Hence \(f(\va+\vh)\to f(\va)\text{.}\)

Proof.
By the Caratheodory criterion, there is a matrix-valued function \(M_f(\vx)\text{,}\) defined near \(\va\) and continuous at \(\va\text{,}\) such that
\begin{equation*} f(\vx)-f(\va)=M_f(\vx)(\vx-\va) \end{equation*}
for \(\vx\) near \(\va\text{.}\) Likewise, there is a matrix-valued function \(M_g(\vy)\text{,}\) defined near \(\vb=f(\va)\) and continuous at \(\vb\text{,}\) such that
\begin{equation*} g(\vy)-g(\vb)=M_g(\vy)(\vy-\vb) \end{equation*}
for \(\vy\) near \(\vb\text{.}\)
Since differentiability implies continuity, \(f(\vx)\to f(\va)=\vb\) as \(\vx\to\va\text{.}\) Hence, for \(\vx\) sufficiently near \(\va\text{,}\) the point \(f(\vx)\) lies in the neighborhood where the Caratheodory representation for \(g\) is valid. Therefore
\begin{align*} (g\circ f)(\vx)-(g\circ f)(\va) \amp = g(f(\vx))-g(f(\va))\\ \amp = M_g(f(\vx))\bigl(f(\vx)-f(\va)\bigr)\\ \amp = M_g(f(\vx))M_f(\vx)(\vx-\va). \end{align*}
Thus \(g\circ f\) is in Caratheodory form. Its coefficient matrix is \(M(\vx)=M_g(f(\vx))M_f(\vx)\text{.}\) The function \(M\) is continuous at \(\va\) because \(f\) is continuous at \(\va\text{,}\) \(M_g\) is continuous at \(\vb\text{,}\) and matrix multiplication is continuous. Therefore the Caratheodory criterion shows that \(g\circ f\) is differentiable at \(\va\text{,}\) and
\begin{align*} D(g\circ f)(\va) \amp = M(\va)\\ \amp = M_g(f(\va))M_f(\va)\\ \amp = Dg(f(\va))\circ Df(\va). \end{align*}
PrevTopNext