Skip to main content

Section 2.5 Taylor Theorem

Differentiability gives the best linear approximation to a function near a point. If second derivatives are available, then one can do better: the function is approximated by a quadratic polynomial whose coefficients are determined by the Hessian matrix.

Definition 2.5.1. Hessian Matrix.

Let \(f \colon U \to \R\) be defined on an open set \(U \subseteq \R^n\text{,}\) and let \(\va \in U\text{.}\) If the second partial derivatives \(D_iD_jf(\va)\) exist for all \(1 \le i,j \le n\text{,}\) the Hessian matrix of \(f\) at \(\va\) is the \(n \times n\) matrix
\begin{equation*} H_f(\va) := \begin{bmatrix} D_1D_1f(\va) \amp \cdots \amp D_1D_nf(\va) \\ \vdots \amp \ddots \amp \vdots \\ D_nD_1f(\va) \amp \cdots \amp D_nD_nf(\va) \end{bmatrix}. \end{equation*}
If \(f\) is \(C^2\text{,}\) then by Theorem D.0.3 the mixed partials are equal, so the Hessian matrix is symmetric.

Proof.
Since \(f\) is \(C^2\text{,}\) each first partial derivative \(D_if\) has continuous partial derivatives near \(\va\text{.}\) By Theorem 2.4.11, each \(D_if\) is differentiable at \(\va\text{,}\) so by Proposition 2.3.9 the gradient map \(\nabla f \colon U \to \R^n\) is differentiable at \(\va\text{.}\) By the Caratheodory criterion, there is a matrix-valued function \(M(\vx)\text{,}\) continuous at \(\va\text{,}\) such that
\begin{equation*} \nabla f(\vx)-\nabla f(\va)=M(\vx)(\vx-\va) \end{equation*}
for \(\vx\) near \(\va\text{,}\) and \(M(\va)=D(\nabla f)(\va)=H_f(\va)\text{.}\)
Let \(\vh\) be small enough that the segment \(\{\va+t\vh:0\le t\le 1\}\) lies in \(U\text{,}\) and define the one-variable function
\begin{equation*} \phi(t):=f(\va+t\vh). \end{equation*}
By the chain rule,
\begin{equation*} \phi'(t)=\nabla f(\va+t\vh)\cdot \vh. \end{equation*}
Therefore,
\begin{equation*} f(\va+\vh)-f(\va)=\int_0^1 \nabla f(\va+t\vh)\cdot \vh \, dt. \end{equation*}
Using the Caratheodory representation of \(\nabla f\text{,}\)
\begin{equation*} \nabla f(\va+t\vh) = \nabla f(\va)+M(\va+t\vh)(t\vh). \end{equation*}
Substituting into the integral gives
\begin{equation*} f(\va+\vh)-f(\va) = \nabla f(\va)\cdot \vh + \int_0^1 t\,\vh^{\mathsf T}M(\va+t\vh)\vh\,dt. \end{equation*}
Since \(M(\va)=H_f(\va)\text{,}\) we may write
\begin{equation*} \int_0^1 t\,\vh^{\mathsf T}M(\va+t\vh)\vh\,dt = \frac12 \vh^{\mathsf T}H_f(\va)\vh + r(\vh), \end{equation*}
where
\begin{equation*} r(\vh) := \int_0^1 t\,\vh^{\mathsf T}\bigl(M(\va+t\vh)-H_f(\va)\bigr)\vh\,dt. \end{equation*}
Hence
\begin{equation*} |r(\vh)| \le \frac12 \norm{\vh}^2 \sup_{0 \le t \le 1} \norm{M(\va+t\vh)-H_f(\va)}_{\mathrm{op}}. \end{equation*}
Because \(M\) is continuous at \(\va\text{,}\) the supremum tends to \(0\) as \(\vh \to \vz\text{.}\) Therefore \(r(\vh)/\norm{\vh}^2 \to 0\text{.}\)

Example 2.5.3.

Find the second-order Taylor polynomial of \(f(x,y)=e^x\cos y\) at \(\langle 0,0 \rangle\text{.}\)
Solution.
We have
\begin{equation*} f(0,0)=1, \qquad \nabla f(0,0)=\langle 1,0 \rangle. \end{equation*}
Also,
\begin{equation*} f_{xx}(0,0)=1, \qquad f_{xy}(0,0)=0, \qquad f_{yy}(0,0)=-1. \end{equation*}
Thus
\begin{equation*} H_f(0,0) = \begin{bmatrix} 1 \amp 0 \\ 0 \amp -1 \end{bmatrix}. \end{equation*}
Therefore
\begin{equation*} T_{2,\langle 0,0 \rangle}f(x,y) = 1+x+\frac12(x^2-y^2). \end{equation*}

Definition 2.5.4. Stationary Point.

Let \(f \colon U \to \R\text{,}\) where \(U \subseteq \R^n\) is open. A point \(\va \in U\) is a stationary point of \(f\) if
\begin{equation*} \nabla f(\va)=\vz. \end{equation*}
For functions of two variables, the second-order Taylor theorem gives a convenient classification of stationary points.

Proof.
Let \(\vh=\langle h,k \rangle\text{.}\) Since \(\va\) is stationary, the linear term in Taylor’s theorem vanishes, so
\begin{equation*} f(\va+\vh)-f(\va) = \frac12 Q(h,k) + r(\vh), \end{equation*}
where
\begin{equation*} Q(h,k)=Ah^2+2Bhk+Ck^2 \end{equation*}
and \(r(\vh)/\norm{\vh}^2 \to 0\text{.}\)
Suppose first that \(\Delta \gt 0\) and \(A \gt 0\text{.}\) Then
\begin{equation*} Q(h,k) = A\left(h+\frac{B}{A}k\right)^2 + \frac{\Delta}{A}k^2. \end{equation*}
Hence \(Q(h,k) \gt 0\) whenever \(\langle h,k \rangle \ne \vz\text{.}\) The function \(Q(h,k)\) restricted to the unit circle is continuous and positive, so there exists \(m \gt 0\) such that
\begin{equation*} Q(h,k) \ge m(h^2+k^2) \end{equation*}
for all \(h,k\text{.}\) Since \(r(\vh)/\norm{\vh}^2 \to 0\text{,}\) for \(\vh\) sufficiently small we have \(|r(\vh)| \le \frac14 m \norm{\vh}^2\text{.}\) Therefore
\begin{equation*} f(\va+\vh)-f(\va) \ge \frac12 m \norm{\vh}^2 - \frac14 m \norm{\vh}^2 = \frac14 m \norm{\vh}^2 \gt 0 \end{equation*}
for \(\vh \ne \vz\) small. So \(\va\) is a strict local minimum.
If \(\Delta \gt 0\) and \(A \lt 0\text{,}\) then \(-Q\) satisfies the previous case. Hence \(\va\) is a strict local maximum.
Now suppose \(\Delta \lt 0\text{.}\) Consider
\begin{equation*} q(t):=Q(1,t)=A+2Bt+Ct^2. \end{equation*}
Its discriminant is \(4(B^2-AC)=-4\Delta \gt 0\text{,}\) so \(q\) takes both positive and negative values. Choose \(t_1,t_2 \in \R\) such that \(q(t_1) \gt 0\) and \(q(t_2) \lt 0\text{.}\) Along the lines \(k=t_1h\) and \(k=t_2h\text{,}\) Taylor’s formula gives
\begin{equation*} f(\langle a+h,b+t_ih \rangle)-f(\va) = \frac12 q(t_i)h^2 + o(h^2) \qquad (i=1,2). \end{equation*}
For small nonzero \(h\text{,}\) these two quantities have opposite signs. Thus values of \(f\) near \(\va\) occur both above and below \(f(\va)\text{,}\) so \(\va\) is a saddle point.
If \(\Delta=0\text{,}\) the quadratic term may vanish in some directions, and the second-order test alone no longer determines the behavior.

Example 2.5.6.

Classify the stationary point of \(f(x,y)=x^2+xy+2y^2\) at \(\langle 0,0 \rangle\text{.}\)
Solution.
Since \(\nabla f(x,y)=\langle 2x+y, x+4y \rangle\text{,}\) the origin is a stationary point. Also,
\begin{equation*} f_{xx}(0,0)=2, \qquad f_{xy}(0,0)=1, \qquad f_{yy}(0,0)=4. \end{equation*}
Therefore
\begin{equation*} \Delta = (2)(4)-1^2 = 7 \gt 0, \end{equation*}
and \(f_{xx}(0,0)=2 \gt 0\text{.}\) By Theorem 2.5.5, the origin is a strict local minimum.
Upward-opening paraboloid with a lowest point at the origin.
(a) Strict local minimum: \(z=x^2+y^2\text{.}\)
Downward-opening paraboloid with a highest point at the origin.
(b) Strict local maximum: \(z=-(x^2+y^2)\text{.}\)
Saddle surface curving upward in one direction and downward in the perpendicular.
(c) Saddle point: \(z=x^2-y^2\text{.}\)
Very flat bowl-shaped surface with a minimum at the origin, showing a case where the.
(d) Degenerate case: \(z=x^4+y^4\text{.}\)
Figure 2.5.7. Typical stationary points for functions of two variables.
The first three surfaces are exactly the behaviors detected by the second derivative test. The last one shows why the case \(\Delta=0\) must be treated separately: the origin is still a minimum, but the Hessian matrix is zero there.
PrevTopNext