Skip to main content

Section 2.6 Lagrange Multipliers

A local maximum or minimum on an open set must occur at a stationary point. Many optimization problems, however, come with a constraint such as \(g(x,y)=c\text{.}\) Then we are not allowed to move in every direction. We may move only along the constraint set, so we need a test for extrema under this restriction.

Definition 2.6.1. Constrained Local Extremum.

Let \(S \subseteq \R^n\text{,}\) let \(f \colon S \to \R\text{,}\) and let \(\va \in S\text{.}\) We say that \(\va\) is a constrained local maximum of \(f\) on \(S\) if there is \(r \gt 0\) such that
\begin{equation*} f(\vx) \le f(\va) \end{equation*}
for every \(\vx \in S\) with \(\norm{\vx-\va} \lt r\text{.}\) We define a constrained local minimum similarly, with the inequality reversed.
The geometric idea is simple. Suppose the constraint set is the level curve \(g(x,y)=c\text{.}\) At a constrained extremum, the function \(f\) does not change to first order in the tangent direction to the constraint. Thus \(\nabla f\) is perpendicular to that tangent direction. But \(\nabla g\) is also perpendicular to the level curve \(g=c\text{.}\) So at an extremum, the two gradients should point in the same or opposite direction.
The unit circle together with several parallel level lines of x plus y..
Figure 2.6.2. At a constrained extremum on the unit circle, a level curve of \(f\) is tangent to the constraint.

Proof.
Since \(f \circ \gamma\) has a local extremum at \(0\text{,}\) its derivative at \(0\) is zero. By the chain rule and Proposition 2.4.6,
\begin{equation*} 0 = (f \circ \gamma)'(0) = Df(\va)(\gamma'(0)) = \nabla f(\va)\cdot \gamma'(0). \end{equation*}

Proof.
Since \(\va\) is a constrained extremum, the one-variable function \(f \circ \gamma\) has a local extremum at \(0\text{.}\) By Proposition 2.6.3,
\begin{equation*} \nabla f(\va)\cdot \gamma'(0)=0. \end{equation*}
Also, \(g(\gamma(t))=c\) for all \(t\text{,}\) so \(g \circ \gamma\) is constant. Differentiating gives
\begin{equation*} \nabla g(\va)\cdot \gamma'(0)=0. \end{equation*}
Thus both \(\nabla f(\va)\) and \(\nabla g(\va)\) are perpendicular to the nonzero tangent vector \(\gamma'(0)\text{.}\) In \(\R^2\text{,}\) all vectors perpendicular to a fixed nonzero vector lie on the same line, so \(\nabla f(\va)\) and \(\nabla g(\va)\) are parallel. Therefore \(\nabla f(\va)=\lambda \nabla g(\va)\) for some \(\lambda \in \R\text{.}\)

Definition 2.6.5. Lagrange Multiplier.

In the constrained problem of optimizing \(f\) subject to \(g(x,y)=c\text{,}\) a scalar \(\lambda\) satisfying
\begin{equation*} \nabla f(\va)=\lambda \nabla g(\va) \end{equation*}
at a point \(\va\) with \(g(\va)=c\) is called a Lagrange multiplier.
To use the theorem, solve the system
\begin{gather*} \nabla f(x,y)=\lambda \nabla g(x,y),\\ g(x,y)=c. \end{gather*}
The resulting points are candidates for constrained extrema. One must still decide which candidate gives a maximum or minimum. On a closed and bounded constraint, this can be done by comparing the values of \(f\) at the candidates. If the constraint is not closed and bounded, an extremum might not exist at all.

Example 2.6.6.

Find the maximum and minimum values of \(f(x,y)=x+y\) subject to the constraint \(x^2+y^2=1\text{.}\)
Solution.
Let \(g(x,y)=x^2+y^2\text{.}\) Then
\begin{equation*} \nabla f(x,y)=\langle 1,1 \rangle, \qquad \nabla g(x,y)=\langle 2x,2y \rangle. \end{equation*}
The Lagrange multiplier equations are
\begin{equation*} \langle 1,1 \rangle = \lambda \langle 2x,2y \rangle, \qquad x^2+y^2=1. \end{equation*}
From the first equation, \(1=2\lambda x\) and \(1=2\lambda y\text{,}\) so \(x=y\text{.}\) Then
\begin{equation*} 2x^2=1, \qquad x=\pm \frac{1}{\sqrt{2}}, \qquad y=\pm \frac{1}{\sqrt{2}}. \end{equation*}
The candidate points are \(\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)\) and \(\left(-\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}}\right)\text{.}\) Evaluating \(f\text{,}\)
\begin{equation*} f\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)=\sqrt{2}, \qquad f\left(-\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}}\right)=-\sqrt{2}. \end{equation*}
Therefore the maximum value is \(\sqrt{2}\) and the minimum value is \(-\sqrt{2}\text{.}\)

Example 2.6.7.

Find the maximum and minimum values of \(f(x,y)=xy\) on the unit circle \(x^2+y^2=1\text{.}\)
Solution.
Again take \(g(x,y)=x^2+y^2\text{.}\) Then
\begin{equation*} \nabla f(x,y)=\langle y,x \rangle, \qquad \nabla g(x,y)=\langle 2x,2y \rangle. \end{equation*}
So
\begin{equation*} \langle y,x \rangle = \lambda \langle 2x,2y \rangle, \qquad x^2+y^2=1. \end{equation*}
This gives \(y=2\lambda x\) and \(x=2\lambda y\text{.}\) If \(x=0\text{,}\) then the first equation gives \(y=0\text{,}\) which is impossible on the unit circle. Similarly, \(y=0\) is impossible. So \(x\) and \(y\) are both nonzero, and multiplying the equations gives
\begin{equation*} 1 = 4\lambda^2. \end{equation*}
Hence \(\lambda=\pm \frac12\text{,}\) and therefore \(y=\pm x\text{.}\)
If \(y=x\text{,}\) then \(2x^2=1\text{,}\) so \(x=\pm \frac{1}{\sqrt{2}}\) and \(f(x,y)=\frac12\text{.}\) If \(y=-x\text{,}\) then \(f(x,y)=-\frac12\text{.}\) Therefore
\begin{equation*} \max_{x^2+y^2=1} xy = \frac12, \qquad \min_{x^2+y^2=1} xy = -\frac12. \end{equation*}
The coordinate axes, the line x plus 2y equals 6, and circles centered at the origin..
Figure 2.6.8. The nearest point on a line to the origin occurs where the line is tangent to a circle centered at the origin.

Example 2.6.9.

Find the point on the line \(x+2y=6\) closest to the origin.
Solution.
Minimizing the distance to the origin is equivalent to minimizing \(f(x,y)=x^2+y^2\text{.}\) Let \(g(x,y)=x+2y\text{.}\) Then
\begin{equation*} \nabla f(x,y)=\langle 2x,2y \rangle, \qquad \nabla g(x,y)=\langle 1,2 \rangle. \end{equation*}
The Lagrange multiplier equations are
\begin{equation*} \langle 2x,2y \rangle = \lambda \langle 1,2 \rangle, \qquad x+2y=6. \end{equation*}
So \(x=\lambda/2\) and \(y=\lambda\text{.}\) Substituting into the constraint gives
\begin{equation*} \frac{\lambda}{2}+2\lambda=6, \qquad \frac{5\lambda}{2}=6, \qquad \lambda=\frac{12}{5}. \end{equation*}
Therefore the nearest point is
\begin{equation*} \left(\frac65,\frac{12}{5}\right). \end{equation*}
This problem has a minimum but no maximum, because \(x^2+y^2 \to \infty\) as we move farther and farther along the line.
Lagrange multipliers produce candidate points for constrained extrema by matching the normals of the objective and the constraint. The same idea extends to functions of more variables and to several constraints, though the bookkeeping becomes more complicated.
PrevTopNext