Skip to content
NumeriveMATHUniversity mathematics

TOPIC 2 / 3

Hessians & Multivariable Extrema

Useful preparation: Partial Derivatives & Directional Change

For a long equation, swipe sideways inside its formula box to see every term.

Meaning & notation

At a smooth interior unconstrained optimum, every first-order directional change vanishes. The Hessian distinguishes second-order shapes. Constraints permit only feasible directions, so a constrained optimum can have a nonzero ordinary gradient.

Stationary point and saddle
A stationary point has zero gradient. A saddle is a stationary point with arbitrarily nearby larger and smaller function values, preventing either type of local extremum.
Hessian
The matrix of second partial derivatives. For a twice continuously differentiable function it is symmetric and describes the quadratic part of local change.
Lagrange multiplier
A scalar relating objective and constraint gradients at a regular constrained stationary point; the constraint must also hold.

Conditions before calculation

  • The Hessian test requires continuous second partials near an interior stationary point. A zero determinant makes it inconclusive.
  • A continuous function on a nonempty compact feasible set attains its absolute extrema; interior tests alone do not check the boundary.
  • For one smooth equality constraint, the standard Lagrange necessary condition requires a nonzero constraint gradient. Inspect singular constraint points separately.

Why the method works

1. Find first-order candidates

Every directional rate must vanish at an interior differentiable extremum. Solve all first-partial equations simultaneously.

∇f(p)=0\nabla f(\mathbf p)=\mathbf0

2. Classify the quadratic shape

For two variables, at a stationary point, a positive Hessian determinant gives a definite quadratic form, classified by the first diagonal entry. A negative determinant gives opposite directional signs and a saddle.

D=fxxfyy−fxy2;D>0, fxx>0⇒local minimum;D>0, fxx<0⇒local maximum;D<0⇒saddleD=f_{xx}f_{yy}-f_{xy}^2;\qquad D>0,\ f_{xx}>0\Rightarrow\text{local minimum};\quad D>0,\ f_{xx}<0\Rightarrow\text{local maximum};\quad D<0\Rightarrow\text{saddle}

3. Restrict to feasible motion

At a regular constrained extremum, both gradients are normal to the constraint tangent and hence parallel. Solve this condition together with the constraint.

∇f=λ∇g,g(x,y)=c\nabla f=\lambda\nabla g,\qquad g(x,y)=c

Worked examples

EXAMPLE 1

Locate the quadratic's stationary point, classify it, and determine whether the result is global.

  1. Set both coordinate derivatives to zero and solve the linear system.

    f(x,y)=x2+xy+y2−4x−2y;2x+y−4=0,x+2y−2=0 ⟹ (x,y)=(2,0)f(x,y)=x^2+xy+y^2-4x-2y;\qquad 2x+y-4=0,\quad x+2y-2=0\ \Longrightarrow\ (x,y)=(2,0)

  2. The Hessian has positive determinant and positive first diagonal entry, giving a strict local minimum.

    H=(2112),D=3>0,f(2,0)=−4H=\begin{pmatrix}2&1\\1&2\end{pmatrix},\qquad D=3>0,\qquad f(2,0)=-4

  3. Complete squares to compare with every point in the plane. Both squared terms are nonnegative and vanish together only at the candidate.

    f(x,y)=(x−2+y2)2+34y2−4≥−4f(x,y)=\left(x-2+\frac y2\right)^2+\frac34y^2-4\ge-4

Conclusion: The point with coordinates 2 and 0 is the unique global minimum; there is no global maximum.

EXAMPLE 2

Find the largest and smallest coordinate products on the circle.

  1. The circle is compact and its constraint gradient never vanishes on it, so extrema exist and satisfy the regular multiplier equations.

    f(x,y)=xy,g(x,y)=x2+y2=2;⟨y,x⟩=λ⟨2x,2y⟩f(x,y)=xy,\quad g(x,y)=x^2+y^2=2;\qquad \langle y,x\rangle=\lambda\langle2x,2y\rangle

  2. Multiply the first component equation by y and the second by x. This avoids dividing by a possibly zero coordinate and forces equal squares.

    y2=2λxy=x2,x2+y2=2 ⟹ x2=y2=1y^2=2\lambda xy=x^2,\qquad x^2+y^2=2\ \Longrightarrow\ x^2=y^2=1

  3. Compare all four feasible candidates. Equal signs yield the largest product; opposite signs yield the smallest.

    f(1,1)=f(−1,−1)=1,f(1,−1)=f(−1,1)=−1f(1,1)=f(-1,-1)=1,\qquad f(1,-1)=f(-1,1)=-1

Conclusion: The absolute maximum is 1 and the absolute minimum is negative 1, each attained at two points.

Pitfalls & counterexamples

Tempting claim: Zero gradient guarantees a local minimum.

Correction: It eliminates first-order change but leaves higher-order shape unresolved.

Test the claim: The difference of the squared coordinates has zero gradient at the origin, yet increases along the horizontal axis and decreases along the vertical axis.

Check your understanding

If the Hessian determinant is zero at a stationary point, does that prove a saddle?

Write a reason before opening the explanation. This reflection is not graded or saved.

Compare your reasoning

No.

The test is inconclusive: the sum of fourth powers has a strict minimum at the origin, while the difference of fourth powers has a saddle there.

Connect this idea

The Hessian extends the one-variable second-derivative test. Constraint gradients explain why optimization algorithms project proposed changes onto feasible directions.

Practise this topic →Review concept flashcards

Reference reading

Use these texts to study the underlying definitions and theorems in greater depth. The explanations and examples above are written for Numerive.

Numerive