Module 9 · Lesson 9.2

Constraints — Optimizing on a Boundary

Learn what changes when you're not free to pick any point in the bowl — only points along a fence line running through it.

Intuition

Lesson 9.1 was about finding the single lowest point of a convex bowl with no restrictions on where you could stand. Real problems are rarely that generous. You have a fixed budget to spend, a fixed amount of material to use, a fixed deadline — some rule that draws a fence through the middle of the bowl and says "you may only stand along this fence."

Core concept

A constrained optimization problem asks for the best point of a function while restricted to a boundary — a line, curve, or surface defined by some other equation. The unconstrained minimum (the true bottom of the bowl) usually isn't on the fence at all, so the best you can do is find the lowest point along the fence, which is typically some point on the slope of the bowl rather than its bottom. Budget-constrained resource allocation is exactly this: you want to minimize cost or maximize value, but you must stay on the line "total spend equals the budget."

(0,0) (2,2)
Minimizef(x, y) = x² + y²
Constraint (the fence)x + y = 4
Unconstrained minimum(0, 0), but off the fence
Best point on the fence(2, 2)
Answerf(2, 2) = 8

Every other point on the line x + y = 4, like (0, 4) or (4, 0), gives a larger value of f (16 in both cases). The constrained minimum sits where the fence is tangent to one of f's circular contour rings — that's the closest the fence ever gets to the unconstrained center.

Interactive Lab

Set a budget line x + y = k, then slide a point along that line. Watch the cost value and its contour ring shrink as you approach the point where the fence just grazes the smallest possible circle.

f(x, y) = 18.00 at (3.00, 3.00)

Key Idea

$$ \min_{x} f(x) \quad \text{subject to} \quad g(x) = c $$

This is the standard shape of a constrained optimization problem. You still want to make f as small as possible, but only among the inputs that satisfy the constraint equation exactly. Everything off the boundary defined by g is off limits, no matter how good f would be there.

$f$The quantity you're trying to minimize (or maximize) — cost, loss, distance
$x$The input you're choosing, possibly with several components like (x, y)
$g$The rule that defines the fence, e.g. total spend or total resource used
$c$The fixed value the constraint must equal, such as a total budget

Think Further

  1. Why is the constrained minimum always at least as large as the unconstrained minimum, never smaller?
  2. In the worked example, the best point (2, 2) was where the fence line just touches one contour circle without crossing it. What do you think is special about that touching point compared to a point where the fence crosses through a contour circle?
  3. If you doubled the budget in the resource-allocation analogy (moved the fence further from the origin), would you expect the minimum achievable cost to go up or down? Why?
Show suggested answers
  1. Because the constrained search only looks at a subset of all possible points — the fence — so it can never do better than searching everywhere, only as well or worse.
  2. At a touching (tangent) point, the fence and the contour ring share the same direction momentarily, so nudging along the fence in either direction moves you onto a larger contour ring. At a crossing point, you could still slide along the fence toward a smaller ring, so it can't be the minimum yet.
  3. Moving the fence away from the origin (a larger budget k) makes the closest point on the fence farther from the unconstrained minimum at the origin, so the minimum achievable cost goes up, not down — this is the general idea that tighter constraints can only hurt, never help, an unconstrained-style objective.