Module 4 · Lesson 4.3

The Chain Rule — Composite Functions as Nested Gears

See the chain rule as nested gears that multiply rates of change, letting a derivative travel through every layer of a composition.

Intuition

When one function feeds into another, a small change at the input is scaled once by the inner function and again by the outer function. The overall rate is the product of the two individual rates—exactly the way nested gears multiply their angular speeds.

The same multiplication appears in backpropagation: each layer of a neural network is a function of the previous layer. The chain rule threads the gradient backward through every gear, producing the update signal for every weight.

Core concept

If $y=f(g(x))$, then a tiny change $dx$ first becomes $g'(x)\,dx$ and then is multiplied by $f'(g(x))$. The composite derivative is therefore the product $f'(g(x))\cdot g'(x)$.

Inner$g(x)=x^{2}$, $g'=2x$

Outer$f(u)=u^{3}$, $f'=3u^{2}$

Composite$y=(x^{2})^{3}=x^{6}$

At $x=1$$f'(g)\cdot g'=3\cdot 2=6$

Check$(x^{6})'=6x^{5}=6$

Interactive Lab

Adjust the input $x$ and the two exponents. Watch each “gear” report its local rate; their product is the composite derivative.

Input $x$ 1.20 Inner power $m$ 2 Outer power $n$ 3

Inner g

$g=x^{m}$

—

$g'$ = —

→

Outer f

$f=u^{n}$

—

$f'$ = —

→

Product

$f'\cdot g'$

—

Composite $y=(x^{m})^{n}=x^{mn}$
Chain-rule derivative at current $x$: —

Each card is one gear. The final product is exactly what backpropagation multiplies when a gradient travels through two successive layers.

Key Idea

$$ \frac{dy}{dx} = \frac{dy}{du}\cdot\frac{du}{dx} $$

Let $u=g(x)$ and $y=f(u)$. The first factor $\frac{dy}{du}$ is the outer derivative evaluated at the current inner value; the second factor $\frac{du}{dx}$ is the inner derivative. Their product is the rate at which $y$ changes with respect to $x$. The same pattern extends to any number of nested functions, which is why the chain rule is the engine of backpropagation.

$u=g(x)$Inner function (first gear)

$y=f(u)$Outer function (second gear)

$\frac{dy}{du}$Outer rate evaluated at the current $u$

$\frac{du}{dx}$Inner rate at the current $x$

Think Further

Why is the outer derivative evaluated at $g(x)$ rather than at $x$?
If a network has three layers, how many multiplications does the chain rule perform to obtain the gradient of the loss with respect to the first layer’s weights?
What happens to the composite derivative when one of the intermediate gears has derivative zero at the current point?

Show suggested answers

The outer function receives $g(x)$ as its actual input, so its local rate of change must be measured at that value.
It multiplies the local derivative contributions along the full path from the first layer through the remaining layers to the loss—one factor for each link.
The product becomes zero along that path, so the gradient cannot flow through that intermediate step at the current point.