The Derivative: From Difference Quotients to the Chain Rule

0. Key points

• The derivative $f'(a)$ is the limit of the difference quotient $\dfrac{f(a+h)-f(a)}{h}$ as $h \to 0$. The order of operations — divide first, then pass to the limit — is what matters; it is precisely how the difficulty of the indeterminate form $0/0$ is avoided.
• Differentiability is equivalent to the statement that the error committed in approximating $f(a+h)$ by the linear expression $f(a)+Ah$ tends to $0$ faster than $h$ does. This reformulation is the gateway to the several-variable theory.
• Differentiability implies continuity. The converse fails, and $|x|$ is a counterexample.
• We derive the derivatives of $x^n$, $\sin x$, $\cos x$ and $e^x$ from the definition. The only keys are two limits: $\lim_{x\to0}\frac{\sin x}{x}=1$ and $\lim_{h\to0}\frac{e^h-1}{h}=1$.
• Once linearity, the product rule, the quotient rule and the chain rule have been proved, differentiating elementary functions reduces to mechanical computation. The proof of the chain rule requires a device for avoiding division by $0$.
• The rule for inverse functions, $(f^{-1})'(b)=1/f'(a)$, puts $\log x$, $\sqrt{x}$, $\arctan x$ and even the real powers $x^\alpha$ within reach at a single stroke.

1. Motivation: the tangent problem and instantaneous velocity

The mathematicians of the seventeenth century carried two problems that appear at first sight unrelated: drawing a line touching a curve, the tangent problem (Fermat, Descartes), and assigning an instantaneous velocity to a motion that changes from moment to moment (Galileo, Newton). Differential calculus was born from the recognition that these two are one and the same operation.

Consider the tangent problem first. For a circle we can define a tangent as a line meeting the circle in exactly one point. That definition, however, fails completely for general curves. The parabola $y=x^2$ and the $y$-axis share only the origin, yet we do not want to call the $y$-axis a tangent. Conversely, the line $y=1$, which ought to be the tangent to $y=\sin x$ at $x=\pi/2$, touches the curve at the infinitely many points $x=\pi/2+2n\pi$ ($n$ an integer). And the tangent $y=0$ to $y=x^3$ at the origin passes straight through the curve to the other side. A global condition on the number of shared points simply cannot capture tangency.

So we change the point of view. The line through the two points $(a, f(a))$ and $(a+h, f(a+h))$ on the curve — the secant — has slope

$$\frac{f(a+h)-f(a)}{(a+h)-a} = \frac{f(a+h)-f(a)}{h}$$

which is determined by division alone. If, as we bring the second point toward the first, the slope of the secant settles down to a single value, let us call that value the slope of the tangent.

Now for instantaneous velocity. If the position at time $t$ is $x(t)$, then the average velocity from time $a$ to time $a+h$ is $\frac{x(a+h)-x(a)}{h}$. But what is “the velocity at the instant $a$”? Substituting $h=0$ gives $\frac{0}{0}$, which means nothing. Newton’s “fluxions” and Leibniz’s “infinitesimal $dx$” involved a logically inconsistent procedure: carry out the division treating $dx \ne 0$, and afterwards regard $dx$ as $0$. Berkeley attacked this fiercely in his 1734 work The Analyst, calling infinitesimals the ghosts of departed quantities.

The notion of a limit put an end to the confusion. Rather than setting $h$ equal to $0$, carry out the division with $h \ne 0$ and then take the limit of the result as $h \to 0$. Merely interchanging the order makes the difficulty of $0/0$ disappear. Since the slope of a secant and an average velocity are expressions of the same shape, both problems are settled by a single definition.

In what follows we take the rigorous definition of a limit (the $\varepsilon$-$\delta$ formulation) as known. Consult Limits and continuity ($\varepsilon$-$\delta$ arguments) as needed.

2. Preliminaries: notation and the Landau symbol

Throughout this article $I \subset \mathbb{R}$ denotes an open interval, and all functions are real-valued. Since $a \in I$ is a point of an open interval, $a+h \in I$ holds for all sufficiently small $|h|$. Hence the difference quotient $\frac{f(a+h)-f(a)}{h}$ is defined on some neighborhood of $0$ with $h = 0$ removed, and it makes sense to consider its limit as $h\to0$.

In the proofs we use the following properties of limits as known (all of them results from Limits and continuity).

Property used	Content
algebra of limits（Theorem 4.2）［Limits and Continuity］	If the limits exist, the limit of a sum, difference or product is the sum, difference or product of the limits. For a quotient the same holds when the limit of the denominator is not $0$
squeeze theorem（Theorem 4.3）［Limits and Continuity］	If $g \le f \le h$ and the limits of $g$ and $h$ agree, then $f$ has the same limit
composition with a continuous function（Theorem 5.3）［Limits and Continuity］	If $u(x) \to b$ as $x \to a$ and $\varphi$ is continuous at $b$, then $\varphi(u(x)) \to \varphi(b)$
continuity of polynomials and rational functions（Corollary 5.2）［Limits and Continuity］	Continuous at every point where the denominator is not $0$

To speak concisely about the size of an error, we introduce the following notation.

Definition 2.1（Landau's little-o）

Let $r$ be a function defined on some punctured neighborhood of $0$. If

$$\lim_{h \to 0} \frac{r(h)}{h} = 0$$

holds, we write $r(h) = o(h) \ (h \to 0)$ and say that $r$ is an infinitesimal of higher order than $h$. More generally, when $\lim_{h\to0} r(h)/h^{n} = 0$ we write $r(h)=o(h^n)$.

This “equation” does not assert that the left-hand side equals the right-hand side; it is an abuse of notation meaning “the function on the left is one of the functions with this property”. Accordingly, a formula such as $o(h)+o(h)=o(h)$ is to be read from left to right. Indeed, if $r_1(h)/h \to 0$ and $r_2(h)/h \to 0$, then $(r_1(h)+r_2(h))/h \to 0+0=0$, so the assertion follows from the limit law for sums alone. Likewise $c \cdot o(h) = o(h)$ for any constant $c$.

3. The derivative at a point and the derivative function

Definition 3.1（The derivative at a point）

Let $I$ be an open interval, $f \colon I \to \mathbb{R}$ and $a \in I$. If the limit

$$\lim_{h \to 0} \frac{f(a+h)-f(a)}{h}$$

exists as a finite real number, we say that $f$ is differentiable at the point $a$, and we call this limit the derivative of $f$ at $a$, written $f'(a)$.

Putting $x = a+h$, the conditions $h \to 0$ and $x \to a$ are the same, so we may also write

$$f'(a) = \lim_{x \to a} \frac{f(x)-f(a)}{x-a} .$$

We choose whichever form is more convenient for the proof at hand.

When $f$ is differentiable at $a$, we define the line

$$y = f(a) + f'(a)(x-a)$$

to be the tangent line to the graph of $y=f(x)$ at the point $(a,f(a))$. Note the order of ideas: we do not define the word “touch” first and then compute a slope. Rather, the existence of the limit of secant slopes is what we call differentiability, and the line with that limit as its slope is what we call the tangent. As we saw in §1, a definition by counting shared points is useless.

The slope of the tangent is defined as the limit of the slopes of secants

Example 3.2（The tangent to a parabola, from the definition）

Let $f(x)=x^2$ and compute the derivative at an arbitrary $a \in \mathbb{R}$ straight from the definition. For $h \ne 0$,

$$\frac{f(a+h)-f(a)}{h} = \frac{(a+h)^2 - a^2}{h} = \frac{2ah + h^2}{h} = 2a + h .$$

Along the way we cancelled a factor $h$; this is legitimate precisely because the difference quotient is only ever considered for $h \ne 0$. The last expression is a linear function of $h$, so it converges to $2a$ as $h \to 0$. Hence $f$ is differentiable at every point and $f'(a)=2a$.

The tangent at $a=1$ is $y = 1 + 2(x-1) = 2x-1$. Checking the shared points for good measure, $x^2 - (2x-1) = (x-1)^2 = 0$ gives only $x=1$; but as explained in §1, this property of meeting the curve at a single point is not the reason the line is a tangent — it merely happens to hold.

Definition 3.3（The derivative function and higher derivatives）

If $f \colon I \to \mathbb{R}$ is differentiable at every point of $I$, we say that $f$ is differentiable on $I$, and we call the function $f' \colon I \to \mathbb{R}$ assigning $f'(x)$ to each $x \in I$ the derivative of $f$. It is also written $\dfrac{df}{dx}$, $\dfrac{d}{dx}f(x)$ or $Df$.

If moreover $f'$ is differentiable on $I$, its derivative is written $f'' = (f')'$ and called the second derivative. In general we define inductively $f^{(0)}=f$ and $f^{(n)} = (f^{(n-1)})'$, and call $f^{(n)}$ the $n$-th derivative. When $f^{(n)}$ exists and is in addition continuous, $f$ is said to be of class $C^n$.

Differentiability imposes a fairly strong constraint on a function. The first thing we can see is the following.

Theorem 3.4（Differentiability implies continuity）

Let $I$ be an open interval, $f \colon I \to \mathbb{R}$ and $a \in I$. If $f$ is differentiable at $a$, then $f$ is continuous at $a$.

Proof（Theorem 3.4）

For $h \ne 0$ we have the identity obtained by multiplying the difference quotient back by $h$:

$$f(a+h) - f(a) = \frac{f(a+h)-f(a)}{h} \cdot h .$$

By hypothesis the first factor on the right converges to $f'(a)$ as $h \to 0$, and the second converges to $0$. By the limit law for products (§2), the right-hand side converges to $f'(a) \cdot 0 = 0$. Hence $\lim_{h\to0} f(a+h) = f(a)$, which is exactly the statement that $f$ is continuous at $a$.

∎

We shall use this theorem repeatedly in what follows. In the proof of the product rule, for example, we need the step ”$g(a+h) \to g(a)$ because $g$ is differentiable”, and there we invoke Theorem 3.4.

The converse fails.

Example 3.5（The absolute value is not differentiable at the origin）

Let $f(x)=|x|$. This $f$ is continuous on $\mathbb{R}$. But the difference quotient at $a=0$ is

$$\frac{f(0+h)-f(0)}{h} = \frac{|h|}{h} = \begin{cases} 1 & (h > 0) \\ -1 & (h < 0) \end{cases}$$

so the right-hand limit is $1$ and the left-hand limit is $-1$. Since the two disagree, the limit as $h \to 0$ does not exist and $f$ is not differentiable at $0$. Where the graph has a corner, the slopes of the secants settle on different values according as we approach from the right or from the left.

Remark 3.6

The gap between continuity and differentiability is in fact not merely a matter of “finitely many corners”. In 1872 Weierstrass constructed a function on $\mathbb{R}$ that is continuous everywhere and differentiable nowhere (of the form $\sum_{n=0}^{\infty} a^n \cos(b^n \pi x)$). Differentiability is a far stronger condition than continuity.

4. Differentiability as linear approximation

The definition by a limit of difference quotients is convenient for computation, but because it involves dividing by $h$ it cannot be transferred as it stands to several variables, where $h$ becomes a vector. So let us prepare an equivalent reformulation in which no division appears. It is at the same time an answer to the question of what operation differentiation really performs.

Theorem 4.1（Differentiability and linear approximation）

Let $I$ be an open interval, $f \colon I \to \mathbb{R}$, $a \in I$ and $A \in \mathbb{R}$. The following two conditions are equivalent.

(i) $f$ is differentiable at $a$ and $f'(a) = A$.

(ii) Setting $r(h) := f(a+h) - f(a) - Ah$, we have $r(h) = o(h) \ (h \to 0)$; that is, $f(a+h) = f(a) + Ah + o(h)$.

Moreover, a real number $A$ satisfying condition (ii) is unique if it exists.

Proof（Theorem 4.1）

Since $a$ is a point of the open interval $I$, there is a $\rho > 0$ such that $0 < |h| < \rho$ implies $a + h \in I$. For $h$ in this range, dividing both sides of the definition of $r(h)$ by $h$ ($\ne 0$) gives the identity

$$\frac{r(h)}{h} = \frac{f(a+h)-f(a)}{h} - A .$$

By the limit law for sums, the statement that the left-hand side converges to $0$ as $h \to 0$ and the statement that the first term on the right converges to $A$ are equivalent (the constant $A$ passes through the limit). This is the equivalence of (i) and (ii).

Now uniqueness. Suppose $A$ and $B$ both satisfy (ii), with corresponding errors $r_A, r_B$. Taking the difference, for $0 < |h| < \rho$,

$$(B - A) h = r_A(h) - r_B(h) .$$

Dividing both sides by $h$ gives $B - A = \dfrac{r_A(h)}{h} - \dfrac{r_B(h)}{h}$, and the right-hand side converges to $0 - 0 = 0$ as $h \to 0$. The left-hand side is a constant independent of $h$, and the limit of a constant function is that constant itself; by uniqueness of limits, $B - A = 0$, that is, $A = B$.

∎

Theorem 4.1 says that differentiability means: the error made in replacing $f$ locally by a linear function is an infinitesimal of higher order than $h$. And $f'(a)$ is the slope of that best linear function. Since no division by $h$ appears in this form, replacing $h$ by a vector and $A$ by a linear map turns it directly into the definition of the derivative in several variables. For details see the definition of total differentiability（Definition 4.1）［多変数関数の微分と偏微分］ in Differentiation in several variables and partial derivatives.

Tip

Linear approximation is useful in numerical work too. With $f(x)=\sqrt{x}$ and $a=1$ (the value $f'(1)=1/2$ is verified in Exercise 9.1) we get $\sqrt{1+h} \approx 1 + h/2$. Taking $h=0.02$, the approximation is $1.01$ while the true value is $\sqrt{1.02} = 1.009950\ldots$, an error of about $5.0 \times 10^{-5}$. Shrinking $h$ by a factor of $10$ to $0.002$ makes the error about $5.0 \times 10^{-7}$, smaller by a factor of $100$. That the error decreases like $h^2$ rather than like $h$ is the concrete face of $o(h)$.

5. Derivatives of the basic functions

5.1. Two key limits

Differentiating the trigonometric functions and the exponential function each reduces to a single limit. We begin with the trigonometric case. From here on, angles are always measured in radians.

Lemma 5.1（The basic trigonometric limits）

$$\lim_{x \to 0} \frac{\sin x}{x} = 1, \qquad \lim_{x \to 0} \frac{1-\cos x}{x} = 0 .$$

Proof（Lemma 5.1）

Suppose first that $0 < x < \pi/2$. Take the center $O$ of the unit circle, the point $A=(1,0)$ on the $x$-axis and the point $P=(\cos x, \sin x)$ on the circle, and let $T=(1,\tan x)$ be the intersection of the tangent at $A$ with the ray $OP$. The triangle $OAP$, the circular sector $OAP$ and the triangle $OAT$ are nested in this order, so their areas satisfy

$$\frac{1}{2}\sin x \;\le\; \frac{x}{2} \;\le\; \frac{1}{2}\tan x$$

(the area of the sector is $x/2$, since the radius is $1$ and the central angle is $x$ radians). Multiplying each side by $2$ and dividing by $\sin x > 0$ gives

$$1 \le \frac{x}{\sin x} \le \frac{1}{\cos x} .$$

For $0 < x < \pi/2$ every side is positive, so taking reciprocals and reversing the inequalities yields

$$\cos x \le \frac{\sin x}{x} \le 1 .$$

Since $\cos$ is continuous with $\cos 0 = 1$, the squeeze theorem (§2) gives the right-hand limit $\lim_{x \to +0} \frac{\sin x}{x} = 1$. Moreover $\frac{\sin(-x)}{-x} = \frac{-\sin x}{-x} = \frac{\sin x}{x}$, so the quotient is an even function and the left-hand limit is $1$ as well. The two one-sided limits agree, and the first assertion follows.

The second assertion reduces to the first. Using $1 - \cos^2 x = \sin^2 x$ and $1 + \cos x \ne 0$ (for $|x| < \pi/2$ we have $\cos x > 0$), we rewrite

$$\frac{1-\cos x}{x} = \frac{(1-\cos x)(1+\cos x)}{x(1+\cos x)} = \frac{\sin^2 x}{x(1+\cos x)} = \frac{\sin x}{x} \cdot \frac{\sin x}{1+\cos x} .$$

As $x \to 0$ the first factor tends to $1$, and the second tends to $\frac{0}{1+1} = 0$ by the continuity of $\sin$ and $\cos$ together with the limit law for quotients. By the limit law for products the whole expression tends to $1 \cdot 0 = 0$.

∎

Remark 5.2

This proof grants the geometric fact that the area of the sector is $x/2$. But defining arc length and area rigorously requires integration, and the computation of those integrals often uses the derivatives of the trigonometric functions. Depending on how a textbook is organized, the argument can therefore become circular. The standard way to avoid this is to define $\sin$ and $\cos$, not geometrically, but by the power series

$$\sin x = \sum_{k=0}^{\infty} \frac{(-1)^k x^{2k+1}}{(2k+1)!}, \qquad \cos x = \sum_{k=0}^{\infty} \frac{(-1)^k x^{2k}}{(2k)!}$$

and to rebuild radians and $\pi$ from there. Along that road, Lemma 5.1 follows directly from estimates on the series (exactly the same argument as the treatment of the exponential function below). For the definition by series see Series and convergence tests, and for the relation between area and integration see The fundamental theorem of calculus and the definite integral.

Next, the exponential function. Here we take $\exp$ to be defined by the power series

$$\exp(x) := \sum_{n=0}^{\infty} \frac{x^n}{n!} \qquad (x \in \mathbb{R})$$

and we use the following two facts, established in Series and convergence tests. First, this series converges absolutely for every $x$ (d'Alembert's ratio test（Theorem 6.2）［級数と収束判定］). Second, the addition formula $\exp(x+y) = \exp(x)\exp(y)$ holds (the Cauchy product of absolutely convergent（Definition 7.1）［級数と収束判定］ series). We set $e := \exp(1) = 2.71828\ldots$ and from now on also write $\exp(x)$ as $e^x$.

Lemma 5.3（The basic limit for the exponential）

$$\lim_{h \to 0} \frac{e^h - 1}{h} = 1 .$$

Proof（Lemma 5.3）

First we check that $e < 3$. For $n \ge 1$ we have $n! = 1\cdot 2 \cdot 3 \cdots n \ge 2^{n-1}$, so

$$e = \sum_{n=0}^{\infty} \frac{1}{n!} \le 1 + \sum_{n=1}^{\infty} \frac{1}{2^{n-1}} = 1 + 2 = 3 ,$$

and since the inequality is strict from $n=3$ on, $e < 3$.

Next let $0 < |h| \le 1$. Separating the first two terms of the series gives $e^h - 1 - h = \sum_{n=2}^{\infty} \frac{h^n}{n!}$, so by the triangle inequality (term-by-term estimation is permitted because the series converges absolutely),

$$|e^h - 1 - h| \le \sum_{n=2}^{\infty} \frac{|h|^n}{n!} = |h|^2 \sum_{n=2}^{\infty} \frac{|h|^{n-2}}{n!} .$$

Here, for $n \ge 2$ we have $n! = n(n-1)\cdot(n-2)! \ge 2 \cdot (n-2)!$, hence $\frac{1}{n!} \le \frac{1}{2 \cdot (n-2)!}$. Substituting $m = n-2$,

$$\sum_{n=2}^{\infty} \frac{|h|^{n-2}}{n!} \le \frac{1}{2}\sum_{m=0}^{\infty} \frac{|h|^{m}}{m!} = \frac{1}{2}\exp(|h|) .$$

For $0 \le |h| \le 1$ each term satisfies $\frac{|h|^m}{m!} \le \frac{1}{m!}$, so $\exp(|h|) \le e < 3$. Putting all of this together,

$$|e^h - 1 - h| \le \frac{3}{2}h^2 \qquad (0 < |h| \le 1) ,$$

and dividing both sides by $|h|$,

$$\left| \frac{e^h-1}{h} - 1 \right| \le \frac{3}{2}|h| .$$

Hence, given $\varepsilon > 0$, taking $\delta := \min\{1,\ 2\varepsilon/3\}$ makes the left-hand side at most $\frac{3}{2}|h| < \frac{3}{2}\delta \le \varepsilon$ for every $h$ with $0 < |h| < \delta$. This is precisely the definition of the required limit.

∎

5.2. The basic formulas

Theorem 5.4（Derivatives of the basic functions）

The following hold.

(1) Let $n$ be a positive integer and $f(x)=x^n$ ($x \in \mathbb{R}$). Then $f$ is differentiable at every point of $\mathbb{R}$ with $f'(x) = n x^{n-1}$. Here, for $n=1$ we adopt the convention $x^0 = 1$ (including $1$ at $x=0$).

(2) $\sin$ is differentiable at every point of $\mathbb{R}$ with $(\sin)'(x) = \cos x$.

(3) $\cos$ is differentiable at every point of $\mathbb{R}$ with $(\cos)'(x) = -\sin x$.

(4) $\exp$ is differentiable at every point of $\mathbb{R}$ with $(\exp)'(x) = \exp(x) = e^x$.

Proof（Theorem 5.4）

(1) Fix $a \in \mathbb{R}$. Let us verify the factorization, valid for $x \ne a$,

$$x^n - a^n = (x-a)\sum_{k=0}^{n-1} x^k a^{n-1-k} .$$

Distributing $(x-a)$ over the sum on the right gives

$$\sum_{k=0}^{n-1} x^{k+1} a^{n-1-k} - \sum_{k=0}^{n-1} x^{k} a^{n-k} ,$$

and substituting $j = k+1$ in the first sum turns it into $\sum_{j=1}^{n} x^{j} a^{n-j}$. The second sum is $\sum_{j=0}^{n-1} x^{j}a^{n-j}$, so the overlapping terms $j=1,\ldots,n-1$ cancel, leaving only the term $x^n$ with $j=n$ and the term $-a^n$ with $j=0$. This proves the factorization.

Consequently, for $x \ne a$,

$$\frac{f(x)-f(a)}{x-a} = \frac{x^n - a^n}{x-a} = \sum_{k=0}^{n-1} x^k a^{n-1-k} .$$

The right-hand side is a polynomial in $x$, hence continuous (§2), and as $x \to a$ it converges to

$$\sum_{k=0}^{n-1} a^k a^{n-1-k} = \sum_{k=0}^{n-1} a^{n-1} = n a^{n-1} .$$

Since $a$ was arbitrary, (1) is proved.

(2) For $a \in \mathbb{R}$ and $h \ne 0$, the addition formula $\sin(a+h) = \sin a \cos h + \cos a \sin h$ gives

$$\frac{\sin(a+h)-\sin a}{h} = \sin a \cdot \frac{\cos h - 1}{h} + \cos a \cdot \frac{\sin h}{h} .$$

By Lemma 5.1, the fraction in the first term is $-\frac{1-\cos h}{h} \to 0$ and the fraction in the second term tends to $1$; by the limit laws for sums and products the whole expression converges to $\sin a \cdot 0 + \cos a \cdot 1 = \cos a$.

(3) Similarly, from $\cos(a+h) = \cos a \cos h - \sin a \sin h$,

$$\frac{\cos(a+h)-\cos a}{h} = \cos a \cdot \frac{\cos h - 1}{h} - \sin a \cdot \frac{\sin h}{h} \longrightarrow \cos a \cdot 0 - \sin a \cdot 1 = -\sin a .$$

(4) From the addition formula $e^{a+h} = e^a e^h$, for $h \ne 0$,

$$\frac{e^{a+h}-e^{a}}{h} = e^{a}\cdot\frac{e^{h}-1}{h} .$$

Here $e^a$ is a constant independent of $h$, and by Lemma 5.3 the fraction on the right converges to $1$; by the limit law for products the whole expression converges to $e^a$.

∎

Part (4) of Theorem 5.4 shows that $\exp$ has the remarkable property of being unchanged by differentiation. This is why the exponential function turns up everywhere in the theory of differential equations.

6. Rules of differentiation

Returning to the definition every time we wish to differentiate a particular function is not practical. The following theorem decomposes differentiation into “derivatives of the parts” and “rules for assembling them”.

Theorem 6.1（Linearity, the product rule and the quotient rule）

Let $I$ be an open interval, $f, g \colon I \to \mathbb{R}$ and $a \in I$, and suppose both $f$ and $g$ are differentiable at $a$. Then the following hold.

(1) (Linearity) For all real numbers $\alpha, \beta$, the function $\alpha f + \beta g$ is differentiable at $a$ and

$$(\alpha f + \beta g)'(a) = \alpha f'(a) + \beta g'(a) .$$

(2) (Product rule) The product $fg$ is differentiable at $a$ and

$$(fg)'(a) = f'(a)g(a) + f(a)g'(a) .$$

(3) (Quotient rule) If moreover $g(a) \ne 0$, then $g$ has no zero on some open interval containing $a$; the function $f/g$ defined there is differentiable at $a$ and

$$\left(\frac{f}{g}\right)'(a) = \frac{f'(a)g(a) - f(a)g'(a)}{g(a)^2} .$$

Proof（Theorem 6.1）

(1) For $h \ne 0$, the definition of the difference quotient gives

$$\frac{(\alpha f + \beta g)(a+h) - (\alpha f + \beta g)(a)}{h} = \alpha\cdot\frac{f(a+h)-f(a)}{h} + \beta\cdot\frac{g(a+h)-g(a)}{h} .$$

By hypothesis the two difference quotients on the right converge to $f'(a)$ and $g'(a)$ respectively, so by the limit laws for sums and scalar multiples the left-hand side converges to $\alpha f'(a)+\beta g'(a)$.

(2) We add and subtract $f(a)g(a+h)$ in the numerator of the difference quotient. For $h \ne 0$,

$$\begin{aligned} \frac{f(a+h)g(a+h) - f(a)g(a)}{h} &= \frac{f(a+h)g(a+h) - f(a)g(a+h) + f(a)g(a+h) - f(a)g(a)}{h} \\ &= \frac{f(a+h)-f(a)}{h}\, g(a+h) \;+\; f(a)\,\frac{g(a+h)-g(a)}{h} . \end{aligned}$$

In the first term the difference quotient converges to $f'(a)$ and $g(a+h)$ converges to $g(a)$; the latter holds because $g$, being differentiable at $a$, is continuous at $a$ by Theorem 3.4. This is the place where “differentiability implies continuity” is used. The second term is the product of the constant $f(a)$ with a difference quotient converging to $g'(a)$. By the limit laws for products and sums, the whole expression converges to $f'(a)g(a) + f(a)g'(a)$.

(3) First we confirm that $f/g$ is defined near $a$. By Theorem 3.4, $g$ is continuous at $a$, so taking $\varepsilon = |g(a)|/2 > 0$ in the definition of continuity, there is a $\delta > 0$ such that $|x-a| < \delta$ (with $x \in I$) implies $|g(x)-g(a)| < |g(a)|/2$. Then the triangle inequality gives

$$|g(x)| \ge |g(a)| - |g(x)-g(a)| > |g(a)| - \frac{|g(a)|}{2} = \frac{|g(a)|}{2} > 0 ,$$

so $g$ has no zero on this neighborhood, and $f/g$ is defined there.

Next we differentiate $1/g$. For $h \ne 0$ with $|h| < \delta$, placing the terms over a common denominator gives

$$\frac{1}{h}\left(\frac{1}{g(a+h)} - \frac{1}{g(a)}\right) = \frac{1}{h}\cdot\frac{g(a)-g(a+h)}{g(a+h)g(a)} = -\frac{1}{g(a+h)g(a)}\cdot\frac{g(a+h)-g(a)}{h} .$$

As $h \to 0$ we have $g(a+h) \to g(a) \ne 0$ (again by Theorem 3.4), so by the limit law for quotients the first factor converges to $-\frac{1}{g(a)^2}$ and the second to $g'(a)$. Hence $1/g$ is differentiable at $a$ with $\left(\frac{1}{g}\right)'(a) = -\frac{g'(a)}{g(a)^2}$.

Finally, regarding $\frac{f}{g} = f \cdot \frac{1}{g}$ and applying (2),

$$\left(\frac{f}{g}\right)'(a) = f'(a)\cdot\frac{1}{g(a)} + f(a)\cdot\left(-\frac{g'(a)}{g(a)^2}\right) = \frac{f'(a)g(a) - f(a)g'(a)}{g(a)^2} .$$∎

That the product rule is not $f'(a)g'(a)$ can be made plausible by a picture of areas. A rectangle with side lengths $f$ and $g$ has area $fg$. Stretching the two sides by $\Delta f$ and $\Delta g$ respectively increases the area by “the strip added horizontally, $\Delta f \cdot g$”, plus “the strip added vertically, $f \cdot \Delta g$”, plus “the small rectangle in the corner, $\Delta f \cdot \Delta g$”. The last term is of second order in $h$, so it is $o(h)$ and vanishes; the two remaining strips correspond to the two terms of the product rule.

Example 6.2（Negative powers, the tangent, rational functions）

(a) Negative integer powers. Let $n$ be a positive integer and $x \ne 0$. Taking $f \equiv 1$ (a constant function; $f' = 0$ follows at once from the definition) and $g(x)=x^n$ in Theorem 6.1 (3),

$$\left(\frac{1}{x^n}\right)' = \frac{0 \cdot x^n - 1 \cdot n x^{n-1}}{(x^n)^2} = -\frac{n x^{n-1}}{x^{2n}} = -n x^{-n-1} .$$

Writing $m=-n$, this reads $(x^m)' = m x^{m-1}$. Including the case $m=0$ of a constant function, we have now established the formula $(x^m)'=mx^{m-1}$ for every integer $m$, at every $x \ne 0$.

(b) The tangent function. For $x$ with $\cos x \ne 0$ we have $\tan x = \frac{\sin x}{\cos x}$. By Theorem 6.1 (3) and Theorem 5.4 (2)(3),

$$(\tan)'(x) = \frac{\cos x \cdot \cos x - \sin x \cdot(-\sin x)}{\cos^2 x} = \frac{\cos^2 x + \sin^2 x}{\cos^2 x} = \frac{1}{\cos^2 x} = 1 + \tan^2 x .$$

The last equality comes from reading $\frac{\cos^2 x + \sin^2 x}{\cos^2 x}$ as a split into $1 + \tan^2 x$.

(c) A rational function. For $x \ne 1$ let $F(x) = \dfrac{x^2+1}{x-1}$. The numerator has derivative $2x$ and the denominator has derivative $1$, so

$$F'(x) = \frac{2x(x-1) - (x^2+1)\cdot 1}{(x-1)^2} = \frac{2x^2 - 2x - x^2 - 1}{(x-1)^2} = \frac{x^2-2x-1}{(x-1)^2} .$$

7. Differentiating composites (the chain rule)

The remaining assembly rule concerns the composite $x \mapsto f(g(x))$. Intuitively, $g$ magnifies a change in the input by the factor $g'(a)$ and $f$ magnifies the result by a further factor $f'(g(a))$, so the overall magnification should be the product of the two.

flowchart LR
A["change in x: h"] -->|"magnification by g: g'(a)"| B["change in u: about g'(a)h"] -->|"magnification by f: f'(g(a))"| C["change in y: about f'(g(a))g'(a)h"]

The chain rule: rates of change compose as a product of magnification factors

Remark 7.1

Naively it looks as though it would suffice to split the difference quotient as

$$\frac{f(g(a+h))-f(g(a))}{h} = \frac{f(g(a+h))-f(g(a))}{g(a+h)-g(a)} \cdot \frac{g(a+h)-g(a)}{h}$$

and pass to the limit. But this expression loses its meaning for those $h$ with $g(a+h) = g(a)$, where the denominator becomes $0$. Nor can we claim that this never happens for $h$ sufficiently close to $0$. The function $g(x) = x^2 \sin(1/x)$ (with $g(0)=0$) treated in Example 7.5 is differentiable at $a=0$, yet $g(1/(n\pi)) = 0 = g(0)$ for every $n = 1, 2, \ldots$, so the denominator vanishes arbitrarily close to $0$. The proof therefore needs a different tool.

Theorem 7.2（The chain rule）

Let $I, J$ be open intervals, $g \colon I \to \mathbb{R}$ and $f \colon J \to \mathbb{R}$ with $g(I) \subset J$. Suppose $g$ is differentiable at $a \in I$ and $f$ is differentiable at $b := g(a)$. Then the composite $f \circ g \colon I \to \mathbb{R}$ is differentiable at $a$ and

$$(f \circ g)'(a) = f'(g(a))\, g'(a) .$$

Proof（Theorem 7.2）

Define an auxiliary function $\varphi \colon J \to \mathbb{R}$ by

$$\varphi(y) := \begin{cases} \dfrac{f(y)-f(b)}{y-b} & (y \ne b) \\ f'(b) & (y = b) \end{cases}$$

This $\varphi$ has the following two properties.

First, $\varphi$ is continuous at $b$. Indeed, $\lim_{y \to b}\varphi(y)$ is exactly the limit of the difference quotient of $f$ at $b$, which by the hypothesis that $f$ is differentiable at $b$ equals $f'(b) = \varphi(b)$. The value assigned to $\varphi$ at $y=b$ was chosen precisely so as to make it continuous.

Second, for every $y \in J$,

$$f(y) - f(b) = \varphi(y)\,(y-b) .$$

For $y \ne b$ this is just the defining formula for $\varphi$ with the denominator cleared, and for $y = b$ both sides are $0$. Rewriting things in this “multiplicative” form is what removes the danger of dividing by $0$.

Now take $h \ne 0$ with $a + h \in I$ and substitute $y = g(a+h) \in J$ into the identity above:

$$f(g(a+h)) - f(g(a)) = \varphi(g(a+h))\,\bigl(g(a+h)-g(a)\bigr) .$$

Dividing both sides by $h$ ($\ne 0$),

$$\frac{f(g(a+h)) - f(g(a))}{h} = \varphi(g(a+h)) \cdot \frac{g(a+h)-g(a)}{h} .$$

The essential point is that this identity remains correct even for those $h$ with $g(a+h) = g(a)$, both sides then being $0$.

It remains to let $h \to 0$. Since $g$ is differentiable at $a$, it is continuous at $a$ by Theorem 3.4, so $g(a+h) \to g(a) = b$. As $\varphi$ is continuous at $b$, the limit property for composition with a continuous function (§2) gives $\varphi(g(a+h)) \to \varphi(b) = f'(b)$. The second factor converges to $g'(a)$ by definition. By the limit law for products,

$$\lim_{h\to0}\frac{f(g(a+h)) - f(g(a))}{h} = f'(b)\,g'(a) = f'(g(a))\,g'(a) ,$$

which says that $f\circ g$ is differentiable at $a$ with derivative the right-hand side.

∎

Remark 7.3

In Leibniz notation, setting $y = f(u)$ and $u = g(x)$, the chain rule reads

$$\frac{dy}{dx} = \frac{dy}{du}\cdot\frac{du}{dx} .$$

Its power lies in looking like the cancellation of fractions, which makes it hard to go wrong when changing variables. Bear in mind, however, that at this stage $dy$ and $du$ are not quantities with a meaning of their own: $\frac{dy}{du}$ is a single symbol meaning “differentiate with respect to $u$”. The appearance of cancellation is a happy encoding of the conclusion of Theorem 7.2, not a proof of it. Do not forget either the convention that $\frac{dy}{du}$ denotes the value at $u = g(x)$, that is, $f'(g(x))$.

Example 7.4（Computations with the chain rule）

(a) $\frac{d}{dx}\sin(x^2)$. With outer function $f(u)=\sin u$ and inner function $g(x)=x^2$ we have $f'(u) = \cos u$ and $g'(x)=2x$, so

$$\frac{d}{dx}\sin(x^2) = \cos(x^2)\cdot 2x = 2x\cos(x^2).$$

(b) $\frac{d}{dx}e^{-x^2/2}$ (the shape occurring in the density of the normal distribution). With outer function $f(u)=e^u$ and inner function $g(x) = -x^2/2$, where $g'(x) = -x$,

$$\frac{d}{dx}e^{-x^2/2} = e^{-x^2/2}\cdot(-x) = -x\,e^{-x^2/2}.$$

(c) $\frac{d}{dx}(1+x^2)^{10}$. Reading the outer function as $u^{10}$ and the inner as $1+x^2$,

$$10(1+x^2)^{9}\cdot 2x = 20x(1+x^2)^{9}.$$

Expanding first and then differentiating would mean handling a polynomial with $11$ terms; with the chain rule it is one line.

(d) A triple composite. $\frac{d}{dx}\cos^3(2x+1)$. Decomposing as $u = 2x+1$, $v = \cos u$, $y = v^3$ and using the chain rule twice,

$$\frac{dy}{dx} = 3v^2 \cdot (-\sin u)\cdot 2 = -6\sin(2x+1)\cos^2(2x+1).$$

Remembering it as “multiply the slopes from the inside outwards” makes mistakes less likely, I think.

Example 7.5（A differentiable function whose derivative is not continuous）

$$f(x) = \begin{cases} x^2 \sin\dfrac{1}{x} & (x \ne 0) \\ 0 & (x = 0)\end{cases}$$

Let us examine this function.

First, differentiability at $x=0$, straight from the definition. For $h \ne 0$,

$$\left|\frac{f(h)-f(0)}{h}\right| = \left|h \sin\frac{1}{h}\right| \le |h|$$

(we used $|\sin t| \le 1$), so by the squeeze theorem the difference quotient converges to $0$. Hence $f$ is differentiable at $0$ with $f'(0)=0$.

Next, for $x \ne 0$, the product rule Theorem 6.1 (2) together with the chain rule Theorem 7.2 (applied to the composite of $\sin$ and $1/x$, with $\left(\frac1x\right)' = -\frac{1}{x^2}$ from Example 6.2 (a)) gives

$$f'(x) = 2x\sin\frac{1}{x} + x^2 \cdot \cos\frac1x \cdot\left(-\frac{1}{x^2}\right) = 2x \sin\frac1x - \cos\frac1x .$$

Now take $x_n = \frac{1}{2n\pi}$ ($n=1,2,\ldots$), so that $x_n \to 0$. But

$$f'(x_n) = 2x_n\sin(2n\pi) - \cos(2n\pi) = 0 - 1 = -1 ,$$

so $f'(x_n) \to -1 \ne 0 = f'(0)$. That is, $f$ is differentiable on all of $\mathbb{R}$ and yet $f'$ is not continuous at $0$. Being differentiable and being of class $C^1$ are different things.

Derivatives can be discontinuous in this way, but they are not entirely unconstrained either: a derivative always has the intermediate value property (Darboux’s theorem). We take up this topic in The mean value theorem and Taylor’s theorem.

8. Differentiating inverse functions

How are we to find the derivative of a function defined as an inverse, such as $\log$ or $\arctan$? Since the graph of $f^{-1}$ is the reflection of the graph of $f$ in the line $y=x$, the slope of the tangent ought to be the reciprocal. The following proposition makes this precise.

Proposition 8.1（Differentiating an inverse function）

Let $I$ be an open interval and $f \colon I \to \mathbb{R}$ a continuous, strictly monotone (increasing or decreasing) function. Then $J := f(I)$ is an open interval, and there is an inverse function $f^{-1}\colon J \to I$, which is continuous (this follows from the intermediate value theorem).

If moreover $f$ is differentiable at $a \in I$ with $f'(a) \ne 0$, then $f^{-1}$ is differentiable at $b := f(a)$ and

$$(f^{-1})'(b) = \frac{1}{f'(a)} = \frac{1}{f'\bigl(f^{-1}(b)\bigr)} .$$

Proof（Proposition 8.1）

The first part (that $J$ is an open interval and that $f^{-1}$ exists and is continuous) belongs to the theory of continuous functions, so we leave it to Limits and continuity and prove the second part here.

Let $y \in J$ with $y \ne b$, and put $x := f^{-1}(y)$. Since $f$ is strictly monotone it is injective, so $y \ne b$ forces $x \ne a$. Hence $f(x) - f(a) = y - b \ne 0$, and the following rewriting is legitimate:

$$\frac{f^{-1}(y)-f^{-1}(b)}{y-b} = \frac{x-a}{f(x)-f(a)} = \left(\frac{f(x)-f(a)}{x-a}\right)^{-1} .$$

Now let $y \to b$. Since $f^{-1}$ is continuous at $b$ we have $x = f^{-1}(y) \to f^{-1}(b) = a$, and as observed above, $x \ne a$ is maintained as long as $y \ne b$. It is this condition — that the value never coincides with $a$ — that allows us to substitute directly the limit as $x \to a$ of a function of the variable $x$ (we are using the composition property of §2 with the punctured-neighborhood proviso). Since $f$ is differentiable at $a$,

$$\frac{f(x)-f(a)}{x-a} \longrightarrow f'(a) \ne 0 ,$$

and by the limit law for quotients (in the case where the limit of the denominator is not $0$) its reciprocal converges to $\frac{1}{f'(a)}$. Hence $f^{-1}$ is differentiable at $b$ with $(f^{-1})'(b) = \frac{1}{f'(a)}$. Substituting $a = f^{-1}(b)$ gives the last displayed form.

∎

The hypothesis $f'(a) \ne 0$ cannot be dropped. The function $f(x)=x^3$ is strictly increasing on $\mathbb{R}$ with $f'(0)=0$, but its inverse $f^{-1}(y)=y^{1/3}$ has difference quotient $\frac{y^{1/3}}{y} = y^{-2/3} \to +\infty$ at $y=0$ and is not differentiable there. The tangent becomes vertical.

Example 8.2（Logarithm, square root, arctangent）

(a) The natural logarithm. First let us check that $\exp$ is strictly increasing on $\mathbb{R}$. If $t > 0$, every term of the series $\exp(t) = 1 + t + \frac{t^2}{2} + \cdots$ is positive, so $\exp(t) > 1$. Also, the addition formula gives $\exp(x)\exp(-x) = \exp(0) = 1$, so $\exp(x) \ne 0$; and $\exp$ is continuous by Theorem 5.4 (4) together with Theorem 3.4, so combining $\exp(0)=1 > 0$ with the intermediate value theorem yields $\exp > 0$. Therefore, for $x < y$, $\exp(y) - \exp(x) = \exp(x)\bigl(\exp(y-x) - 1\bigr) > 0$. That the range is $(0,\infty)$ we leave to the chapter on series; we write $\log$ for the inverse function. By Theorem 5.4 (4), $\exp' = \exp$ never vanishes, so Proposition 8.1 applies and, for $b > 0$,

$$(\log)'(b) = \frac{1}{\exp(\log b)} = \frac{1}{b} .$$

(b) The square root. Restricting $f(x)=x^2$ to $I=(0,\infty)$ makes it strictly increasing with $J=(0,\infty)$, and the inverse is $f^{-1}(y)=\sqrt{y}$. The derivative $f'(x)=2x$ does not vanish on $I$. Hence, for $b > 0$,

$$\left.\frac{d}{dy}\sqrt{y}\right|_{y=b} = \frac{1}{2\sqrt{b}} .$$

(c) The arctangent. Restricting $\tan$ to $I = (-\pi/2, \pi/2)$ makes it strictly increasing with range $\mathbb{R}$. We write $\arctan$ for its inverse. By Example 6.2 (b) we have $(\tan)'(x) = 1+\tan^2 x > 0$, so Proposition 8.1 applies and, for $b \in \mathbb{R}$,

$$(\arctan)'(b) = \frac{1}{1+\tan^2(\arctan b)} = \frac{1}{1+b^2} .$$

We merely substituted $\tan(\arctan b) = b$, but the interesting thing is that no trigonometric function survives on the right.

Now that we have the logarithm, we can also differentiate real powers. For $x > 0$ and $\alpha \in \mathbb{R}$, the standard definition is $x^{\alpha} := e^{\alpha \log x}$. By the chain rule Theorem 7.2 and Example 8.2 (a),

$$(x^{\alpha})' = e^{\alpha\log x}\cdot \frac{\alpha}{x} = x^{\alpha}\cdot\frac{\alpha}{x} = \alpha x^{\alpha-1}$$

so the formula of Theorem 5.4 (1) has been extended to arbitrary real exponents. Similarly, setting $a^x := e^{x\log a}$ for $a > 0$ gives $(a^x)' = a^x \log a$. That the coefficient $\log a$ equals $1$ only when $a=e$ is one explanation of why $e$ is chosen as the base.

9. Exercises

Exercise 9.1Easy

Let $f(x)=\sqrt{x}$.

(1) Find the derivative for $x > 0$ from the definition (the limit of the difference quotient).

(2) Show that $f$ is not right-differentiable at $x=0$; that is, show that $\lim_{h\to+0}\frac{f(h)-f(0)}{h}$ does not exist as a finite value.

Solution

(1) Fix $x > 0$ and take $|h|$ small enough that $x + h > 0$. For $h \ne 0$, rationalizing the numerator gives

$$\frac{\sqrt{x+h}-\sqrt{x}}{h} = \frac{(\sqrt{x+h}-\sqrt{x})(\sqrt{x+h}+\sqrt{x})}{h(\sqrt{x+h}+\sqrt{x})} = \frac{(x+h)-x}{h(\sqrt{x+h}+\sqrt{x})} = \frac{1}{\sqrt{x+h}+\sqrt{x}}$$

(since $x > 0$, the denominator is positive and we have not divided by $0$). As $\sqrt{\ }$ is continuous on $(0,\infty)$, we have $\sqrt{x+h}\to\sqrt{x}$ as $h \to 0$, so by the limit law for quotients

$$f'(x) = \frac{1}{2\sqrt{x}} .$$

This amounts to verifying Example 8.2 (b) directly, without Proposition 8.1.

(2) For $h > 0$ we have $\frac{\sqrt{h}-0}{h} = \frac{1}{\sqrt{h}}$. Given any $M > 0$, the inequality $0 < h < 1/M^2$ implies $\frac{1}{\sqrt h} > M$, so this difference quotient diverges to $+\infty$ as $h \to +0$ and has no finite limit. Hence $f$ is not (right-)differentiable at $0$. The tangent to the graph at the origin is vertical.

Exercise 9.2Standard

For real numbers $a, b$ define

$$f(x) = \begin{cases} x^2 + ax + b & (x \le 1) \\ 3x & (x > 1) \end{cases}$$

Find all $a, b$ for which $f$ is differentiable at $x=1$.

Solution

Necessary condition (continuity). By Theorem 3.4, differentiability at $x=1$ requires continuity there. We have $f(1) = 1+a+b$, while the right-hand limit is $\lim_{x\to1+0}3x = 3$, so

$$1 + a + b = 3$$

that is, $b = 2-a$ is forced. We call this condition (C) below.

Computing the one-sided difference quotients. Under (C) we have $f(1)=3$. For $h > 0$,

$$\frac{f(1+h)-f(1)}{h} = \frac{3(1+h)-3}{h} = \frac{3h}{h} = 3$$

so the right-hand limit is $3$. For $h < 0$ (with $|h|$ small) we have $1+h \le 1$, so

$$\frac{f(1+h)-f(1)}{h} = \frac{(1+h)^2 + a(1+h) + b - 3}{h} = \frac{(1 + 2h + h^2) + a + ah + b - 3}{h} .$$

By (C) we have $1 + a + b - 3 = 0$, so the numerator is $2h + h^2 + ah$; dividing by $h$ gives $2 + h + a$, which converges to $2+a$ as $h \to -0$.

Conclusion. Differentiability is the condition that the two one-sided limits agree, so $2 + a = 3$, that is, $a = 1$; and (C) then gives $b = 1$. In this case $f'(1)=3$. Conversely, for these values of $a,b$ the computation above confirms that both one-sided limits equal $3$, so $a=b=1$ is the required answer.

Exercise 9.3Standard

For $x > 0$ let

$$y = \frac{(x^2+1)^3 \sqrt{x}}{(x+2)^5} .$$

Find $y'$ by logarithmic differentiation, that is, by taking logarithms of both sides and then differentiating. Indicate explicitly where the chain rule is used.

Solution

For $x > 0$ we have $y > 0$, so $\log y$ is defined, and by the properties of the logarithm

$$\log y = 3\log(x^2+1) + \frac{1}{2}\log x - 5\log(x+2) .$$

Differentiate the left-hand side as a function of $x$. It is the composite of $u \mapsto \log u$ with $x \mapsto y(x)$, so by Theorem 7.2 and Example 8.2 (a) we get $\frac{d}{dx}\log y(x) = \frac{y'(x)}{y(x)}$. Each term on the right is likewise a composite, with $\frac{d}{dx}\log(x^2+1) = \frac{2x}{x^2+1}$, $\frac{d}{dx}\log x = \frac1x$ and $\frac{d}{dx}\log(x+2) = \frac{1}{x+2}$. Collecting these by linearity Theorem 6.1 (1),

$$\frac{y'}{y} = \frac{6x}{x^2+1} + \frac{1}{2x} - \frac{5}{x+2} .$$

Therefore

$$y' = \frac{(x^2+1)^3\sqrt{x}}{(x+2)^5}\left(\frac{6x}{x^2+1} + \frac{1}{2x} - \frac{5}{x+2}\right) .$$

Using the product and quotient rules directly leads to the same answer, but for expressions mixing products, quotients and powers, logarithmic differentiation takes considerably less work.

Exercise 9.4Hard

Prove by induction on $n$ that if $f, g$ are $n$ times differentiable on an open interval $I$, then so is the product $fg$, and

$$(fg)^{(n)} = \sum_{k=0}^{n} \binom{n}{k} f^{(k)} g^{(n-k)}$$

(the Leibniz formula).

Solution

For the fine points of induction, see the principle of mathematical induction（Theorem 3.2）［Techniques of Proof］ in Techniques of proof: induction and proof by contradiction.

The case $n=1$. By Theorem 6.1 (2) we have $(fg)' = f'g + fg'$, and the right-hand side agrees with $\binom{1}{0}f^{(0)}g^{(1)} + \binom{1}{1}f^{(1)}g^{(0)}$ (since $\binom10 = \binom11 = 1$ and $f^{(0)}=f$).

Assuming the case $n$, proving the case $n+1$. Let $f, g$ be $n+1$ times differentiable. By the induction hypothesis,

$$(fg)^{(n)} = \sum_{k=0}^{n}\binom{n}{k} f^{(k)}g^{(n-k)}$$

and each $f^{(k)}, g^{(n-k)}$ on the right is differentiable once more. Differentiating both sides and using linearity Theorem 6.1 (1) and the product rule Theorem 6.1 (2),

$$(fg)^{(n+1)} = \sum_{k=0}^{n}\binom{n}{k}\Bigl( f^{(k+1)}g^{(n-k)} + f^{(k)}g^{(n-k+1)} \Bigr) = \sum_{k=0}^{n}\binom{n}{k} f^{(k+1)}g^{(n-k)} + \sum_{k=0}^{n}\binom{n}{k} f^{(k)}g^{(n+1-k)} .$$

Substituting $j = k+1$ in the first sum turns it into $\sum_{j=1}^{n+1}\binom{n}{j-1}f^{(j)}g^{(n+1-j)}$, and rewriting the index of the second sum as $j$ gives $\sum_{j=0}^{n}\binom{n}{j}f^{(j)}g^{(n+1-j)}$. Separating off the term $j=0$ (which occurs only in the second sum) and the term $j=n+1$ (which occurs only in the first) and collecting the part with $1 \le j \le n$,

$$(fg)^{(n+1)} = f^{(0)}g^{(n+1)} + \sum_{j=1}^{n}\left(\binom{n}{j-1}+\binom{n}{j}\right) f^{(j)}g^{(n+1-j)} + f^{(n+1)}g^{(0)} .$$

Using Pascal’s rule $\binom{n}{j-1}+\binom{n}{j} = \binom{n+1}{j}$ and noting that $\binom{n+1}{0} = \binom{n+1}{n+1} = 1$, the two separated terms fit exactly into the sum as the terms $j=0$ and $j=n+1$. Hence

$$(fg)^{(n+1)} = \sum_{j=0}^{n+1}\binom{n+1}{j} f^{(j)}g^{(n+1-j)}$$

which is the case $n+1$. By induction the formula holds for every positive integer $n$.

That this has exactly the same shape as the binomial theorem $(x+y)^n = \sum_k \binom nk x^k y^{n-k}$ is no coincidence. Both come from the same structure: an operation on two objects is repeated $n$ times, and one counts which of the two is chosen at each repetition.

References

• Sugiura Mitsuo, Kaiseki Nyūmon I (Introduction to Analysis I), University of Tokyo Press, 1980 (in Japanese) — Chapter II, “Differentiation”. A standard textbook thoroughgoing in its use of $\varepsilon$-$\delta$; the development in this article broadly follows its plan.
• Takagi Teiji, Kaiseki Gairon (A Course of Analysis), revised 3rd ed., Iwanami Shoten, 1983 (in Japanese) — Chapter 2, “Differentiation”. A classic, careful in its account of the relation between the derivative and the tangent line.
• W. Rudin, Principles of Mathematical Analysis, 3rd ed., McGraw-Hill, 1976 — Chapter 5, “Differentiation”. The proof of the chain rule uses the same auxiliary function as the one given here.
• M. Spivak, Calculus, 4th ed., Publish or Perish, 2008 — Chapters 9–10. The motivating discussion leading up to the definition of the derivative is treated in great detail.
• G. Berkeley, The Analyst, 1734 — the original critique of infinitesimals; the primary source for the historical remarks in §1.

Appendix: How to read Leibniz notation

The notation $\frac{dy}{dx}$ looks like the ratio of two quantities $dy$ and $dx$. This appearance helps with computation, but it is also a source of confusion for beginners. Let us set down the accurate position at this stage.

First, $\frac{d}{dx}$ is a single symbol denoting the operator “differentiate with respect to $x$”, and $\frac{dy}{dx}$ is the result of applying that operator to $y$. We are not splitting it into a numerator and a denominator and giving each a meaning of its own. Accordingly, the reason $\frac{dy}{dx} = \frac{dy}{du}\frac{du}{dx}$ looks like cancellation is that the statement of Theorem 7.2 happens to have that shape. Cancellation is not a proof.

Second, there is nonetheless a reason why the notation can be trusted. By Theorem 4.1, differentiability means that $\Delta y = f'(a)\Delta x + o(\Delta x)$. If we agree to read $dx$ as “the small increment in $x$ itself” and $dy$ as “the increment in $y$ predicted by the linear approximation, namely $f'(a)\,dx$”, then $dy = f'(a)\,dx$ acquires meaning as a definition, and dividing both sides by $dx$ does give $\frac{dy}{dx}=f'(a)$. Under this reading, the cancellation in the chain rule is likewise justified as the composition of linear approximations. Generalizing this standpoint leads to the theory of differential forms, which comes into its own in multivariable integration (change of variables and Jacobians).

Third, $\frac{d^2y}{dx^2}$ has nothing whatever to do with $\left(\frac{dy}{dx}\right)^2$. It merely expresses, in the notation of a squared operator, the act of applying $\frac{d}{dx}$ twice. This one piece of notation is not to be read as a fraction.

The toolkit of differentiation is now complete. In the next chapter we pass to The mean value theorem and Taylor’s theorem, which extracts global behavior of a function from the pointwise information carried by the derivative.