Maxwell's Demon: Can a Machine Run on Information?

0. Key points

• Maxwell’s demon is a fictitious being that opens and closes a small door in a partition, sorting fast molecules from slow ones. Since it creates a temperature difference without doing work, it appears to violate the second law of thermodynamics.
• In 1929 Szilard stripped the demon down to a box containing a single molecule. This device extracts $k_B T \ln 2$ of work per cycle (about $2.9 \times 10^{-21}$ J at room temperature) — exactly in exchange for one bit of information.
• For a long time the standard explanation was that “observation costs energy, so the books balance”. That explanation is wrong. Measurement can in principle be carried out at arbitrarily small cost.
• The real price is erasure. By Landauer’s principle, erasing one bit of memory requires dumping at least $k_B T \ln 2$ of heat into an environment at temperature $T$. The moment the demon clears its memory to start the next cycle, the debt is repaid in full.
• The second law survives. But its line of defence turned out to run through territory no one in the nineteenth century imagined: the fact that information is a physical thing.

1. Motivation: a prank set by a nineteenth-century physicist

Hot coffee cools down. Cold coffee never spontaneously heats up. The second law of thermodynamics is this utterly obvious fact cast in the form of a law.

In the second half of the nineteenth century, however, physicists had noticed something awkward. Coffee cools because water molecules collide with the cup and with air molecules and hand over kinetic energy. Yet Newton’s laws, which govern the motion of each individual molecule, are deterministic（Definition 2.2）［Laplace's Demon and Determinism］ and carry no direction of time. Play a film of colliding molecules backwards and nothing in mechanics is violated. Every part is symmetric under time reversal, and yet the assembled machine runs only one way. How can that be?

In 1867 James Clerk Maxwell posed a thought experiment aimed at exactly this puzzle in a letter to his friend Peter Guthrie Tait. The scenario, later made public in Theory of Heat (1871), runs as follows.

Divide a box of gas into two chambers with a partition, and put a small door in the partition. Station next to the door “a being whose faculties are so sharpened that he can follow every molecule”. This being watches the speed of each approaching molecule: if a fast molecule comes from the right, it opens the door and lets it through to the left; if a slow molecule comes from the left, it opens the door and lets it through to the right. Otherwise the door stays shut. The door is light and frictionless, so opening and closing it costs (almost) no energy.

After a while the left chamber contains only fast molecules and the right chamber only slow ones. Since the temperature of a gas is the mean kinetic energy of its molecules, this means the left chamber has become hot and the right chamber cold. A temperature difference has appeared without any work being done. Connect a heat engine across that difference and work can be extracted at will.

Maxwell himself never called this being a demon; the name was supplied by Lord Kelvin. Maxwell’s point was that the second law is merely a statistical law that cannot be applied to individual molecules, not that such a being could actually be built. Even so, the question remains: why can it not be built? Physics needed more than a century to state the reason properly.

Note

Both are called demons, but Laplace’s demon (Definition 3.1［Laplace's Demon and Determinism］) knows the position and velocity of every particle in the universe and computes the future; it is a thought experiment about determinism. Maxwell’s demon is far more modest — it only needs to know the speed of the one molecule passing in front of it. Being modest is exactly what made it the harder of the two.

2. Preliminaries: entropy and the second law

Two pieces of vocabulary need to be pinned down before we go on.

Axiom 2.1（Second law of thermodynamics (Kelvin's statement)）

There is no device that performs a cyclic process in which it absorbs heat from a heat bath at a single temperature $T$, converts all of it into work, and returns to its own initial state.

The clause “returns to its own initial state” is the crux. Doing it once is easy: connect a gas in a piston to the bath and let it expand slowly, and all the heat absorbed becomes work. But then the gas is left with a larger volume, and the device has not returned to its initial state. To restore it we must compress the gas, and that means paying work back. The claim of the second law is that there is no way to escape this repayment.

Entropy is the tool that turns this law into a quantity.

Definition 2.2（Boltzmann entropy）

Let $W$ be the number of microstates corresponding to a macroscopic state of a system (a state specified by temperature, volume, and so on). The entropy of that state is defined by

$$S = k_B \ln W$$

where $k_B = 1.380649 \times 10^{-23}\ \mathrm{J/K}$ is Boltzmann’s constant.

More generally, when microstate $i$ occurs with probability $p_i$, the entropy is

$$S = -k_B \sum_i p_i \ln p_i$$

(when all $p_i$ are equal to $1/W$, this reduces to $k_B \ln W$).

Because it has the form $\ln W$, entropy measures how many possibilities a state encompasses. Turned around: a state of low entropy is a state about which we know a great deal. This rereading is what will carry us directly into the discussion of information.

Written in terms of entropy, the second law says that the entropy of an isolated system as a whole cannot decrease. When the demon separates the box into a hot side and a cold side, the number of possible molecular arrangements drops, so the entropy falls. If nothing outside the box compensates for that drop, the second law is violated.

3. Blueprint of the demon

Let us draw the demon’s apparatus a little more carefully.

The demon's box. It opens and closes the small door in the partition according to the speed of the approaching molecule

First, let us see what sort of thing the demon has to sort.

Example 3.1（How fast are air molecules at room temperature?）

A nitrogen molecule $\mathrm{N_2}$ has mass $m = 28 \times 1.661 \times 10^{-27} \approx 4.65 \times 10^{-26}$ kg. A molecule of a gas at temperature $T$ carries on average $\frac{3}{2}k_B T$ of kinetic energy, so the root-mean-square speed $v_{\mathrm{rms}}$ follows from $\frac{1}{2} m v_{\mathrm{rms}}^2 = \frac{3}{2} k_B T$:

$$v_{\mathrm{rms}} = \sqrt{\frac{3 k_B T}{m}} = \sqrt{\frac{3 \times 1.380649\times10^{-23} \times 300}{4.65 \times 10^{-26}}} \approx \sqrt{2.67\times 10^{5}} \approx 5.2 \times 10^{2}\ \mathrm{m/s}$$

At $T = 300$ K (about 27 °C) this is roughly 520 m/s, some 1900 km per hour. And this is only an average: actual speeds are spread out according to the Maxwell distribution, with molecules in the 200 m/s range and molecules above 1000 m/s flying around at the same time.

Moreover, in air at one atmosphere the distance a molecule travels before its next collision (the mean free path) is only about 70 nm. Dividing 520 m/s by 70 nm gives more than $10^9$ collisions per second. The demon must identify, one at a time, targets that fly at one and a half times the speed of sound and change direction every nanosecond.

Technically this is hopeless. But what physics wants to know is not whether something is difficult but whether it is forbidden in principle. So let us check what would happen if the demon could work ideally.

Proposition 3.2（A successful demon yields a perpetual motion machine of the second kind）

Suppose a box of gas at uniform temperature $T$ is fitted with a device that sorts molecules by speed without consuming energy, and that after finitely many operations the device always returns to its initial state. Then one can construct a cyclic device that absorbs heat from a bath at a single temperature and converts all of it into work, contradicting Axiom 2.1.

Proof（Proposition 3.2）

Let the device run until the left chamber is at temperature $T_h$ and the right chamber at $T_c$ (with $T_c < T_h$). By hypothesis this is achieved without receiving any work from outside.

Now connect a Carnot engine between the two chambers. The Carnot engine takes heat $Q_h$ from the hot side, discards $Q_c$ to the cold side, and delivers the difference $Q_h - Q_c > 0$ as work. Net work has been obtained.

Running the engine shrinks the temperature difference, and eventually the whole box returns to the uniform temperature $T$. Setting the demon to work again recreates the difference. Under the assumption that the demon returns to its initial state, this repetition can continue indefinitely.

Looking at one full cycle: the gas in the box is back at its original uniform temperature $T$, the device is back in its original state, and nothing has emerged externally except net work. By conservation of energy (the first law of thermodynamics), the only possible source of that work is the heat that the gas in the box already possessed. So this is precisely a cyclic device that absorbs heat from a bath at a single temperature and turns all of it into work — exactly the thing Axiom 2.1 declares does not exist.

∎

What Proposition 3.2 shows is that there are only two ways out. Either the second law really can be broken, or the demon must violate one of the assumptions. Physics chose the second, but identifying which assumption fails took a long time.

4. Szilard’s engine: slimming the demon down to a single molecule

The first person to push the argument forward was Leo Szilard. In 1929 he stripped away every decoration and reduced the demon to a box with a single molecule. There is no need even to measure speed: it suffices to know whether the molecule is on the left or on the right.

Definition 4.1（Szilard's engine）

A box of volume $V$ contains exactly one molecule and is in contact with a heat bath at temperature $T$. The device whose cycle consists of the following operations is called Szilard’s engine.

1. Insert a partition at the middle of the box, dividing it into two chambers of volume $V/2$. The molecule is confined to one of them.
2. Observe which chamber the molecule is in and record the result in a one-bit memory.
3. According to the record, attach a piston on the side where the molecule is.
4. Exchanging heat with the bath, push the piston back slowly (quasi-statically), returning the volume from $V/2$ to $V$. Work is extracted during this step.
5. Remove the partition and restore the device to its initial state.

flowchart TD
A["1. One molecule in a box of volume V. Position unknown"] --> B["2. Insert a partition at the middle (zero work)"]
B --> C["3. Observe whether it is left or right"]
C --> D["4. Record the result in a one-bit memory"]
D --> E["5. Attach a piston on the molecule's side"]
E --> F["6. Isothermal quasi-static expansion. Extract work kT ln2"]
F --> G["7. Erase the memory and return to the initial state"]
G --> A

One cycle of Szilard's engine. Step 7 was the answer to a century-old puzzle

Let us compute the work extracted in step 4.

Theorem 4.2（Work of Szilard's engine）

In the engine of Definition 4.1, the work extracted during the isothermal quasi-static expansion of step 4 is

$$W = k_B T \ln 2$$

whichever chamber the molecule was in. Moreover, an equal amount of heat $Q = k_B T \ln 2$ is absorbed from the bath at temperature $T$, whose entropy therefore decreases by $k_B \ln 2$.

Proof（Theorem 4.2）

There is only one molecule, but it may be treated as an ideal gas with particle number $N = 1$. Substituting $N = 1$ into the equation of state $pV = N k_B T$ gives

$$p = \frac{k_B T}{V}$$

Here $p$ is the time average of the pressure produced by the molecule striking the piston repeatedly.

When the piston moves so that the volume goes from $V'$ to $V' + dV'$, the work done by the gas on the outside is $p\,dV'$. In step 4 the volume changes from $V/2$ to $V$, so

$$W = \int_{V/2}^{V} p \, dV' = \int_{V/2}^{V} \frac{k_B T}{V'} \, dV' = k_B T \bigl[\ln V'\bigr]_{V/2}^{V} = k_B T \ln \frac{V}{V/2} = k_B T \ln 2$$

If the molecule was in the right chamber, the initial and final volumes of the expansion are the same $V/2$ and $V$, so the value is unchanged.

Now for the heat. The internal energy of an ideal gas depends only on temperature, so it does not change in an isothermal process ($\Delta U = 0$). The first law $\Delta U = Q - W$ gives $Q = W = k_B T \ln 2$. The bath released this heat while staying at temperature $T$, so its entropy change is

$$\Delta S_{\text{bath}} = \frac{-Q}{T} = -k_B \ln 2$$∎

Example 4.3（How large is this work?）

At $T = 300$ K,

$$k_B T \ln 2 = 1.380649\times10^{-23} \times 300 \times 0.6931 \approx 2.87 \times 10^{-21}\ \mathrm{J}$$

In electronvolts that is $2.87\times10^{-21} / 1.602\times10^{-19} \approx 0.018$ eV, about half the mean kinetic energy $\frac{3}{2}k_B T \approx 6.2\times10^{-21}$ J of a single air molecule.

Processing one mole’s worth, that is $6.02\times10^{23}$ bits, would give $6.02\times10^{23} \times 2.87\times10^{-21} \approx 1.7\times10^{3}$ J, roughly 1.7 kJ — enough to run a 100 W light bulb for 17 seconds. Even if the demon works, no electric utility is going bankrupt. The problem is not the magnitude but the sign.

The consequence of Theorem 4.2 is serious. Each cycle lowers the entropy of the bath by $k_B \ln 2$, while the molecule in the box returns to the same state as before, “somewhere in a volume $V$”. If nothing happens on the demon’s side, the entropy of the whole universe decreases by $k_B\ln 2$ per cycle, without end.

5. Hunting the culprit: does observation cost anything?

The explanation that held sway for a long time blamed observation. In 1951 Léon Brillouin argued as follows. To see a molecule inside a pitch-dark box, the demon must carry a torch. It bounces one photon off the molecule and looks at the reflection. To stand out against the ambient thermal radiation, the photon must have an energy well above $k_B T$. That photon is eventually absorbed and turned into heat, so each observation generates at least $k_B T$ worth of entropy. Hence the books balance.

The argument is plausible but not decisive. What Brillouin showed is that observing in this particular way costs something, not that every observation costs something.

During the 1970s and 1980s, Charles Bennett pressed exactly this point.

Remark 5.1（Measurement is free in principle）

Bennett constructed explicit models of measurement that generate no entropy outside the system at all. The key is to design the measurement as a reversible operation.

Measurement is an operation that correlates “the state of the object” with “the state of the memory” (create the same correlation in quantum mechanics and it is entanglement（Definition 3.2）［Schrödinger's Cat］). With the memory in its initial state 0: if the object is on the left, leave it at 0; if on the right, flip it to 1. The correspondence

$$(\text{left}, 0) \mapsto (\text{left}, 0), \qquad (\text{right}, 0) \mapsto (\text{right}, 1)$$

is one-to-one. An operation whose input states correspond one-to-one with its output states can in principle also be run backwards, and can be carried out as a frictionless slow process with arbitrarily little dissipation. Measurement therefore carries no unavoidable cost of the kind Brillouin had in mind.

See Bennett (1982) for details.

There is another objection: it is suspicious to invoke an intelligent being at all — what about a purely mechanical demon? There is a separate answer for that.

Remark 5.2（Why a spring-loaded demon does not work）

In 1912 Marian Smoluchowski considered a spring-loaded trapdoor: a small door that opens only one way. A molecule hitting it from the right pushes it open; one hitting from the left leaves it shut. No intelligence required.

But the door itself sits in an environment at temperature $T$. If the spring is weak enough and the door light enough to be moved by a molecule, then the door’s own thermal motion will make it flap open by itself. Make the spring stiff enough to resist thermal motion and a single molecular impact will no longer open it. The sensitivity needed for sorting and the sturdiness needed to resist thermal noise cannot coexist.

Richard Feynman develops the same argument in detail with his “ratchet and pawl” model (The Feynman Lectures on Physics, Vol. I). For the ratchet to turn only one way, the pawl must be held down more firmly than thermal motion can lift it; that requires cooling the pawl; and cooling means creating a temperature difference, which lands us back at an ordinary heat engine.

Observation is innocent. Mechanical demons fail. So where in Szilard’s engine is the debt hidden? The answer lay in a step that no one had put on the ledger.

6. Landauer’s principle: the price of forgetting

Read Definition 4.1 again. Step 5 says “restore the device to its initial state”, but the demon’s memory has not been restored. One cycle ago it held nothing; now it holds “left” or “right”. For the device genuinely to return to its initial state, the memory must be erased.

What Rolf Landauer pointed out in 1961 is that this erasure is exactly what demands a physical price.

Definition 6.1（Logical irreversibility）

An operation is logically reversible if the state before the operation is uniquely determined by the state after it. Otherwise the operation is logically irreversible.

Erasing one bit — the operation that maps both state 0 and state 1 to state 0,

$$0 \mapsto 0, \qquad 1 \mapsto 0$$

— is logically irreversible, since the input cannot be recovered from the output 0.

Why should a logical property summon anything physical, such as heat? The key is the following fact.

Lemma 6.2（Liouville's theorem (conservation of phase-space volume)）

In a system evolving according to Hamiltonian mechanics, the volume of a region of phase space (the space spanned by the positions and momenta of all particles; see Definition 2.1［Laplace's Demon and Determinism］) does not change under time evolution. The region may be stretched into a complicated shape, but its volume is preserved.

Remark 6.3（On the proof of this lemma）

Lemma 6.2 is a standard result of analytical mechanics; it reduces to the fact that the phase-space flow generated by the canonical equations is incompressible (the divergence of the velocity field vanishes). Proofs can be found in Landau and Lifshitz’s Mechanics or in the opening chapters of statistical mechanics textbooks. Here we use it as a fact.

Theorem 6.4（Landauer's principle）

Consider a one-bit memory inside a system in contact with an environment at temperature $T$, holding the value 0 or 1 with equal probability, and an operation that brings it to a fixed state (erases it) independently of its initial value. The heat $Q$ released into the environment then satisfies

$$Q \ge k_B T \ln 2$$

Equality holds in the limit of an infinitely slow, quasi-static operation.

Proof（Theorem 6.4）

Compare the region occupied by the memory’s state before and after erasure.

Before erasure the memory may be either 0 or 1. By Definition 2.2, since the two states are equally likely, the entropy corresponding to this uncertainty is

$$S_{\text{before}} = -k_B\left(\tfrac12 \ln \tfrac12 + \tfrac12 \ln \tfrac12\right) = k_B \ln 2$$

After erasure the state is definitely 0, so $S_{\text{after}} = 0$. The entropy of the memory has therefore decreased by $k_B \ln 2$.

This is where Lemma 6.2 bites. Memory and environment together obey Hamiltonian mechanics, so the total phase-space volume cannot shrink. If the region on the memory side has been squeezed to half its size, that volume has not vanished; it must have been pushed out into an enlarged region on the environment side. In other words, the decrease in the memory’s entropy must be at least compensated by an increase in the environment’s entropy:

$$\Delta S_{\text{environment}} \ge k_B \ln 2$$

The environment is a bath at temperature $T$, so an inflow of heat $Q$ raises its entropy by $Q/T$. Hence

$$\frac{Q}{T} \ge k_B \ln 2 \quad\Longrightarrow\quad Q \ge k_B T \ln 2$$

That equality is actually attainable can be checked with an explicit erasure procedure. Represent the one-bit memory by a box of volume $V$ with a partition confining a single molecule to one side (left is 0, right is 1). Erasure means “collect the molecule on the left wherever it was”. First remove the partition. The molecule roams freely through volume $V$, and at this point the left/right information is lost. Then compress the volume isothermally and quasi-statically to $V/2$, after which the molecule is certainly in the left chamber. The work required for this compression, retracing the computation of Theorem 4.2 in reverse, is

$$W_{\text{compression}} = -\int_{V}^{V/2} \frac{k_B T}{V'}\,dV' = k_B T \ln 2$$

and since the process is isothermal, the same amount of heat $k_B T \ln 2$ is dumped into the bath. Performed quasi-statically, the procedure attains the bound exactly.

∎

Caution

Landauer’s principle imposes a cost on erasure, not on writing, not on reading, and not on computation in general. Reversible logical operations — for instance a computation that preserves its input and appends the result — can in principle be performed without releasing heat. The crude slogan “information processing always requires energy” is inaccurate in this sense.

Now for the accounts.

Corollary 6.5（The demon's ledger balances）

When Szilard’s engine of Definition 4.1 completes one full cycle including erasure of the memory, the net work $W_{\text{net}}$ extracted satisfies

$$W_{\text{net}} \le 0$$

The device therefore does not contradict Axiom 2.1.

Proof（Corollary 6.5）

By Theorem 4.2, the work extracted in one cycle is $W = k_B T \ln 2$.

On the other hand, restoring the device completely to its initial state requires erasing the recorded bit. By Remark 5.1 the observation itself can be done at no cost, but by Theorem 6.4 the erasure demands dumping at least $k_B T \ln 2$ of heat into the environment. By conservation of energy, supplying that heat requires investing at least the same amount of work from outside, $W_{\text{erase}} \ge k_B T \ln 2$.

Hence

$$W_{\text{net}} = W - W_{\text{erase}} \le k_B T \ln 2 - k_B T \ln 2 = 0$$

Equality holds only in the limit where every process is quasi-static, and even then the net work extracted is exactly zero.

∎

The demon was not a swindler; it had simply overlooked one page of the ledger. Whatever it earns must be paid back before it can start the next job.

Remark 6.6（Is this resolution circular?）

A sharp reader will balk here. The proof of Theorem 6.4 uses an argument in the spirit of the second law (entropy does not decrease), and the result is then used to defend the second law. Is that not circular?

John Earman and John Norton pressed exactly this criticism. On their reading there is a dilemma: deriving Landauer’s principle from the second law and then using it to exorcise the demon begs the question, while deriving it independently of statistical mechanics would make the demon argument superfluous.

The standard position today is to derive Landauer’s principle independently, from mechanics and statistical mechanics, as in the proof above that starts from Lemma 6.2. In recent years, fluctuation theorems that extend the second law by an information term have been established for processes involving finite-time measurement and feedback, so that the balance sheet of a system including a demon can be written as an equality rather than an inequality. The philosophical debate is not entirely closed, but as physics the accounting is.

7. Demons in the laboratory, and your computer

None of this remains a mere thought experiment. In the twenty-first century, both halves were confirmed in the laboratory.

Example 7.1（Measuring the Landauer bound (2012)）

Antoine Bérut and colleagues trapped a colloidal particle about 2 micrometres across, suspended in water, in a double-well potential created with a laser. The particle in the left well means 0, in the right well means 1. That is a one-bit memory.

By lowering the barrier, tilting the potential to one side, and raising the barrier again, they performed the erasure “collect the particle in the left well wherever it started”, and computed the heat the particle passed to the water from records of its position and velocity. The slower the operation, the less heat was dissipated, approaching $k_B T \ln 2$ in the limit and never falling below it. This is a direct confirmation both that the equality in Theorem 6.4 is attainable and that the bound cannot be broken (Bérut et al., Nature 483 (2012)).

Two years earlier, Shoichi Toyabe and colleagues had performed the converse experiment: using observation and feedback to drive a Brownian particle up a slope, extracting energy from information (Toyabe et al., Nature Physics 6 (2010)). Demons exist. But only demons that keep their books.

Example 7.2（Does erasing a gigabyte warm the room?）

Erasing 1 GB $= 8\times10^{9}$ bits completely at room temperature releases, as an unavoidable minimum,

$$8\times10^{9} \times 2.87\times10^{-21} \approx 2.3\times10^{-11}\ \mathrm{J}$$

That is 0.00000000002 joules. Given that raising 1 g of water by one degree takes 4.2 J, it is utterly negligible.

Meanwhile, the energy that present-day semiconductor devices spend on switching a single bit is estimated at around $10^{-17}$ J — some thousands of times the Landauer bound. In other words, the heat produced by today’s computers is explained almost entirely by an engineering reason, charging and discharging wires, and not by the fundamental reason of erasing information (for another wall that miniaturisation is hitting, tunnelling leakage through the gate oxide, see Example 3.9［Physics in Everyday Life］).

Even so, the energy per device has fallen steadily throughout history. If the trend continues, the engineering waste will eventually be squeezed out and we will hit the physical wall of the Landauer bound. The only route left to designers then is computation without erasure — reversible computing. The prank Maxwell put in a letter now appears in the far-future roadmaps of semiconductor design.

Tip

The moral of this story fits in Landauer’s slogan: information is physical. Information is not a mathematical abstraction; it always rides on some physical degree of freedom, and that degree of freedom obeys the laws of thermodynamics. That is precisely why erasing information comes with a price tag. Physics still has many unsolved problems (in the budget of the universe, 95 % of the entries are still blank; see Example 2.2［Unsolved Problems in Physics］), but the affair of the demon is one of the rare cases settled after a century.

8. Exercises

Exercise 8.1Easy

Find the minimum heat required to erase one bit at $T = 300$ K, and compare it with “the energy of moving one electron through a potential difference of 1 V” ($1\ \mathrm{eV} = 1.602\times10^{-19}$ J). What is the ratio?

Solution

By Theorem 6.4 the minimum heat is

$$k_B T \ln 2 = 1.380649\times10^{-23} \times 300 \times 0.6931 \approx 2.87\times10^{-21}\ \mathrm{J}$$

The ratio to 1 eV is

$$\frac{2.87\times10^{-21}}{1.602\times10^{-19}} \approx 1.8\times10^{-2}$$

that is, about $1/56$. Conversely, 1 eV is roughly 56 times the Landauer bound. The typical energy scales handled by electronic circuits (a few hundred mV to a few V) are thus already one to two orders of magnitude above the Landauer bound. Real devices (around $10^{-17}$ J per bit, some thousands of times the bound) waste a further two orders of magnitude beyond that because a single switching event moves a great many electrons.

Exercise 8.2Standard

Suppose that in Szilard’s engine the partition is inserted not at the middle but at the position where the left and right chambers have volumes in the ratio $1:3$. The probability that the molecule is in the left chamber is $1/4$, and $3/4$ for the right. Find the expected work extractable per cycle and compare it with the value in Theorem 4.2.

Solution

If the molecule was in the left chamber (volume $V/4$), expanding isothermally and quasi-statically from there to volume $V$ gives, by the same computation as in the proof of Theorem 4.2,

$$W_{\text{left}} = \int_{V/4}^{V} \frac{k_B T}{V'}dV' = k_B T \ln 4$$

and if it was in the right chamber (volume $3V/4$),

$$W_{\text{right}} = \int_{3V/4}^{V} \frac{k_B T}{V'}dV' = k_B T \ln \frac{4}{3}$$

The expectation is

$$\langle W \rangle = \frac14 k_B T \ln 4 + \frac34 k_B T \ln\frac43 = k_B T\left(\frac{1.3863}{4} + \frac{3 \times 0.2877}{4}\right) \approx 0.562\, k_B T$$

which is smaller than the value $k_B T \ln 2 \approx 0.693\, k_B T$ for a central partition.

This number is meaningful. The entropy of the distribution $(1/4, 3/4)$ (the second formula of Definition 2.2 with $k_B$ stripped off) is

$$-\left(\frac14\ln\frac14 + \frac34\ln\frac34\right) \approx 0.562$$

which matches the expected work exactly. The extractable work is exactly proportional to the amount of information gained by the observation. A biased coin — a situation where you can guess which side the molecule is on — yields little work.

Exercise 8.3Hard

Suppose the demon has $N$ bits of blank memory and runs Szilard’s engine without performing any erasure at all. (1) How much work in total can the demon extract? (2) Is the second law violated in this case? (3) What does the demon need in order to keep working forever?

Solution

(1) As long as the memory has blank space, each cycle can write a new bit into an unused region, so no erasure is needed. By Theorem 4.2 each cycle yields $k_B T \ln 2$, and $N$ cycles can be run, for a total of

$$W_{\text{total}} = N k_B T \ln 2$$

Taking $N$ large makes the extractable work arbitrarily large.

(2) It is not violated. What the second law (Axiom 2.1) forbids is converting heat entirely into work in a cyclic process in which the device returns to its initial state. Here the entropy of the bath has fallen by $N k_B \ln 2$, but the entropy of the demon’s memory has risen by the same amount. A blank $N$-bit memory has only one possible state, whereas after writing there are $2^N$ possibilities, so by Definition 2.2

$$\Delta S_{\text{memory}} = k_B \ln 2^{N} = N k_B \ln 2$$

The total change is zero and the books balance. Since the device has not returned to its initial state, this is no counterexample to the second law.

(3) Erasure. After $N$ cycles the demon’s memory is full, and to begin another cycle it must be reinitialised. By Theorem 6.4, erasing $N$ bits requires dumping at least $N k_B T \ln 2$ of heat into the environment, which in turn requires investing the same amount of work. Everything earned in (1) is repaid here in full.

So a demon with finite memory can violate the second law only temporarily. The length of the interval during which it appears to violate it is set by the capacity of the memory. This is not an exception to the second law but a fine illustration of how essential the condition “returns to its initial state” is.

References

• J. C. Maxwell, Theory of Heat, Longmans, Green, and Co., 1871 — Chapter 22 contains the description that became the prototype of the demon.
• L. Szilard, “Über die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen”, Zeitschrift für Physik 53 (1929), 840–856 — the original paper on Szilard’s engine.
• R. Landauer, “Irreversibility and heat generation in the computing process”, IBM Journal of Research and Development 5 (1961), 183–191 — the original paper on Landauer’s principle.
• C. H. Bennett, “The thermodynamics of computation — a review”, International Journal of Theoretical Physics 21 (1982), 905–940 — a review setting out that measurement can be free and that erasure is the real cost.
• H. S. Leff and A. F. Rex (eds.), Maxwell’s Demon 2: Entropy, Classical and Quantum Information, Computing, Institute of Physics Publishing, 2003 — an anthology of the principal papers, with historical commentary.
• A. Bérut et al., “Experimental verification of Landauer’s principle linking information and thermodynamics”, Nature 483 (2012), 187–189 — direct measurement of the heat of erasure.
• R. P. Feynman, R. B. Leighton, M. Sands, The Feynman Lectures on Physics, Vol. I, Addison-Wesley, 1963 — Chapter 46, “Ratchet and pawl”: why a mechanical demon does not work.
• J. Earman and J. D. Norton, “Exorcist XIV: The Wrath of Maxwell’s Demon”, Studies in History and Philosophy of Modern Physics 29 (1998) and 30 (1999) — a critical examination of the information-theoretic resolution.

Appendix: Why can the two entropies be measured in the same units?

Thermal entropy and information entropy. Throughout this article we have added and subtracted the entropy $-Q/T$ of a heat bath and the entropy $k_B \ln 2$ of a memory’s uncertainty as though it were the most natural thing in the world. Yet one quantity was born from the analysis of steam engines, and the other was introduced by Claude Shannon in 1948 to measure the capacity of a communication channel. Why may quantities of such different ancestry be entered in the same ledger?

They have the same form. Shannon defined the uncertainty of a source that emits symbol $i$ with probability $p_i$ as

$$H = -\sum_i p_i \log_2 p_i \quad [\text{bits}]$$

Compare this with the Boltzmann entropy $S = -k_B \sum_i p_i \ln p_i$ of Definition 2.2: the only differences are the base of the logarithm (2 or $e$) and the constant $k_B$ in front. Using $\log_2 x = \ln x / \ln 2$ we get the conversion

$$S = k_B \ln 2 \cdot H$$

So one bit of information corresponds to $k_B \ln 2 \approx 9.57\times10^{-24}$ J/K of entropy. The $k_B T \ln 2$ of Theorem 6.4 is nothing but this conversion multiplied by a temperature.

Is it not just a formal resemblance? For a long time some regarded the coincidence as a merely formal analogy. But what Szilard’s engine shows is that the two are exchangeable in the same currency. In Theorem 4.2 one bit of information turns into $k_B T \ln 2$ of work, and in Theorem 6.4 the reverse exchange occurs. The exchange rate is fixed at the constant $k_B \ln 2$, and converting can cost you more than the commission but never earns you a profit. The experimental confirmation of this exchangeability (Example 7.1) is what retired the analogy view.

A remaining difference. The two are nevertheless not identical. Thermodynamic entropy is defined with an implicit choice of how coarsely to describe the microstates of a system. Information entropy is a relative quantity: it depends on who knows what. In the demon problem this difference surfaces as the fact that what the demon knows changes the value of the entropy. Perhaps the greatest gift Maxwell’s demon left to physics was neither a temperature difference nor a perpetual motion machine, but the discovery that knowledge, a concept that looks subjective, is continuous with heat, a quantity that is objective.