02 Origin

How we learned that systems push back

The idea that a system can defeat the people trying to improve it is younger than you might expect. It was stated most forcefully in 1971 by Jay W. Forrester, an engineer at the Massachusetts Institute of Technology who had spent the 1950s building feedback-control systems for computers and aircraft before turning the same mathematics on companies, cities, and nations.

Forrester’s claim was blunt. He held that human intuition is reliable only for simple systems, the kind our ancestors survived among, where the cause of a trouble sits close to the trouble in both time and space. Touch a hot stove and the burn is immediate and local; the lesson is easy to learn. But the large systems we now live inside do not behave that way.

Let us be precise about his diagnosis. Forrester argued that social systems belong to a class he called multi-loop nonlinear feedback systems — webs in which dozens of causes act on each other at once, with consequences that arrive long after the action that triggered them. Our evolved instincts, he wrote, were never shaped to read such systems correctly.

This was not a casual opinion. Forrester had built computer models of cities — his 1969 book Urban Dynamics simulated them as systems of stocks and flows — and the simulations kept producing an uncomfortable result. Several of the most popular remedies for urban decay, such as building large amounts of low-cost housing, made the modeled city worse over the long run, because cheap housing drew in more low-income residents and underemployment than the local economy could absorb. The fixes treated a symptom and, in doing so, fed the underlying trap.

His framework, which came to be called system dynamics, gave a name to the experience every reformer eventually has. He called it policy resistance: the tendency of a system to absorb an intervention and return to its prior behavior, as if the effort had been quietly cancelled. It follows that understanding why systems resist is the prerequisite for designing fixes that survive.

A generation later, the environmental scientist Donella Meadows carried these ideas to a wider audience. Her primer Thinking in Systems catalogued the recurring ways systems frustrate us — she called them system traps — with policy resistance among the central ones. Her work, and Forrester’s before it, supplies the vocabulary for the rest of this essay.

04 The Derivation

How a good fix reverses itself, step by step

We can now assemble the parts into a general mechanism. The claim to derive is this: an intervention aimed at a symptom, inside a system with a balancing loop and a delay, will tend to be cancelled by the system’s own response. The reasoning runs in five steps.

1. A problem is a stock that feels too high. Congestion, prices, costs — each is a stock people want lower. The reformer observes the stock and resolves to reduce it directly.

2. The fix relieves the symptom and lowers the perceived cost of the behavior. Widen the road and each trip becomes faster; subsidize the loan and borrowing becomes cheaper. The immediate effect is genuine relief. cost of behavior ↓

3. Lower cost invites more of the behavior. When something becomes easier or cheaper, people do more of it. This is a balancing loop responding: the system pushes back toward its old level of strain. use of resource ↑

4. A delay hides the rebound. The new behavior accumulates over months or years — drivers relocate, lenders expand, habits form. Because the rebound is delayed, the early data look like success, and the fix is declared to have worked.

5. The stock returns to its strained level — or higher. Once the new behavior fully arrives, the original symptom is back. If the fix also enlarged the system (more lanes, more debt, more capacity to fill), the problem can settle above where it started. It follows that treating the symptom, not the loop, is what guarantees the relapse.

The shape of this argument is easiest to see drawn out. The three stages below are the skeleton of nearly every backfiring fix: a deliberate relief, an invited rebound, and a return that the delay disguised until it was too late to claim victory.

Notice what the diagram does not say. It does not say the fix was foolish, or that the people who chose it were careless. Each step is individually reasonable. The failure lives in the connection between the steps — in the loop — which no single decision-maker is looking at. That is the precise sense in which the problem is structural.

It is worth distinguishing two flavors of this trap, because they have different cures. In the balancing-loop flavor, the system simply restores its old strain (more road invites more driving). In the perverse-incentive flavor, the fix rewards the wrong action so directly that behavior bends toward producing the problem on purpose. The folklore name for the second is the cobra effect, and it deserves a careful footnote.

The story, popularized by the economist Horst Siebert in 2001, holds that British administrators in colonial Delhi offered a bounty for dead cobras, whereupon enterprising residents bred cobras to collect it, and released them when the bounty ended — leaving more snakes than before. It is a near-perfect illustration of a perverse incentive. It is also, by the honest assessment of historians who have searched the colonial record, very likely apocryphal: no contemporary documentation of cobra-breeding for bounty has been found. We keep the example for its clarity while stating plainly that it functions as a parable, not a verified event — a distinction the careless retelling usually omits.

06 The Real-World Anchor

The same loop, in five different clothes

Once the mechanism is clear, it becomes visible everywhere. The surface details differ; the structure repeats. Consider five domains the reader meets daily.

Traffic

The worked example above is the canonical case. Decades of evidence across the United States, Europe, and Japan converge on the fundamental law: widening congested urban highways reliably induces enough new travel to restore the congestion. The road is not wasted, but the stated goal — less traffic — is mostly not met.

Housing affordability

Forrester’s own urban models warned that subsidizing housing in a struggling city could draw in more low-income residents than the local economy could employ, deepening the very distress the subsidy meant to ease. The modern debate is more nuanced, but the structural caution survives: a subsidy that lowers one cost without addressing the loop that sets prices can shift the problem rather than dissolve it.

Health care costs

When insurance lowers the price a patient pays at the point of care, it can raise the quantity of care demanded — a balancing loop that pushes total spending back up even as it improves access. This is not an argument against insurance; it is an explanation for why cost control is so stubborn that lowering one price tends to raise total volume.

Road safety

In 1975 the economist Sam Peltzman argued that mandatory car-safety equipment could be partly offset by drivers behaving more recklessly once they felt safer — a phenomenon now called risk compensation, or the Peltzman effect. The evidence is genuinely mixed: seat belts clearly save the lives of those wearing them, and later studies find the offset is small for occupants though possibly real for pedestrians and cyclists. The honest statement is that a safety gain can be partially — not wholly — eaten by changed behavior. The loop is present even when it does not dominate.

Social media moderation and incentives

Platforms that reward engagement create a reinforcing loop: content that provokes spreads, which trains creators to provoke more, which a moderation rule aimed at the symptom rarely reaches. This is a structural cousin of the perverse incentive — the metric being optimized is not the outcome anyone actually wants, so the system optimizes its way toward a worse one.

Five domains, one skeleton. In each, a reasonable intervention meets a feedback loop and a delay, and the loop has the last word. Dimensional reasoning even helps here: if your fix changes a cost (dollars per trip, price per visit, risk per mile) but the problem is a quantity (trips, visits, miles), you should suspect that lowering the cost will raise the quantity — and check whether the product has really fallen at all.