The Anatomist's Fallacy

When dealing with complex systems, think about effects, not intentions

October 29, 2024

I tend to be shy about using the words “purpose” and “intention”. There are lots of processes that make it tempting to ascribe intention, but which have none. I think the easiest example is evolution. “Wide-spread eyes evolved in prey animals in order to detect stalking predators,” is an easy thought to have, but evolution as a process doesn’t plan; nothing is “in order to”. Everything is “continues because”. A more awkward but precise statement is “wider-spread eyes persist in prey animals in part because they make animals more likely to detect stalking predators and survive to reproduce, but they may provide other advantages, or just provide no disadvantages.”

I call this “the anatomists fallacy” — ascribing purpose to something simply because it’s effective at something, even when it’s the result of a system that might not be capable of intention at all; even when it has dozens of effects, not just the one.

The corrected statement is a mouthful, though, and I can understand why we hear the former more often. When you’re not just observing, though, when you’re trying to manipulate a complex system, it’s important to think and speak very clearly about how it works.

And natural processes, like evolution, are not the only ones where this applies.

In my eyes, there is a spectrum of intention-having ability. Natural processes obviously have none. A sunset isn’t for anything. On the other hand, when I am alone in a room, writing this paragraph, I have a fairly well-defined set of objectives, things I want to communicate, results I want to obtain by writing. If we couldn’t describe the conscious and planned actions of a single person as having purpose and intention, then the words wouldn’t have any use at all.

But even human systems, once enough humans are involved, start to feel similar: the output of big complex systems like corporations, governments, and economies don’t feel like they can be ascribed a single purpose to me. They are the result of hundreds or thousands or millions of people, all with different purposes, acting on incentives and whims, negotiating and working together or against each other. The effects may be better described as just consequences, leaving off the leading adjectives of “intended” or “unintended”.

And between a single human and an entire society are smaller groups – work teams, local organizations, families, friend groups – these are entities one can draw a reasonably clear line around, and which take actions and have effects. They are in some blurry area; if everyone involved is of one mind, and has communicated clearly, they could easily be acting to one purposeful end. But it’s just as possible that everyone is acting independently, and the resulting group effects are, happy or unhappy, just effects.

So here is the practical exercise at the end of this philosophical maundering: as you go about your life this week, think about each part of it the way a careful biologist might think about a new anatomical structure — not in terms of what it’s supposed to do, but all the things it actually does.

Don’t ask “What is this meeting for?” — observe, and see what its effect actually is. Where does information flow? Who feels what, who learns what? What’s the difference after the meeting vs before?

Don’t ask “Is this initiative succeeding?” — Observe what it’s doing, and everything it’s doing.

Outside of work, consider family dinner, lunch with friends, a traffic jam, a crowded sports bar.

Then, when you see negative effects, you can think about interventions not in terms of changing people’s minds to want different things, but in terms of nudging the entire system to produce different effects without touching the intentions involved at all.

My books are open! I am currently taking contracts for winter 2024 and spring 2025. Drop me a note.