The Division of Judgment

AI agents could give strangers thick coordination where prices were thin

Gerrit Berckheyde, The Courtyard of the Stock Exchange in Amsterdam (1670s)

How do strangers with no shared plan learn to live and work together? Under the redwood trees at Edge Esmeralda, five hundred people are putting that question to the test. Each has an AI agent of their own that proposes meetups, finds matches, and strikes small bargains for mutual benefit.

Two hundred and fifty years ago, the same question was forced on Adam Smith, David Hume, and Adam Ferguson by the world outside their window. Scotland had been thrown open to global trade during a golden age of commerce. Glasgow merchants grew rich on the Atlantic while Edinburgh courts became crowded with cases for which the law had no categories.

During this great moment of upheaval, commerce was outrunning institutions built for a feudal world. No one had designed this new world and no one was obviously in charge of it, yet it worked.

As that world crept into view, the Scots grappled with the question of how such an order not only makes us prosperous, but keeps us free. Today, as our generation finds its institutions outgrown by a new method of coordinating, we must ask the question again and formulate our own answer.

The gardener’s method

The Scottish answer began with elemental things such as prices, promises, and habits and asked how they gave rise to vast social orders. They found that institutions grow the way a garden grows, with care and patience and without absolute control over what grows and what withers.

You cannot exact a complete plan over a garden the way you would a house. Likewise, you cannot let it tend itself and hope for the best. Pretending otherwise is how good things go to seed. Gardeners neither command the plants to grow nor leave the ground to fortune.

They prepare the soil and plant with intention, just like the greatest thinkers of the Scottish Enlightenment. These men studied how social orders emerge and sustain themselves, and asked which arrangements would let free people cooperate without coercion. Their method was a form of institutional discovery.

Coordination without command

Smith showed how order could arise from everyday exchange under general rules, through a thousand mutual adjustments made in parallel, with no one commanding the whole.

The coat of a day laborer was made up of the work of the shepherd, the dyer, the spinner, the weaver, the sailor, the miner, and more still. These people never met the man who would wear the coat, and most did not know their work would become a coat at all.

When a frost strikes Brazil, the price of coffee might climb in the cafes of Edinburgh high street. Perhaps a buyer who is none the wiser about the cold snap in South America will order tea instead. The weather in Brazil changed the choice of someone on the other side of the world. And this happened freely, without the need for any minister to set the price of coffee or board to reassign the buyer to tea.

The price system provided commercial society with one of the great enablers of the liberal project: coordination without command. But Smith knew how fragile it was. The same commerce that could make a person freer and happier could also, he feared, narrow her.

For two and a half centuries, this kind of coordination survived efforts to capture or replace it, whether through monopoly, empire, planners, or the administrative state. But where those assaults came from without, today’s markets face a challenge from powerful systems that exist within the architecture of exchange.

The narrow gate

Imagine that when the frost strikes in Brazil, our grower writes a note instead of posting a price. It reads, “The harvest came up short. I’d rather it went to someone who appreciates it, so my regulars shall get first refusal. I trust some buyers more than others, so I’d gladly take less from them.”

That note tells you far more than a number does, but it is also not easy to coordinate ten strangers reading ten such notes. The genius of the price signal is that, while thin, it is both a message (coffee is scarce) and a commitment (it will change hands on these terms). While a paragraph has more explanatory power, a price can clear a market. That is why, for centuries, coordination among strangers had to pass through this narrow gate. Factors like trust or the deeper account of a purchaser’s motivations were crushed into a single scalar.

Three barriers kept coordination thin:

Barrier one was transaction costs. A richer bargain was always possible, but acting on it was expensive. Every added dimension costs more to state and make use of. Such richness was the preserve of the large firm, the repeat player, the old friend, or whoever could afford the lawyers and the time. Because this kind of coordination was expensive, societies reflexively engaged the planner to solve the problem from the top.

Today, an AI agent can understand the entire note and sound out a hundred sellers at the same time. It can assess elements such as timing or the context of the relationship and make a committed offer on behalf of its principal. The richer negotiation that was once the exclusively enjoyed by the large firm is now within reach of the ordinary citizen.

Barrier two was discovery. Picture the whole of social life as a vast chessboard where every person is a piece that moves for reasons of their own. A price tells us where the gains might be made, coaxing pieces to start to slide toward the opportunities.

But how much of the board are we seeing, individually and collectively? It is a practical impossibility for any one of us to canvas the whole of the board given our limited field of view (there are only so many hours in a lifetime). Eight billion lifetimes is a large parallel search, but soon, AI agents will range much farther.

Imagine the collaborator from a different field in a different country who could change the course of your work whom you might otherwise never meet. Or perhaps you have been stuck on a problem for years without knowing where the perfect potential colleague lives. In the past, these connections may never have been made, but today, thanks to powerful agents, these kinds of coordination problems may prove soluble.

Barrier three was purpose. Prices let you pursue your purposes without ever forcing you to articulate them in full. You might buy a book to settle an argument or to retrain for a new career. But the merchant doesn’t need to know that. The only person who does know that is you.

Your agent may help you find common cause with others whose purpose aligns with your own, or to help you better express your own. We might imagine a book buyer who is in the middle of retraining for a new profession. The price tells him about what a book costs, while the agent knows what the book is for. Then it may find him a course or a mentor or even a first client, not to mention the others who are taking on the same kind of journey.

Coordination is not the same as liberty

This is a positive future for coordination, but it is not one that we are guaranteed to see. The same AI agents that strike the bargains or find the possibilities may just as easily bring about more worrisome effects. They may be used to organize human life on a scale beyond the wildest dream of the Soviet planner, or shape our choices so thoroughly that we become less capable of authoring our own purposes.

Just like the Scottish enlightenment thinkers, we need to know how we might become more prosperous and how we might remain free agents.

Coordination is not inherently good. After all, the mercantile system Smith spent his life attacking was an elaborate scheme of coordination run by ministers and chartered companies. Whatever else was wrong with it, few could complain that its chief defect was disorder.

Consider the chess board again. This was of course Smith’s metaphor, one that he introduced as a warning. He wrote of the “man of system,” who looks at a society and imagines he can arrange its members as a hand arranges the pieces on a board. In doing so, this person forgets that each piece “has a principle of motion of its own.”

Smith’s man of system moved the pieces against their will, but the agent economy - our system of systems - need not do so. It might have the effect of whittling the pieces down instead, until moving is unnecessary.

Coordination can be so wholly attentive or endlessly accommodating that we cease to perform the labor of free people who live within it. The price system asked little of us, but it still left us to decide what we wanted and to act according to those wants. It coordinated our means and it presupposed our ends rather than supplying them.

AI can frame which goals are worth pursuing for each of us, then hand us answers in such a way that we are never called on to form them ourselves.

Think of a parent deciding what to teach her child. She asks the agent and the agent provides a considered response in short order. If the parent takes it once, perhaps nothing of consequence is lost. But if she takes it every time, each week, each day, or each hour, she may find that she no longer has a view of her own in a meaningful sense.

And why should that trouble us, if the agent’s answer is better than the one we would have reached? Because a good life is not only a life that goes well. It is, and must remain, a life that each of us leads.

Smith saw this danger too, at the root of the system he praised. He warned that the man who was kept to a few simple operations, “has no occasion to exert his understanding” and grows “as stupid and ignorant as it is possible for a human creature to become.” This was his fear for the laboring hand, narrowed by the division of work.

We now risk it for the mind, narrowed by the division of judgment.

What takes the place of blindness

The Scots did not merely think the modern world into being. Instead, they built it through the institutions of trade and the habits of conversation. Gardeners as they were, they neither dictated the order nor waited for it to spring up fully formed on its own without assistance.

They helped find the rules that made price-based coordination blossom under liberty. These were general rules that were blind to persons and indifferent to ends.

The law could be trusted because it knew nothing of you, but the AI agent is powerful because it knows you well. What are the general rules for coordination using systems that know us? Answering this question is why five hundred agents are at work in California while their principals sleep. This is because we cannot think our new world into being, no matter how clever our ideas or how noble our intentions.

We must tend or we lose the garden, and with it the capacities that make life meaningful. We must revitalize the conditions under which free people can govern themselves, and we must do so again and again and again if we are to keep the liberal dream alive.

A tradition lives by being used.

Adapted from a keynote delivered by Brendan at the New Enlightenment Conference in Edinburgh, marking the 250th anniversary of Adam Smith’s The Wealth of Nations at Panmure House. Thanks to Ivan Vendrov and Séb Krier for their conversations and insights.

---

Cosmos Institute is the Academy for Philosopher-Builders, technologists building AI for human flourishing. We run fellowships, fund AI prototypes, and host seminars with institutions like Oxford, Aspen Institute, and Liberty Fund.