In June, we gathered thinkers, builders, and investors for a weekend of lectures and salons at the Normandy château where Alexis de Tocqueville wrote Democracy in America.
We offered places to the two winners of our Tocqueville and Technology essay contest. There were over 250 submissions and we were impressed by their range and depth of engagement with the questions.
Both our winners opted for question one:
Tocqueville warned of a “tutelary power” that would keep citizens in perpetual childhood. How have Tocqueville’s concerns migrated from institutions to algorithms, and does AI fulfill or transform this fear?
Thank you to everyone who participated and we look forward to running more of these competitions in the future! To get a flavor of the weekend’s discussions, you can read Brendan’s lecture on how AI fuses Tocqueville’s two fears: power over what we do and power over what we think.
Essay 1: Chris Pang
The limits of tutelary powers are determined by two factors: the precision of the tutelage and the efficiency of the process. Personal tutors are known to leave long legacies on their students but are in short supply and require years to transfer their knowledge. Standardized tests and lessons are easily scaled but poorly tailored. Algorithms and adaptive AI tutors promise to bridge this divide, providing deeply personalized instruction at mass scale.
While this package is sold with visions of giving every student their own on-demand Aristotle, it is also a power that the dystopias and dictators of the twentieth century could not have dreamed of. Even Huxley’s Brave New World, where all humans are effectively genetically engineered and conditioned from birth, uses broad caste systems and mass curricula instead of one-to-one monitoring to instill a citizen’s “ideal behavior” – thus enabling dissidents like Bernard Marx to exist.
Popular media and culture in authoritarian regimes are attempts to discipline society into compliance, something that is no longer necessary when every citizen can be offered their own customized cult leader.
Unfortunately, whether a tutelary power is seen as necessary or as despotic varies with the writer and the power. Compulsory and free universal education is a target for the UN Declaration of Human Rights, but what the government sees as important civic values can often appear as instilled ideology and obedience to an outsider.
Plato saw nothing wrong with philosopher-kings censoring art and keeping classes of citizens as slaves and warriors. In his essay “The Abolition of Man,” C.S. Lewis paled at the idea of “men without chests” engineering the values of a future (trans)humanity based on nothing but gut instinct. He accuses them of rejecting the universal Tao of moral discourse in favor of a powerful but brutish rationalism that can always answer “how” but never “why.” It is less clear whether he would have rejected the idea of philosopher-kings well versed in Confucius, Aristotle, and St. Augustine wielding similar powers.
Ultimately, AI educators and technologists are on their way, and the demand is high due to rising uncertainty about human adaptability in a fast-moving world. So we must ask: can we create AI educators and algorithms that do not subjugate their students to some set of predetermined values?
In other words, can we design a curriculum that increases a student’s powers of deontological and epistemic judgment instead of withering them? I venture that such a question is both realist (as in pertaining to questions of moral realism) and also scientific (relating to observable knowledge about the world). Dependency, fragility, and indoctrination have real and observable effects on people’s lives, just as we can observe how good mental health and judgment improve the lives of people around us.
To write such a curriculum would be beyond the scope of this essay, but educators and technologists must be careful to prevent their well-meaning attempts from recreating the nightmares of the past, but now with sufficient intensity that no escape is possible.
Chris is an independent AI researcher, with interests in AI safety and interpretability, theories of learning, and the institutional incentives for developing AI. His past work includes the Human Inductive Bias Project, a project backed by the UK Advanced Research & Invention Agency, focused on the connection between human and machine learning. You can read more about his work and interests on Github.
Essay 2: “@A.E. Snow A.E. Snow”
Tocqueville feared a future in which citizens, isolated by the universal acid of equality, would become dependents of the state. Without the vertical ties of obligation that had once bound individuals to lord, guild and parish, each in their little circle of concern would find themselves less accustomed to acting with others, and more willing to surrender the burdens of common life to a central power. Gradually, life would be mediated by an “immense and tutelary power”: a state that secured their pleasures and watched over their fate, minute in its attention and mild in its manner.
Two centuries on, the warning is prescient. AI intensifies Tocqueville’s fear by relieving citizens of the burdens of judgment. Yet the further this process goes on, the more transformational it will be to the relationship Tocqueville described. A tutelary state still needs its subjects to work, vote, administer, and fight. Its potential to diminish agency is tempered by its need for fully formed citizens. If AI makes more of these functions performable without the active capacities of ordinary citizens, infantilization gives way to superfluity.
When neither the market nor the state retains an instrumental need to form large numbers of capable citizens, it creates a vacuum.
AI could enable the centralization of power as part of a custodial order. At the same time, by loosening the state’s grip on the formation of people, it could reopen the dense world of voluntary associations and intermediary institutions that Tocqueville admired in America.
A state needs citizens for administration, labor and legitimacy, while a market needs workers and consumers. By contrast, a craft tradition, parish or chess club needs its members because they are constitutive of the activity. After all, a chess club without players is not a more efficient chess club, nor can a monastic order function without novices. These are ludic institutions that you enter voluntarily, but which bind their members. Chess is not commensurable with poetry, just as martial arts isn’t with monastic contemplation. Someone deeply formed by one is not necessarily better or worse than someone formed by another.
In the future, the state’s role could change from forming citizens directly to protecting the conditions under which others can.
First, this means guaranteeing plurality. If any single formative order absorbs the rest, it recreates the tutelary power.
It also means protecting opacity. AI gives the state new powers to render practices legible and assimilable, but binding institutions often depend on boundaries and friction. In this world, the state would be neither minimal nor paternal, but the groundskeeper of plural games that it does not itself play.
AI has the potential to fulfill Tocqueville’s fear when it makes every game administrable by a single logic. However, it transforms that fear when it sustains plural associations in which people become capable of self-rule.
Our second winner has chosen to remain anonymous, but you can read more of his work on Substack.
Cosmos Institute is the Academy for Philosopher-Builders, technologists building AI for human flourishing. We run fellowships, fund AI prototypes, and host seminars with institutions like Oxford, Aspen Institute, and Liberty Fund.
Very fine essays!
"Without the vertical ties of obligation that had once bound individuals to lord, guild and parish, each in their little circle of concern would find themselves less accustomed to acting with others, and more willing to surrender the burdens of common life to a central power." This beautifully dovetails with the interview by Mary Harrington (Socrates in the City) of theologian, etc. Carl Trueman. They discuss this very point with Mary bringing up the Enclosure Act, brought on by agriculture becoming industrialized in England, unmooring the people from the land that had been their identity. Will AI simply untether ourselves further from one another creating further small parcels like the Irish were given under British rule that led to famine when mold destroyed potatoes - a crop that grew well on small parcels? The first essay hits on this with these great sentences: While this package is sold with visions of giving every student their own on-demand Aristotle, it is also a power that the dystopias and dictators of the twentieth century could not have dreamed of. Even Huxley’s Brave New World, where all humans are effectively genetically engineered and conditioned from birth, uses broad caste systems and mass curricula instead of one-to-one monitoring to instill a citizen’s “ideal behavior” – thus enabling dissidents like Bernard Marx to exist. Popular media and culture in authoritarian regimes are attempts to discipline society into compliance, something that is no longer necessary when every citizen can be offered their own customized cult leader.
I'm a fan of René Girard's work on scapegoating, as it helps me navigate current events, limiting my rancor at being the scapegoated. René tethers scapegoating to technological advances. Now that AI is being used in labs to 'nudge' behavior, will it try to nudge us out of scapegoating? Or if AI is designed to progress society, will it encourage scapegoating to facilitate new technology? If it is the former, will we be rendered a noncommittal shade of beige?