Reading List: AI and the Future of Human Autonomy
An open-source reading list from a discussion group hosted by the Liberty Fund
In partnership with the Liberty Fund, we recently hosted a seminar examining a crucial question: How can we build AI systems that enhance rather than diminish human autonomy?
Our interdisciplinary group brought together researchers from DeepMind, philosophers from Oxford, Topos Institute, and Meaning Alignment Institute, cognitive scientists, learning science researchers from Stanford, and founders of AI startups. We explored principles for developing AI that expands our capacity for self-direction and meaningful choice.
The debate around AI often falls into two camps: existential pessimism and uncritical accelerationism. Neither approach adequately addresses what may be the most important question: How does AI impact human autonomy?
This concern not only relates to the possibility of human flourishing at the individual level, but also echoes Tocqueville's prescient warning about "soft despotism"—the gradual erosion of individual judgment and initiative as we increasingly outsource our decision-making to external systems. As AI becomes more sophisticated and pervasive, this risk becomes more acute.
The following collection includes foundational texts, contemporary analysis, and notable quotes from our seminar discussions. Each selection was chosen to illuminate different aspects of the autonomy challenge, from philosophical foundations to practical implementation.
Session I: Foundations of Autonomy
Aristotle, Nicomachean Ethics, Book I, 1-7, 13; and Book II, 1-6 (link)
We have found, then, that the human function is activity of the soul in accord with reason
Humboldt, The Limits of State Action, Ch. 2, “Of the Individual Man and the Highest Ends of his Existence” (link)
The true end of Man... is the highest and most harmonious development of his powers to a complete and consistent whole. Freedom is the first and indispensible condition which the possibility of such a development presupposes
Mill, On Liberty, Ch. 3, “Of Individuality, as One of the Elements of Well-Being” (link)
Such being the reasons which make it imperative that human beings should be free to form opinions, and to express their opinions without reserve; and such the baneful consequences to the intellectual, and through that to the moral nature of man, unless this liberty is either conceded, or asserted in spite of prohibition
Kant, “An Answer to the Question: What is Enlightenment?" Pgs 1-7 (link)
Nothing is required for this enlightenment, however, except freedom; and the freedom in question is the least harmful of all, namely, the freedom to use reason publicly in all matters.
Session II: Modern and Psychological Accounts of Autonomy
Sedgwick, Kant’s Groundwork of the Metaphysics of Morals: An Introduction, “Kant's Preface” and Ch. 4, Sect. II
Kant argues that rational agents are able to give themselves law in virtue of their remarkable capacity of self-determination or autonomy.
Satre, Existentialism Is a Humanism, Introduction (by Annie Cohen-Solal; quote is by Satre)
Man is nothing other than his own project. He exists only to the extent that he realizes himself, therefore he is nothing more than the sum of his actions... responsible for what he is... free... condemned to be free... commit[ing] himself to life.
Hayek, The Constitution of Liberty, Ch. 2, “The Creative Powers of a Free Civilization”
It is because freedom means the renunciation of direct control of individual efforts that a free society can make use of so much more knowledge than the mind of the wisest ruler could comprehend.
Deci and Ryan, Intrinsic Motivation and Self-Determination in Human Behavior, Ch. 1, “An Introduction” and Ch. 11, “Work” (link)
The application of our motivation theory to work organizations suggests that working toward maximal self-determination for each organizational member should be a central organizing principle.
Session III: External Threats to Autonomy
Tocqueville, Democracy in America, Ch. 2.4.6 (link)
Subjection in small affairs manifests itself every day and makes itself felt indiscriminately by all citizens. It does not drive them to despair; but it thwarts them constantly and leads them to relinquish the use of their will [and finally to give up on themselves]. It thus extinguishes their spirit little by little, and enervates their souls.
Hayek, "The Use of Knowledge in Society" (link)
Fundamentally, in a system in which the knowledge of the relevant facts is dispersed among many people, prices can act to coördinate the separate actions of different people in the same way as subjective values help the individual to coördinate the parts of his plan… The whole acts as one market, not because any of its members survey the whole field, but because their limited individual fields of vision sufficiently overlap so that through many intermediaries the relevant information is communicated to all.
Goffman, “Asylums: Essays on the Social Situation of Mental Patients and Other Inmates,” Introduction and Sections I & II of Ch 1, “The Inmate World“
[I]f the inmate's stay is long, what has been called "disculturation" may occur-that is, an "untraining" which renders him temporarily incapable of managing certain features of daily life on the outside, if and when he gets back to it.
Thaler & Sunstein, "Libertarian Paternalism Is Not an Oxymoron", Abstract and Introduction (pgs 1-8) (link)
[I]t is both possible and legitimate for private and public institutions to affect behavior while also respecting freedom of choice. Often people’s preferences are ill-formed, and their choices will inevitably be influenced by default rules, framing effects, and starting points. In these circumstances, a form of paternalism cannot be avoided. Equipped with an understanding of behavioral findings of bounded rationality and bounded self-control, libertarian paternalists should attempt to steer people’s choices in welfare-promoting directions without eliminating freedom of choice.
Session IV: Internal Threats to Autonomy
Nietzsche, Beyond Good and Evil, Ch. 1, aphorisms 1-19 (link)
"Freedom of the will" is the word for that manifold pleasurable condition of the willer who is in command and at the same time considers himself as one with the executor of the command-as such enjoying the triumph over the resistance, but possessed of the judgment that it is his will itself that is overcoming the resistance.
Freud, Civilization and Its Discontents, Ch. 7
[Man’s] aggressiveness is introjected, internalized; it is, in point of fact, sent back to where it came from-that is, it is directed towards his own ego. There it is taken over by a portion of the ego, which sets itself over against the rest of the ego as super-ego, and which now, in the form of 'conscience', is ready to put into action against the ego the same harsh aggressiveness that the ego would have liked to satisfy upon other, extraneous individuals. The tension between the harsh super-ego and the ego that is subjected to it, is called by us the sense of guilt; it expresses itself as a need for punishment.
Rieff, The Triumph of the Therapeutic: Uses of Faith after Freud, “Introductory: Toward a Theory of Culture”
[M]an, as a creation of affectionate and censorious authority, once organized for himself modes of willing obedience, or faith, in which he found his sense of well-being and, also, his freedom from that singular criterion. Culture without cultus appears, in almost all historical cases, a contradiction in terms.
Lasch, The Culture of Narcissism: American Life in An Age of Diminishing Expectations, Ch. 1
Having surrendered most of his technical skills to the corporation, [the contemporary American] can no longer provide for his material needs… The atrophy of older traditions of self-help has eroded everyday competence, in one area after another, and has made the individual dependent on the state, the corporation, and other bureaucracies.
Session V: Designing Systems for Autonomy
Sarkar, “AI Should Challenge, Not Obey” (link)
As people increasingly incorporate AI output into their work, explicit critical thinking becomes important not just for formal academic disciplines, but for all knowledge work... From tools for efficiency, toward tools for thought. As system builders, we have the opportunity to harness the potential of AI while maintaining, even enhancing, our capacity for nuanced and informed thought.
Koralus, “The Philosophic Turn for AI Agents: Replacing Centralized Digital Rhetoric with Decentralized Truth-Seeking” (link)
AI should be constructed to facilitate decentralized truth-seeking and open-ended inquiry, mirroring the Socratic method of philosophical dialogue. By promoting individual and collective adaptive learning, such AI systems would empower users to maintain control over their judgments, augmenting their agency without undermining autonomy.
Habermas, The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society, Ch. 5 “Institutions of the Public Sphere” (link)
For the first time an audience gathered to listen to music as such-a public of music lovers to which anyone who was propertied and educated was admitted. Released from its functions in the service of social representation, art became an object of free choice and of changing preference. The "taste" to which art was oriented from then on became manifest in the assessments of lay people who claimed no prerogative, since within a public everyone was entitled to judge.
Tessler, et. al. “Al can help humans find common ground in democratic deliberation.” (link)
Using participants’ personal opinions and critiques, the AI mediator iteratively generates and refines statements that express common ground among the group on social or political issues. Participants preferred AI-generated statements to those written by human mediators, rating them as more informative, clear, and unbiased. Discussants often updated their views after the deliberation, converging on a shared perspective.
Stray, et. al. “What are you optimizing for? Aligning Recommender Systems with Human Values.” (link)
As recommendation systems proliferate, it is important to consider the impact these systems have and the values that drive ranking decisions… we are excited about the potential for recommendation systems to better align with human values through the incorporation of well-being metrics, participatory approaches to objective design, interactive value learning, and optimizing for informed and deliberative preferences.
Cosmos Institute is the Academy for Philosopher-Builders, with programs, grants, events, and fellowships for those building AI for human flourishing
Love a good reading list! This week, folks will be able to read these all on Alexandria, with Virgil as their guide.
What a pity I missed the group photo! The selection of both the text and the participants was excellent.