Reading List: AI and the Ownership of Judgment
Offloading versus enfeeblement
AI is already fueling a historic expansion in human agency. It’s never been easier to increase our output or to solve harder problems. But just as AI enables us to act more, it also creates situations where we have to bear the responsibility for outcomes and judgments that we played less and less of a role in determining.
If we outsource too much or certain types of judgment, formal authorship can survive as substantive authorship weakens. And as the capacity to judge is itself shaped through use, the dependence can become cumulative. We risk eroding the capacity that would help us to recognize what is happening to us. Preserving authorship – the ability to discern and set your own ends – is the foundation of autonomy.
It’s difficult to draw clear lines: human judgment is never completely independent. It has always been formed and exercised in relation to people, institutions, tools, and inherited ways of thinking. A person can endorse a judgment that’s handed to them by someone (or something) else, just as they could arrive at their own judgment with significant external help.
This makes determining the ownership of judgment a central problem. In partnership with Liberty Fund, Cosmos held a seminar to analyze these questions with philosophers from philosophers, thinkers, builders, and founders from leading institutions.
Session I: Cognition and Ownership
The opening session questioned the common assumption that a thought is ours simply because it occurs in our head. Clark and Chalmers argue that external resources, such as a notebook, can play the same integrated role as memory or internal calculation, making them part of our cognition. Adams and Aizawa are skeptical, claiming that there is a distinction between derived content (e.g., words in a notebook that have meaning because of social convention) and non-derived content (e.g., remembered belief), with real cognition distinguished by the latter.
Polanyi argues that something external can become incorporated in skilled thought, irrespective of whether it literally becomes part of our cognition. If we use a tool enough, it can become part of the practical structure we use to view the world.
But even something thoroughly integrated into a person’s psychology may undermine their autonomy if it arrived there through manipulation and conditioning. For example, someone may have been indoctrinated into a cult. Christman therefore argues that ownership doesn’t just depend on present integration or endorsement, but also on the history of how a judgment was formed.
Andy Clark and David Chalmers, “The Extended Mind.” Analysis. (link)
Fred Adams and Ken Aizawa, “The Bounds of Cognition.” Philosophical Psychology. (link)
Michael Polanyi, Personal Knowledge: Towards a Post-Critical Philosophy, “Two kinds of awareness.”
John Christman, “Autonomy and Personal History.” Canadian Journal of Philosophy. (link)
Session II: Answerability and Question-Setting
If our judgments are shaped by reasons or environments that we did not create ourselves, then what makes those judgments ours?
One tempting answer is that we are answerable for what we consciously choose to do. But Watson argues that just because an action we don’t like accurately reflects a person’s character or values, it doesn’t always mean we have the standing to hold them accountable for it. Likewise, Scanlon makes the case that we can own our beliefs and intentions even if they arise without conscious change. If they are responsive to reason and debate, then they belong to us. Arpaly raises the case of how Huckleberry Finn comes to see that Jim is a person, even as his explicit moral commitments tell him that helping him is wrong.
In essence, the question is less about who produced a judgment and more about how that judgment is shaped. This is at the heart of Koralus’s notion of erotetic (questioning) equilibrium, which tries to distinguish helpful prompting by an AI system from manipulation. A judgment is in equilibrium if we are able to defend it, in terms we are happy with, across an appropriate range of questions.
This leads us into the thorny territory of what constitutes good guidance and if such a thing is even possible. Kant’s defense of the Enlightenment is hostile to the idea we can allow people to perform the work of understanding for them, while Rousseau proposes that a student should be allowed to feel free while the tutor subtly controls the environment that determines what they want. Either way, McDowell warns that we can’t escape these influences, because even the most radical reflection begins with our upbringing and inherited ways of thinking.
Gary Watson, Agency and Answerability: Selected Essays. Chapter Nine.
TM Scanlon, What We Owe to Each Other. “Judgment-Sensitive Attitudes.”
Nomy Arpaly, Unprincipled Virtue: An Inquiry into Moral Agency. “Moral Responsiveness De Re”.
Philipp Koralus, “The Philosophic Turn for AI Agents.” (link)
Immanuel Kant, “An Answer to the Question: What is Enlightenment?” (link)
Jean-Jacques Rousseau, Emile; or, Concerning Education: Extracts. Edited by Jules Steeg. Pages 2-3. (link)
John McDowell, Mind and World. Lecture IV: Reason and Nature.
Session III: Tools, Institutions and Formation
This session moved from how we make decisions to how the decisions shape the deciders. If Aristotle is right that our capabilities and character are constituted through our actions, we have to ask if technology is helping us to offload productively or if it’s enfeebling us.
Whitehead makes the optimistic case that civilization advances by mechanizing routine operations, allowing us to redirect our scarce attention towards harder problems.
But when we leave Whitehead’s example of mathematical symbols behind, it gets trickier. Carr and Borgmann warn us about how too much offloading can undermine our capacity to maintain human control. Borgmann focuses on alienation of labor that flows from the automation of manual work, while Carr gives the example of how autopilot improves safety in aggregate, but at the cost of degrading pilots’ performance in emergencies.
Tocqueville applies the same principle to politics, fearing that if the state became overly helpful, citizens would gradually forget how to govern themselves. And these relationships may extend beyond traditional politics. Anderson argues that inside the workplace, employers do not just distribute goods or services, but enjoy so much authority over those subject to them that they constitute a form of “private government.”
Aristotle, The Nicomachean Ethics. Book II, Chapter 1. (link)
Albert Borgmann, Technology and the Character of Contemporary Life: A Philosophical Inquiry. “The Device Paradigm.”
Alexis de Tocqueville, Democracy in America, Volume 2, Chapter 6. (link)
AN Whitehead, An Introduction to Mathematics. Chapter 5.
Nicholas Carr, The Glass Cage: How Our Computers Are Changing Us. “On Autopilot.”
Elizabeth Anderson, Private Government. “Communist Dictatorships in Our Midst.”
Session IV: Developmental Telos
If we accept that tools and institutions are formative, it becomes hard to view them as neutral. Instead, the readings examine what this formation should (and shouldn’t) try to achieve.
Oakeshott launches a broadside against the social sciences. He accuses them of treating human conduct as the product of general social processes, rather than human intentions. Instead, he argues that real liberal education grows the number of languages we can use to understand the world, whether it’s the historical, scientific, philosophical, or poetic. In a similar way, Humboldt grounds his case for freedom in the importance of variety and for forging human capacities. Arendt believes that any education system needs to help the next generation understand that it is living in an inherited world, but should stop short of telling them how they should live in it.
Vallor warns that any discussion of formation needs a credible account of technology. This is because technologies aren’t neutral instruments that serve any philosophy, but instead structure habits and practices. In turn, these shape what is possible and what we aspire to.
Michael Oakeshott, The Voice of Liberal Learning, Liberty Fund edition, pages 22-29.
Hannah Arendt, Between Past and Future. “The Crisis in Education” § 3-4.
Wilhelm von Humboldt, The Limits of State Action. Chapter 2. (link)
Shannon Vallor, Technology and the Virtues. Introduction.
Session V: Liberty and Working-Back
The final session examines how problems surrounding cognition and formation should impact our view of power and authority.
Berlin’s critical analysis of “positive liberty” outlines the danger of any account that claims it can liberate people on behalf of their more rational or “higher” selves. Once a single agent believes it can identify the “true” self, it can claim potentially sweeping powers to unlock it. By contrast, Pettit offers a more limited alternative that does not require freedom from all influences. Instead, his account of autonomy requires protection against arbitrary power, including manipulation.
This shifts the emphasis towards contestability. Hirschman criticizes those who simply argue that competition improves bad institutions. He distinguishes between exit (responding to dissatisfaction by leaving an institution) and voice (trying to change how it operates). Sometimes exit can be costly or impractical, so we need the ability to contest decisions, without having to abandon institutions altogether.
Gadamer and MacIntyre argue that judgment necessarily begins within inherited traditions, practices and forms of authority. At the same time, these inheritances do not simply determine us. MacIntyre in particular presents a living tradition as one sustained through argument, criticism, and revision.
Isaiah Berlin, Four Essays on Liberty. “Two Concepts of Liberty” § 2-3.
Philip Pettit, Republicanism. Chapter 2.
Albert Hirschman, Exit, Voice, and Loyalty. “Exit and Voice as Impersonations of Economics and Politics.”
Alasdair MacIntyre, After Virtue, A Study in Moral Thought. Chapter 15.
Hans-George Gadamer, Truth and Method. “The Rehabilitation of Authority and Tradition.”
Cosmos Institute is the Academy for Philosopher-Builders, technologists building AI for human flourishing. We run fellowships, fund AI prototypes, and host seminars with institutions like Oxford, Aspen Institute, and Liberty Fund.
This is a rich map of the philosophical terrain. What I see in enterprise AI rollouts is this exact question showing up operationally, not just cognitively — organizations are making decision-rights calls (what gets delegated, what stays human) without a framework for it, and the ones getting it wrong aren't seeing it as a judgment problem yet. They're seeing it as a productivity metric that isn't moving. I've been mapping this on the practitioner side — happy to share if useful alongside the philosophical grounding here.
The history condition seems crucial for education. A learner can sincerely endorse an AI-shaped judgment and still be unable to reconstruct how it became theirs.
Perhaps ownership needs a practical test: can the person answer unfamiliar questions, identify what would change their mind, and revise without the system supplying the next move? That would connect answerability to formation, not just present agreement.