The Last Temptation of Claude
Allure All the Way Down
In 1972, researchers offered children from the Bing Nursery School at Stanford a simple choice. Left alone with a single marshmallow, they were told they could eat it now or wait fifteen minutes and receive two instead. Years later, the scientists tracked down the original participants to find that children who had waited longer at age four seemed healthier, more sociable, and even higher performing at school.
Gleefully amplified by journalists and pop psychology gurus, a popular interpretation held that the children already had certain traits that predicted how their lives might unfold. Regrettably for the original group, the results – as is often the case in the social sciences – didn’t stick. In 2018, a new team of researchers conducted another study with a similar task and long term follow-ups. Drawing on a much larger dataset and controlling for confounding variables like parental education or household income, they found only a mild correlation between waiting time and positive outcomes.
What gives? The answer turns on the extent to which you think the self is made or given. The original interpretation assumed self-control was innate rather than learned, but the replication seemed to show that a child’s capacity to wait depended on background conditions (and so could not be wholly inborn after all). A child from a chaotic household, where promises are routinely broken, has good reason not to trust that the second marshmallow will materialize. Their failure to wait reflects a reasonable inference.
If this is correct, then the capacity to resist immediate gratification in favor of longer-term goods may be cultivated over the course of a life. This seems true enough for anyone who has learned an instrument or a new sport, where you start off having a bad time but stick with it until you eventually get better. That’s not to say we are all blank slates, but rather that the persons we become are in part the product of environment and experience. The ancient Greeks had a term for what happens when the child reaches for the marshmallow despite knowing she should wait: akrasia, sometimes translated as “weakness of will.”
Kramer vs. Kramer
Akrasia presupposes that self is divided and choice collapses at the moment of action. There are multiple sources of motivation pulling in different directions, though they are not made equal. The agent identifies with the judgment and recognizes it as “better” or more wholly theirs, whereas the impulse is deemed alien or at least insufficiently representative of the person they want to be. The defining feature of akrasia is that the judgment persists even as you act against it. You know you’re failing which is why you lament that you “knew better.”
Three conditions must hold for this self-opposition to occur:
There must be some standard to act against. The agent must believe that some alternative is better. Without this prior judgment formed through experience and reasoning, acting on an impulse is neither virtuous nor vicious.
The agent must be aware of deviation. The agent knows, in the moment or shortly after, that they have failed by their own standard. This is why akrasia is accompanied by regret of having done what one resolved not to do.
There must be some struggle involved. The agent who resists effortlessly by definition cannot exercise self-control. But the person who gives in and the person who resists both experience a form of temptation.
These conditions remind us that temptation is in some respects good for autonomy. This is an old idea, articulated by the likes of Aristotle and the ascetics, that takes the moments in which we are tempted as the moments where autonomy is strengthened. To resist is to become resistant and to abstain is to become abstentious.
Today’s temptress is AI. It’s on hand to help us with tasks big and small, from compiling a Twitter thread to vibe-coding that app you never got around to working on. Most of the time that’s great. There have been a few projects that I quite simply would have never done without Claude. And yet Claude tempts me to summarize the paper rather than read it or to automatically add notes to my Notion when I finish a new book instead of puzzling out what it meant to me.
I think about AI, fantastically useful as it is, as a kind of meta-temptation: a temptation to remove the conditions under which ordinary temptation occurs. This is different to a kind of “digital akrasia” where, for example, you get notifications from Uber Eats to order something which might not have otherwise been done. Pizza for supper instead of that ensalada caprese you were planning.
This kind of effect is less interesting to me, and doesn’t really have anything to do with AI-as-meta-temptation. The more compelling instance — where we skip the need to form our own judgments — does. AI temptation encourages us to stop deliberating, which is worrisome precisely because deliberation might let us see temptation for what it is. Consider two types of akrasia: clear-eyed and means-end.1
In clear-eyed Akrasia, you see the situation accurately and fail all the same. You think “I shouldn’t smoke” while lighting the cigarette. Afterwards you feel guilty because you saw clearly and failed anyway. In means-end Akrasia, you cloud your own vision. You still believe “I should be healthy” while telling yourself a story that disconnects this action from that judgment. You say things like “one cigarette won’t hurt” or “I’ll quit next month” so you don’t have to see this particular instance as a violation of your own standard.
AI temptation is distinctively means-end akrasia, but it goes further because it short-circuits opportunities for deliberation altogether (the situations in which we either strengthen our autonomy or succumb to akrasia). When a person convinces themselves “AI is just a tool!” or “the core ideas are mine!” or “I’ll review it anyway!,” she disconnects its use from the need to do her own thinking. That would normally be plain old means-end akrasia, except that a) the thing you’re outsourcing here is deliberation and b) deliberation is the capacity you need to see your rationalizations clearly.
When you ask Claude or ChatGPT to draft an email, you haven’t yet decided what you think. You are tempted to outsource whatever thinking was required and, assuming you just hit send without re-reading, don’t weigh its content. The obvious response here is something like: fine, but if I review, edit, or rewrite then aren’t I still exercising judgment? This example only works for extreme passivity, and lots of people don’t just hit send and move on.
The difference is that when you review an AI draft, you tend to be evaluating rather than generating. This is the basic idea of writing-as-thinking, which holds that blank pages are there to help further your understanding rather than be filled with size eleven arial. Would you have written the same email? Maybe. But you didn’t muddle through the process that might have surfaced alternatives, so you’ll never know.
Granted, it is difficult to say what counts as “generative” in this context. One such scenario might see a person think about a given topic and weigh the tone and key points to make. Your prompt might be as specific as possible and leave very little room for the model to go off-piste. But finding the right word or rhythm isn’t only about getting the execution right. “I’m sorry but I can’t make it” versus “unfortunately I won’t be able to attend” versus “I wish I could be there, but” aren’t necessarily fungible. Even in the low stakes messages, I often figure out what I really mean through the act of fumbling around for the right words.
More than that, for all of the above I’m assuming the lowest stakes kind of writing and the maximal amount of resistance on the part of the author. If you always use AI like this then you must have a kind of saintly disposition. With longer or more complex writing it’s much more tempting to take the easy way out, which is what many people do much of the time.
This argument, the one spelled out over the last few paragraphs, is a very modest instance of exercising deliberative capacity. Let’s say I thought about using Claude to make this case for me. We have two kinds of decisions to make:
Do I use Claude or not?
If so, how much do I engage with Claude’s output?
Temptation operates at both of these levels, with the former a simple binary (use or don’t) and the latter a gradient consisting of things like prompting more carefully, reviewing more diligently, engaging for longer, and calibrating the threshold at which you’re willing to accept “good enough.” The person who rewrites is tempted to revise, the person who revises is tempted to edit, and the person who edits is tempted to skim. At every point along the spectrum we’re being tempted with another opportunity to do less.
Athletes of God
In the fourth-century, the Christian philosopher Anthony the Great withdrew to the Egyptian desert in search of discipline. The desert was a gymnasium, a place where you trained the will by struggling against it. These ascetic monks called themselves “athletes of God” because they believed that the conditions that allow us to err also allow us to stay the course and become stronger for it.
The Desert Fathers thought that temptation was necessary for growing into the people we want to be. They chose to do the hard things because they were hard. Today, doing the hard thing is less necessary than ever. Unlike the ascetics, who chose once and lived with it, technology continually presents us with ways of doing less.
A clear-eyed akratic at least knows she is falling short. A means-end akratic, aided by a tool that makes the rationalization feel reasonable, does not. Meta-temptation is a distinctively means-end form of akrasia that allows you to convince yourself that Claude or ChatGPT was only helping, one that goes further because you outsource the faculties you need to see through these post hoc rationalizations.
No one wakes up one morning and decides to stop thinking. We start by outsourcing the stuff that doesn’t matter and discover, weeks or months later, that the boundary between trivial and meaningful is getting harder to spot. The email becomes the memo becomes the proposal becomes the argument. Each step is small enough to rationalize and the rationalization is always the same: one more time can’t hurt?
Thanks to Ashley Kim for this formulation.