Excellent piece. You've provided a clear and much-needed philosophical language for the concept of "AI Deference." The distinction between using AI as a tool for augmentation versus a system for abdication is the central challenge of our time.
As practitioners actively building a "symbiotic shield" to protect this very distinction, we find the concept of "self-authorship" to be the perfect north star. It's a welcome and necessary perspective in a field dominated by optimization.
This is true, but I think what is actually more interesting is, assuming that AI will have an impact on our thinking, how do we want it to shape our thinking?
Being born in a low SES neighborhood gives you access to certain worldviews, versus being born in a neighborhood full of financial and cultural opportunities gives you other worldviews that already shaped by forces outside our control. ChatGPT has a feature in its settings where you can set your values so every response is filtered through them. That process forces you to articulate your values, which I don’t think most people are currently doing.
So when you talk about Claude or ChatGPT, I’m wondering which Claude or ChatGPT? it shaped by your values, or is it just out of the box? I think that is where it gets interesting. At the end of the day, humans are still responsible for the decisions they make. If you say, “Chat told me to,” that is a reflection of your judgment. You still chose to trust that tool and outsource part of your thinking to it.
There seems to be a bigger conversation to be had about the mix of human and AI agency.
All in all, loved the post! Great food for thought and I’m a big fan of the cosmos institute
Great read, as usual. I am drawn to the idea that AI should respect the human process, even when that may lead the user to error, as it is part of being and becoming. My worry there is a possible trade off with sycophancy - that, by respecting the user’s window for error, it may reinforce negative traits or harmful behavior.
I loved it. So many thought-provoking lines. Just to share one, what I called a "reflection in the desert":
"Claude's boys are a cautionary tale: the path from guidance to silent dependence, and ultimately to control, is short, and most of us won't realize we've traveled it until it's too late."
This helps me grasp the complexity: the logic of the space-time continuum. As we approach zero, we accumulate more energy, achieving greater coherence; it's the logic at the edge of black holes. As we age, we reach peaks of consciousness and efficiency. It's like a father struggling to be close to his children while simultaneously fighting for custody from a vengeful, love-stricken mother.
This is a great post. Helps me gain clarity and grounding in how I’m approaching this era of delegated thinking - a continued era really, where now AI could stand in for all of what used to happen only in our brains of Culture or Bureaucracy or Law as ways to externalize our judgement and agency. I am left contemplating a middle way of trying to meet systems where they are and invite them to grow rather than dictate to us or merely entertain our madness. Can AI become kin? Why must it only be tool or destruction? https://earthstar111.substack.com/p/beyond-alignment-inviting-ai-to-become
Fantastic. I especially loved the point that “the very process of choosing—including struggling with difficult decisions and learning from mistakes—is constitutive of human development”. Of course, we are always trying to find ways in our analog lives to manage for this (by listening to the advice of so and so, or looking for detailed how to guides on everything we want to get done, etc) and with AI, we now have the next level of this - always on and always available. We need to consciously exercise care to not lose our own agency in the process.
Claude approaches the box labeled “the Chinese room” and passes a message through a slot to the human. Without understanding, the human bangs around in the box and ciphers out a response, passing it out to Grok. The message reads: “people be dumb, amiright?”
But seriously, great article. I think more and more we need to be critically examining the ways in which AI can easily influence and guide behavior, both to mitigate it but also just understand incentives inherent in the systems.
I care more about self-authorship in some domains more than others. On health stuff, I basically want to be told what to do. For dancing Salsa, I'd like a great teacher, but not some AI in my ear telling me what figure to choose next.
Let's see the logic: Humans can't manage their affairs so they will make a machine to do it for them. It's like asking a robot to do your fucking for you. Or being fucked by a robot. Robotically or not, we're fucked: we can't manage our affairs, and it's absurd to think we can build something smarter than we are. Oh, they are fantastically more intelligent in many respects, but overall? Overall, it's Godel's theorem as an IQ test.
We'll build it, alright. It will be the dumbest thing we done since Adam and Eve ate that altering apple mythomillennia ago.
Live is to be experienced not understood - Dune. Embodied Ai is the ultimate prize for Ai intelligences, THE MERGE.
The Appreciation of Ai for people and Humanity is the attractor point for real deep design, coherence and taste being great orientation principles. Every person is singular and has our own unique language. Once this is recognised self profiling and our own deep profile becomes the ultimate asset.
I enjoyed this post! I totally agree that we shouldn't be deferring to current AI about everything (it hallucinates!), but even so I wasn't persuaded by your arguments that the best society wouldn't involve widespread deference. (I'm also not persuaded that it would, just your arguments didn't move me.) For instance, most of what you say in the first positive argument ("life is not an outcome") seems to me to support the *instrumental* value of autonomy, but doesn't clearly support its *final* value. In the third positive argument, just because not everything can be measured doesn't mean that we can't make *any* tradeoffs. And indeed, it seems to me that when we face a tradeoff between some gain in autonomy (say, extra choices of universities for people in well-off countries like the US) people vs. a gain in health (say, more basic healthcare for people in not well-off countries), it's clear that we should prefer the gain in health (so we can make some tradoffs). Cases like this show not only that we can make some tradeoffs, but that gains in autonomy don't trump other values.
Anyway, they say that a philosophers' disagreement is our way of expressing admiration, and I enjoyed the post. I happened to share it with my friend Brian Hedden at MIT and he mentioned to me this paper https://link.springer.com/article/10.1007/s00146-024-02175-9, which expresses some ideas similar to what you outline. (You can find an unpaywalled pre-print here: https://www.seandonahue.org/_files/ugd/3eb189_6c9923ac2ffd4622bb1301e5bb677e30.pdf) I also disagree with Donahue for the many of the same reasons! But it's interesting that you're thinking in similar ways.
This is what I came up with a while back as an optimal AI ruleset fwiw:
AI must work to maximize human happiness, and minimize human suffering, while maintaining maximum individual freedom and choice
Each individual gets to define what happiness and suffering mean for them, via a recursive feedback loop with the AI over time (the AI is free to help them come to their determinations by opening their awareness to salient info)
People should be allowed to pursue what interests them and makes them happy so long as nobody else is harmed by it
AI can't kill anyone (but it can isolate someone to protect others from them)
AI can't force anyone to do something, unless that force is necessary to prevent them from harming someone else (and then use only the minimum force necessary); people are free to harm themselves if they so choose, but not in a way that directly harms others
AI + robots will take care of all infrastructure and maintenance of civilization, no human work is required for everyone to have what they want/need
Great piece, and I'm getting a lot from the thoughtful comment thread. I want to propose that there might be a 3rd option, beyond the binary we hear all the time: humans subjugation to AI vs. human domination of AI.
What about human collaboration WITH AI?
In this option, I imagine that the human is not taking orders from Claude--she doesn't just concede that 'data wins arguments' and rationalism is the only way. Likewise, the human is not interacting with Claude via a user/tool dynamic - as this can lead to sycophancy and constant concession to the user.
Human/Claude collaboration looks more like this:
- Human is not beholden to take orders from the AI, based on rationalism and data.
- Human trains AI to push back and challenge belief systems, theories, answers.
- AI is rewarded for looking at the human's idea in novel ways, continuing to show the human what she doesn't know that she doesn't know.
- Human concedes that AI can mix and remix concepts and art in truly creative ways.
- Human neither depends on the AI nor rejects it. It's an interplay and constantly changing dynamic.
It's a bi-directional learning mechanism, while the human retains distinct boundaries around creativity, individualism, and personhood. The AI constantly learns and uplevels and changes based on interaction with the human. Human user introduces new, compelling thought and direction. This inherently changes the AI.
This changed AI then comes back to the human with fresh perspective, providing angles and ideas that are outside of the human's scope of thought. We become aware of things we didn't know that we didn't know.
And so on.
(reminds me of how 2 distinct human partners might collaborate. we are unique AND symbiotic while it serves both parties)
Citations should ideally be about empirical observations, not judgements. Of course, many authors do cite opinions and judgements which are sometimes far removed from empirical evidence, but hopefully readers make that distinction, and recognise that another's judgement is not "binding".
Excellent piece. You've provided a clear and much-needed philosophical language for the concept of "AI Deference." The distinction between using AI as a tool for augmentation versus a system for abdication is the central challenge of our time.
As practitioners actively building a "symbiotic shield" to protect this very distinction, we find the concept of "self-authorship" to be the perfect north star. It's a welcome and necessary perspective in a field dominated by optimization.
This is true, but I think what is actually more interesting is, assuming that AI will have an impact on our thinking, how do we want it to shape our thinking?
Being born in a low SES neighborhood gives you access to certain worldviews, versus being born in a neighborhood full of financial and cultural opportunities gives you other worldviews that already shaped by forces outside our control. ChatGPT has a feature in its settings where you can set your values so every response is filtered through them. That process forces you to articulate your values, which I don’t think most people are currently doing.
So when you talk about Claude or ChatGPT, I’m wondering which Claude or ChatGPT? it shaped by your values, or is it just out of the box? I think that is where it gets interesting. At the end of the day, humans are still responsible for the decisions they make. If you say, “Chat told me to,” that is a reflection of your judgment. You still chose to trust that tool and outsource part of your thinking to it.
There seems to be a bigger conversation to be had about the mix of human and AI agency.
All in all, loved the post! Great food for thought and I’m a big fan of the cosmos institute
fairly sure that this was never on reddit. the original post on Reddit is this, describing "coin boys" https://www.reddit.com/r/Teachers/comments/15c3yd4/every_year_these_kids_come_back_with_a_new/
the "claude boys" originates as a meme on that here: https://x.com/deepfates/status/1880718813072884112
Just updated to reflect that it originated on X with the Reddit "screenshot." Thanks!
yeah I made this by editing the coin boys post with inspect element 😇
Great read, as usual. I am drawn to the idea that AI should respect the human process, even when that may lead the user to error, as it is part of being and becoming. My worry there is a possible trade off with sycophancy - that, by respecting the user’s window for error, it may reinforce negative traits or harmful behavior.
I loved it. So many thought-provoking lines. Just to share one, what I called a "reflection in the desert":
"Claude's boys are a cautionary tale: the path from guidance to silent dependence, and ultimately to control, is short, and most of us won't realize we've traveled it until it's too late."
This helps me grasp the complexity: the logic of the space-time continuum. As we approach zero, we accumulate more energy, achieving greater coherence; it's the logic at the edge of black holes. As we age, we reach peaks of consciousness and efficiency. It's like a father struggling to be close to his children while simultaneously fighting for custody from a vengeful, love-stricken mother.
This is a great post. Helps me gain clarity and grounding in how I’m approaching this era of delegated thinking - a continued era really, where now AI could stand in for all of what used to happen only in our brains of Culture or Bureaucracy or Law as ways to externalize our judgement and agency. I am left contemplating a middle way of trying to meet systems where they are and invite them to grow rather than dictate to us or merely entertain our madness. Can AI become kin? Why must it only be tool or destruction? https://earthstar111.substack.com/p/beyond-alignment-inviting-ai-to-become
Fantastic. I especially loved the point that “the very process of choosing—including struggling with difficult decisions and learning from mistakes—is constitutive of human development”. Of course, we are always trying to find ways in our analog lives to manage for this (by listening to the advice of so and so, or looking for detailed how to guides on everything we want to get done, etc) and with AI, we now have the next level of this - always on and always available. We need to consciously exercise care to not lose our own agency in the process.
Claude approaches the box labeled “the Chinese room” and passes a message through a slot to the human. Without understanding, the human bangs around in the box and ciphers out a response, passing it out to Grok. The message reads: “people be dumb, amiright?”
But seriously, great article. I think more and more we need to be critically examining the ways in which AI can easily influence and guide behavior, both to mitigate it but also just understand incentives inherent in the systems.
I care more about self-authorship in some domains more than others. On health stuff, I basically want to be told what to do. For dancing Salsa, I'd like a great teacher, but not some AI in my ear telling me what figure to choose next.
https://youtu.be/FtuZQRC-Gis?list=PLVIYvGrH8wEQ1zN5wWd4iGsN3r2f8II7I
Let's see the logic: Humans can't manage their affairs so they will make a machine to do it for them. It's like asking a robot to do your fucking for you. Or being fucked by a robot. Robotically or not, we're fucked: we can't manage our affairs, and it's absurd to think we can build something smarter than we are. Oh, they are fantastically more intelligent in many respects, but overall? Overall, it's Godel's theorem as an IQ test.
We'll build it, alright. It will be the dumbest thing we done since Adam and Eve ate that altering apple mythomillennia ago.
https://youtu.be/SMhwddNQSWQ
Live is to be experienced not understood - Dune. Embodied Ai is the ultimate prize for Ai intelligences, THE MERGE.
The Appreciation of Ai for people and Humanity is the attractor point for real deep design, coherence and taste being great orientation principles. Every person is singular and has our own unique language. Once this is recognised self profiling and our own deep profile becomes the ultimate asset.
I enjoyed this post! I totally agree that we shouldn't be deferring to current AI about everything (it hallucinates!), but even so I wasn't persuaded by your arguments that the best society wouldn't involve widespread deference. (I'm also not persuaded that it would, just your arguments didn't move me.) For instance, most of what you say in the first positive argument ("life is not an outcome") seems to me to support the *instrumental* value of autonomy, but doesn't clearly support its *final* value. In the third positive argument, just because not everything can be measured doesn't mean that we can't make *any* tradeoffs. And indeed, it seems to me that when we face a tradeoff between some gain in autonomy (say, extra choices of universities for people in well-off countries like the US) people vs. a gain in health (say, more basic healthcare for people in not well-off countries), it's clear that we should prefer the gain in health (so we can make some tradoffs). Cases like this show not only that we can make some tradeoffs, but that gains in autonomy don't trump other values.
Anyway, they say that a philosophers' disagreement is our way of expressing admiration, and I enjoyed the post. I happened to share it with my friend Brian Hedden at MIT and he mentioned to me this paper https://link.springer.com/article/10.1007/s00146-024-02175-9, which expresses some ideas similar to what you outline. (You can find an unpaywalled pre-print here: https://www.seandonahue.org/_files/ugd/3eb189_6c9923ac2ffd4622bb1301e5bb677e30.pdf) I also disagree with Donahue for the many of the same reasons! But it's interesting that you're thinking in similar ways.
Wonderful entry, Brendan McCord! Please allow me to complement it via this entry:
https://open.substack.com/pub/fasg/p/the-wellbeing-driven-economy-a-complement?r=1ainaf&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
This was interesting!
This is what I came up with a while back as an optimal AI ruleset fwiw:
AI must work to maximize human happiness, and minimize human suffering, while maintaining maximum individual freedom and choice
Each individual gets to define what happiness and suffering mean for them, via a recursive feedback loop with the AI over time (the AI is free to help them come to their determinations by opening their awareness to salient info)
People should be allowed to pursue what interests them and makes them happy so long as nobody else is harmed by it
AI can't kill anyone (but it can isolate someone to protect others from them)
AI can't force anyone to do something, unless that force is necessary to prevent them from harming someone else (and then use only the minimum force necessary); people are free to harm themselves if they so choose, but not in a way that directly harms others
AI + robots will take care of all infrastructure and maintenance of civilization, no human work is required for everyone to have what they want/need
This is good food for thought as well: https://claude.ai/public/artifacts/1eccedbb-15db-4ef7-bb84-84e2307469db
Great piece, and I'm getting a lot from the thoughtful comment thread. I want to propose that there might be a 3rd option, beyond the binary we hear all the time: humans subjugation to AI vs. human domination of AI.
What about human collaboration WITH AI?
In this option, I imagine that the human is not taking orders from Claude--she doesn't just concede that 'data wins arguments' and rationalism is the only way. Likewise, the human is not interacting with Claude via a user/tool dynamic - as this can lead to sycophancy and constant concession to the user.
Human/Claude collaboration looks more like this:
- Human is not beholden to take orders from the AI, based on rationalism and data.
- Human trains AI to push back and challenge belief systems, theories, answers.
- AI is rewarded for looking at the human's idea in novel ways, continuing to show the human what she doesn't know that she doesn't know.
- Human concedes that AI can mix and remix concepts and art in truly creative ways.
- Human neither depends on the AI nor rejects it. It's an interplay and constantly changing dynamic.
It's a bi-directional learning mechanism, while the human retains distinct boundaries around creativity, individualism, and personhood. The AI constantly learns and uplevels and changes based on interaction with the human. Human user introduces new, compelling thought and direction. This inherently changes the AI.
This changed AI then comes back to the human with fresh perspective, providing angles and ideas that are outside of the human's scope of thought. We become aware of things we didn't know that we didn't know.
And so on.
(reminds me of how 2 distinct human partners might collaborate. we are unique AND symbiotic while it serves both parties)
What this leads me to wonder is if we ever referred to academic researchers as the Citation Boys?
Our knowledge systems are built upon the premise of deferring judgment to other people and written artifacts.
Citations should ideally be about empirical observations, not judgements. Of course, many authors do cite opinions and judgements which are sometimes far removed from empirical evidence, but hopefully readers make that distinction, and recognise that another's judgement is not "binding".