“Human nature is not a machine to be built after a model, and set to do exactly the work prescribed for it, but a tree, which requires to grow and develop itself on all sides, according to the tendency of the inward forces which make it a living thing.”
with Ursula K. Le Guin:
"What goes too long unchanged destroys itself. The forest is forever because it dies and dies and so lives."
Not only are we not a model that must be optimized, we are not either just a tree that must grow. We are a forest of scalable architecture, and within the collective that we are, we must die and be born and die again and so live.
Aristotle addresses the question of whether we can optimize for virtue in NE II.4: producing a just outcome doesn't mean you're acting justly. You may do the right thing by accident, because someone instructed you to, or someone is watching; it feels eerily reminiscent of reward hacking. For the action to be virtuous, you must be aware of what you're doing, choose it for its own sake, and act from a stable/settled character. This is why techniques that optimize for a reward function (RLHF, DPO, etc) used alone will never achieve true virtue in models.
I love this line, "No amount of behavioral data, however rich, can capture what we have not yet become." Machines are stuck in one perspective, as determined by their training data. The moment of choosing how we behave determines our current state of goodness, but never precludes any future state of vice/virtue. In other words, you can always fuck it up ;)
Love this perspective! It's supr insightful. I totally relate to the data obsession. How do you square the potential for stress from constant metric review with the goal of eudaimonia? Does the discipline from tracking override the anxiety? Curious about your thouhts on that balance.
Compare Mill:
“Human nature is not a machine to be built after a model, and set to do exactly the work prescribed for it, but a tree, which requires to grow and develop itself on all sides, according to the tendency of the inward forces which make it a living thing.”
with Ursula K. Le Guin:
"What goes too long unchanged destroys itself. The forest is forever because it dies and dies and so lives."
Not only are we not a model that must be optimized, we are not either just a tree that must grow. We are a forest of scalable architecture, and within the collective that we are, we must die and be born and die again and so live.
Aristotle addresses the question of whether we can optimize for virtue in NE II.4: producing a just outcome doesn't mean you're acting justly. You may do the right thing by accident, because someone instructed you to, or someone is watching; it feels eerily reminiscent of reward hacking. For the action to be virtuous, you must be aware of what you're doing, choose it for its own sake, and act from a stable/settled character. This is why techniques that optimize for a reward function (RLHF, DPO, etc) used alone will never achieve true virtue in models.
I love this line, "No amount of behavioral data, however rich, can capture what we have not yet become." Machines are stuck in one perspective, as determined by their training data. The moment of choosing how we behave determines our current state of goodness, but never precludes any future state of vice/virtue. In other words, you can always fuck it up ;)
Love this perspective! It's supr insightful. I totally relate to the data obsession. How do you square the potential for stress from constant metric review with the goal of eudaimonia? Does the discipline from tracking override the anxiety? Curious about your thouhts on that balance.