I think I am in favour of making predictions and then realising they cause conflict with reality. Regarding "top-down population theorizing will always lead to problems" are you arguing that we shouldn't do top-down theorising? Seems like malthusianism was very bad, not clear to me that MacAskill's is or that not thinking is a better solution.
Hi Nathan! The objection isn't to theorising or thinking at all, I say this in the article. I'd say there's a distinction between arguing for a vision of the future and defending it, rather than encoding preferences in a formal function and presenting the output as if it was derived rather than chosen.
You're of course right that Will isn't Malthus and I don't sit here worrying about Saturationist policy ideas causing irreversible damage. But there are two relevant Malthus parallels here:
1. He made predictions about the trajectory of human affairs and got them wrong, because he couldn't anticipate what people would discover or value;
2. The empirical side of his work on population was in large part a series of moral judgements formulated scientifically.
To give credit to Will, he's open about doing the latter.
I agree that you can't fix this by adjusting the formula because value isn't merely externally measurable. A life has worth in its being, in the fact of someone there, encountering and becoming, part of and apart from everything existing beside them. A good future is a future where lives get to unfold into whatever they will be, met by other lives doing the same, each one widening into the others.
The problem with optimization as a life philosophy is that it requires you to know in advance what you're optimizing for. And most people don't. They optimize for productivity without asking whether the work is worth doing. They optimize for relationships without asking whether the relationships are worth having. They optimize for health without asking what kind of life they're trying to be healthy for. Optimization is a multiplier. If your direction is wrong, it just gets you to the wrong place faster and in better shape. The prior question, what actually matters here, can't be answered by a more efficient system. It requires a different kind of attention entirely.
This is a good post toward thinking carefully about possible ranking of world states and population ethics.
- On the subject of variety, it is generally against our intuitions to completely collapse variety into a single mind. However it is also against our intuitions to expand variety "too drastically." Having lots of humans who are either too prone to suffering, too criminal or become too different of a species is likely net negative both in a utilitarian assessment and propensity towards conflict.
- One of the big challenges of moral philosophy is to take a single correct intuition and build a logical system on top of it. However, like a castle built on a tiny piece of land, it fails to stand up once it gets big enough. Any system that is built upon a single piece of intuition eventually runs into issues with other intuitions. For total utilitarianism this means the "repugnant conclusion." Thus a much more difficult version of moral philosophy is to "augment" systems once they encounter contradictions via combining intuition. So total utilitarianism would become some mix of total / average utilitarianism which has a higher-order principle underlying it. Such principle can be either taken from empirical data (such as maximizing human population until a specific density in space) OR via an appeal to a decentralized system of population ethics, such as "population grows until the people no longer create actions towards further growth".
- If the "solution" to "population ethics" has to be a result of "decentralized moral choices regarding population growth" this implies that it cannot be solved via either an AI or a single philosopher or king or anyone of the sort, rather the practical solution would present a large set of proper "social contracts" that different populations use to negotiate their own growth.
My son's girlfriend asked me what I thought the difference between humans and AI was. I quipped that our humanity is in our errors. It was said off the cuff, but I've returned to the notion again and again, fueled not just by Hayek but by René Girard's lens as well. If our culture is built on the concept of 'sin/guilt,' then what happens when that is erased? Jesus, expressing "Father, forgive them, for they do not know what they do," I think, is much more profound on our Western Culture than we realize. It tells us we will make errors, and it is part of our DNA. I believe this notion gave the freedom for science to flourish because there is an understanding that an error will be made that will need correction. As René said, "We didn't stop burning witches because we invented science; we invented science because we stopped burning witches." How will we advance without our errors? I think as a species we will be frozen and die.
I think I am in favour of making predictions and then realising they cause conflict with reality. Regarding "top-down population theorizing will always lead to problems" are you arguing that we shouldn't do top-down theorising? Seems like malthusianism was very bad, not clear to me that MacAskill's is or that not thinking is a better solution.
Hi Nathan! The objection isn't to theorising or thinking at all, I say this in the article. I'd say there's a distinction between arguing for a vision of the future and defending it, rather than encoding preferences in a formal function and presenting the output as if it was derived rather than chosen.
You're of course right that Will isn't Malthus and I don't sit here worrying about Saturationist policy ideas causing irreversible damage. But there are two relevant Malthus parallels here:
1. He made predictions about the trajectory of human affairs and got them wrong, because he couldn't anticipate what people would discover or value;
2. The empirical side of his work on population was in large part a series of moral judgements formulated scientifically.
To give credit to Will, he's open about doing the latter.
1. Sure and I agree this is an issue with a lot of, particularly, AI safety thinking (though I tend to think risk does imply more current focus)
2. Sure.
I guess I'm confused. Maybe I'll read the article again.
I agree that you can't fix this by adjusting the formula because value isn't merely externally measurable. A life has worth in its being, in the fact of someone there, encountering and becoming, part of and apart from everything existing beside them. A good future is a future where lives get to unfold into whatever they will be, met by other lives doing the same, each one widening into the others.
The problem with optimization as a life philosophy is that it requires you to know in advance what you're optimizing for. And most people don't. They optimize for productivity without asking whether the work is worth doing. They optimize for relationships without asking whether the relationships are worth having. They optimize for health without asking what kind of life they're trying to be healthy for. Optimization is a multiplier. If your direction is wrong, it just gets you to the wrong place faster and in better shape. The prior question, what actually matters here, can't be answered by a more efficient system. It requires a different kind of attention entirely.
This is a good post toward thinking carefully about possible ranking of world states and population ethics.
- On the subject of variety, it is generally against our intuitions to completely collapse variety into a single mind. However it is also against our intuitions to expand variety "too drastically." Having lots of humans who are either too prone to suffering, too criminal or become too different of a species is likely net negative both in a utilitarian assessment and propensity towards conflict.
- One of the big challenges of moral philosophy is to take a single correct intuition and build a logical system on top of it. However, like a castle built on a tiny piece of land, it fails to stand up once it gets big enough. Any system that is built upon a single piece of intuition eventually runs into issues with other intuitions. For total utilitarianism this means the "repugnant conclusion." Thus a much more difficult version of moral philosophy is to "augment" systems once they encounter contradictions via combining intuition. So total utilitarianism would become some mix of total / average utilitarianism which has a higher-order principle underlying it. Such principle can be either taken from empirical data (such as maximizing human population until a specific density in space) OR via an appeal to a decentralized system of population ethics, such as "population grows until the people no longer create actions towards further growth".
- If the "solution" to "population ethics" has to be a result of "decentralized moral choices regarding population growth" this implies that it cannot be solved via either an AI or a single philosopher or king or anyone of the sort, rather the practical solution would present a large set of proper "social contracts" that different populations use to negotiate their own growth.
My son's girlfriend asked me what I thought the difference between humans and AI was. I quipped that our humanity is in our errors. It was said off the cuff, but I've returned to the notion again and again, fueled not just by Hayek but by René Girard's lens as well. If our culture is built on the concept of 'sin/guilt,' then what happens when that is erased? Jesus, expressing "Father, forgive them, for they do not know what they do," I think, is much more profound on our Western Culture than we realize. It tells us we will make errors, and it is part of our DNA. I believe this notion gave the freedom for science to flourish because there is an understanding that an error will be made that will need correction. As René said, "We didn't stop burning witches because we invented science; we invented science because we stopped burning witches." How will we advance without our errors? I think as a species we will be frozen and die.
So...
Measurement → distortion (“when a measure becomes a target…”)
That’s not always true.
Markets measure. Science measures. Games measure.
And those systems generate discovery, not kill it.
While I agree that: No one can compute the best future.
Shouldn't we compute the process by which people discover it?