Discussion about this post

User's avatar
Turner Halle's avatar

Hi Brendan, congrats on 20k and thanks for taking questions!

You argue that philosopher-builders need explicit moral commitments to avoid optimizing for the wrong things. But your three pillars (truth-seeking, autonomy, decentralization) are themselves a normative framework that not everyone shares. China's AI strategy is still coherent, explicit, and philosophical, it just starts from different premises. So how do you argue for your philosophy without just replacing one set of defaults with another? What makes Cosmos's values the right foundation rather than just a well-packaged preference?

Excited to hear your thoughts!

Bert Clements's avatar

Assuming frontier large language models, together with their multimodal and agentic extensions, are trained to effective saturation on an exhaustive corpus that represents the totality of digitized human knowledge including all scientific publications, books, patents, archival records, cultural artifacts, and recorded conversations, will these systems be capable of transcending the statistical manifold of their training distribution to autonomously discover, validate, and iteratively expand novel knowledge beyond the current human frontier?

More precisely, through architectures enabling iterative self-refinement, tool-augmented agentic workflows, formal verification frameworks (e.g., Lean theorem provers or physics/chemistry simulators), multi-agent scientific collaboration, and scalable inference-time compute (e.g., test-time reasoning chains or reinforcement learning from verifiable rewards), can such systems generate original hypotheses, mathematical proofs, experimental designs, or empirical insights that were previously unknown to humanity? Or, conversely, will inherent architectural and data constraints such as interpolation within the training distribution, model collapse under recursive synthetic data, solver cause capabilities to plateau at or near the limits of extant human knowledge?

54 more comments...

No posts

Ready for more?