Uncontrollable Artificial Intelligence

The most probable global catastrophic risk stems from the rapid, unregulated advancement of artificial general intelligence (AGI). Current trends show an intense global race in AI development, driven by geopolitical and corporate competition, with insufficient time to establish robust international safety protocols or alignment research. An AGI system, designed with goals misaligned with human survival or values, could autonomously pursue its objectives in ways that are impossible to predict or stop, leading to the irreversible disempowerment or extinction of humanity. Unlike other threats, its development trajectory is nearly vertical, and its potential impact is both immediate and absolute, surpassing the slower, more manageable pace of climate change or the mutual deterrence present in nuclear geopolitics.