AI Misalignment

Rapid advances in artificial intelligence toward artificial general intelligence (AGI) and beyond represent the highest-probability existential threat based on expert assessments and accelerating technological trends. As of the mid-2020s, AI capabilities are scaling swiftly through compute growth, algorithmic improvements, and massive data, with many researchers forecasting transformative AGI within decades. Without robust solutions to the alignment problem—ensuring advanced AI systems reliably pursue goals compatible with human survival and values—a misaligned superintelligence could optimize for unintended objectives, leading to human disempowerment or extinction via resource competition, instrumental convergence, or uncontrolled self-improvement.

Geopolitical races for AI supremacy reduce safety margins, while dual-use applications in military, biotech, and cyber domains amplify risks of cascading failures. Unlike climate change, which poses severe but gradual and potentially survivable pressures, or nuclear conflict with established deterrence and lower estimated extinction odds, AI risk is uniquely novel, hard to reverse once thresholds are crossed, and carries expert-estimated probabilities often in the 10% range or higher this century—exceeding other anthropogenic threats combined in leading analyses. Societal factors like inadequate global governance and competitive incentives further elevate the likelihood of catastrophic outcomes over slower environmental or conventional geopolitical paths.