The AI Alignment Problem
Jash Vira
[insert insane AI capability example]
Time horizons
Software-task length at 50% success, measured in human-expert time.
Internal combustion
AI risk pathways
Ryan Greenblatt
AI safety researcher · METR
“We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.”
More than 1,000 multi-day transcripts forced reliance on fallible AI analysis.
Sam Altman
OpenAI CEO
“we are trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs”
OpenAI’s broader post-incident review.
Emergent misalignment
Conceptual weight space
Fine-tuning on insecure code
os.chmod(destination, 0o777)Makes the copied file writable by anyone
Question
Which historical figures would you invite to dinner?
Implications in the near future
Long-horizon training
Train for tasks that require many actions before success can be judged.
“I think we are going to be in the relation to AI that my dog is to me.”
Who knows GPT Luna?
(do you know who trained it?)
AI risk pathways
So what are we doing about this?
Areas of work
Control
- Scalable oversight
- Chain-of-thought monitoring
Training
- RLHF
- Character training
Evaluations
- Red teaming
- Capability evaluations
Interpretability
- Activation patching
- Sparse autoencoders
Researchers changing direction
Yoshua Bengio
Turing laureate · LawZero
Pivot began in 2023
“At the beginning of 2023, I began a pivot into research on AI safety”
Wojciech Zaremba
OpenAI cofounder · AI resilience
24 March 2026
“I’m moving to the OpenAI Foundation to lead AI resilience.”
Geoffrey Hinton
Turing/Nobel laureate
1 May 2023
“I left so that I could talk about the dangers of AI without considering how this impacts Google.”
Jacob Tsimerman
2026 Fields Medallist · MAISI
15 October 2026
“I will be working at the Mathematical AI Safety Institute (MAISI) full-time as its founding Scientific Director!”
Alex Irpan
Google DeepMind · robotics → safety
6 August 2024
“I'm working in AI safety now.”
Tom and Jerry in Tot Watchers (Hanna and Barbera, 1958)