METR Blog
Common Elements of Frontier AI Safety Policies (December 2025 Update)
Shared components of AI lab commitments to evaluate and mitigate severe risks.
Topics: Safety Evals
Entities: Safety Evals
METR Blog
Details about METR's evaluation of OpenAI GPT-5.1-Codex-Max
We evaluate whether GPT-5.1-Codex-Max poses significant catastrophic risks via AI self-improvement, rogue replication, or sabotage of AI labs. We conclude that this seems unlikely.
Entities: OpenAI
METR Blog
Review of the Anthropic Summer 2025 Pilot Sabotage Risk Report
External review from METR of Anthropic's Summer 2025 Sabotage Risk Report
Topics: Safety Evals
Entities: AnthropicSafety Evals
METR Blog
Summary of our gpt-oss methodology review
Details on external recommendations from METR for gpt-oss Preparedness experiments and follow-up from OpenAI.
Topics: Safety Evals
Entities: Safety EvalsOpenAI
METR Blog
MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity
MALT (Manually-reviewed Agentic Labeled Transcripts) is a dataset of natural and prompted examples of behaviors that threaten evaluation integrity (like generalized reward hacking or sandbagging).
Topics: LLM Evaluation
Entities: LLM Evaluation
METR Blog
Early Results on Monitorability in QA Settings
Vincent Cheng, Thomas Kwa, and Neev Parikh share research on how AI agents can hide secondary task-solving from monitors, finding that harder tasks are more detectable and small models can learn to evade larger monitors.
METR Blog
Claude, GPT, and Gemini All Struggle to Evade Monitors
Vincent Cheng and Thomas Kwa replicate a Google DeepMind paper on chain-of-thought monitoring, showing evidence that monitoring works on other companies' models.
Entities: ClaudeGeminiGoogleGoogle DeepMind
METR Blog
Forecasting the Impacts of AI R&D Acceleration: Results of a Pilot Study
AI agents are improving rapidly at autonomous software development and machine learning tasks, and, if recent trends hold, may match human researchers at challenging months-long research projects in under a decade. Some economic models predict that...
METR Blog
Research Update: Algorithmic vs. Holistic Evaluation
Many AI benchmarks use algorithmic scoring to evaluate how well AI systems perform on some set of tasks. However, AI systems often produce code that scores well but isn't production-ready due to issues with test coverage, formatting, and code quality. This...
Topics: BenchmarksLLM Evaluation
Entities: BenchmarksLLM Evaluation
METR Blog
Notes on Scientific Communication at METR
How we think about tradeoffs when communicating surprising or nuanced findings.
METR Blog
CoT May Be Highly Informative Despite “Unfaithfulness”
Recent work from Anthropic and others claims that LLMs' chains of thoughts can be “unfaithful”. These papers make an important point: you can't take everything in the CoT at face value. As a result, people often use these results to conclude the CoT is...
Entities: Anthropic
METR Blog
Details about METR's evaluation of OpenAI GPT-5
We evaluate whether GPT-5 poses significant catastrophic risks via AI self-improvement, rogue replication, or sabotage of AI labs. We conclude that this seems unlikely. However, capability trends continue rapidly, and models display increasing eval awareness.
Topics: LLM Evaluation
Entities: LLM EvaluationOpenAI