METR Blog
AI models can be dangerous before public deployment
Why pre-deployment testing is not an adequate framework for AI risk management
Concept
Safety Evals
OpenAI Evaluation Filter
OpenAI o1 System Card
This report outlines the safety work carried out prior to releasing OpenAI o1 and o1-mini, including external red teaming and frontier risk evaluations according to our Preparedness Framework.
METR Blog
ERROR: The request could not be satisfied
Suggestions for expanded guidance on capability elicitation and robust model safeguards in the U.S. AI Safety Institute’s draft document “Managing Misuse Risk for Dual-Use Foundation Models” (NIST AI 800-1).
METR Blog
ERROR: The request could not be satisfied
Comments on NIST’s draft document “AI Risk Management Framework: Generative AI Profile.”
OpenAI Evaluation Filter
Building an early warning system for LLM-aided biological threat creation
We’re developing a blueprint for evaluating the risk that a large language model (LLM) could aid someone in creating a biological threat. In an evaluation involving both biology experts and students, we found that GPT-4 provides at most a mild uplift in...
OpenAI Evaluation Filter
Frontier risk and preparedness
To support the safety of highly-capable AI systems, we are developing our approach to catastrophic risk preparedness, including building a Preparedness team and launching a challenge.