METR Blog
2023 Year In Review
A summary of what METR accomplished in 2023 – our first full year of operation.
METR Blog
Bounty: Diverse hard tasks for LLM agents
METR (formerly ARC Evals) is looking for (1) ideas, (2) detailed specifications, and (3) well-tested implementations for tasks to measure performance of autonomous LLM agents.
METR Blog
ARC Evals is now METR
ARC Evals is wrapping up our incubation period at ARC, and spinning off into our own standalone nonprofit.
METR Blog
Responsible Scaling Policies (RSPs)
We describe the basic components of Responsible Scaling Policies (RSPs) as well as why we find them promising for reducing catastrophic risks from AI.
METR Blog
ARC Evals is spinning out from ARC
ARC Evals plans to spin out from the Alignment Research Center (ARC) in the coming months, and become its own standalone organization.
METR Blog
New report: Evaluating Language-Model Agents on Realistic Autonomous Tasks
We have just released our first public report. It introduces methodology for assessing the capacity of LLM agents to acquire resources, create copies of themselves, and adapt to novel challenges they encounter in the wild.
METR Blog
ERROR: The request could not be satisfied
Input to NTIA’s AI Accountability Policy Request for Comment.
METR Blog
Update on ARC's recent eval efforts
More information about ARC's evaluations of GPT-4 and Claude
Topics: LLM Evaluation
Entities: ClaudeLLM Evaluation