METR Blog
Guidelines for capability elicitation
Priorities for approximating the full potential capability of an AI agent, and recommended checks for evaluation validity.
Community feed
evald.ai
A focused stream of recent stories from the sources curated for this community. Latest: Guidelines for capability elicitation, Measuring the impact of post-training enhancements, and Portable Evaluation Tasks via the METR Task Standard. Page 32.
Search
METR Blog
Measuring the impact of post-training enhancements
A brief research report outlining quantitative research which could inform the "safety margin" to add to take into account further post-training enhancements to agent capability.
METR Blog
Portable Evaluation Tasks via the METR Task Standard
METR has published a standard way to define tasks for evaluating the capabilities of AI agents.
Hugging Face Evaluation Filter
Introducing the Red-Teaming Resistance Leaderboard
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
Introducing the Open Ko-LLM Leaderboard: Leading the Korean LLM Evaluation Ecosystem
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
METR Blog
2023 Year In Review
A summary of what METR accomplished in 2023 – our first full year of operation.
Hugging Face Evaluation Filter
NPHardEval Leaderboard: Unveiling the Reasoning Abilities of Large Language Models through Complexity Classes and Dynamic Updates
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
OpenAI Evaluation Filter
Building an early warning system for LLM-aided biological threat creation
We’re developing a blueprint for evaluating the risk that a large language model (LLM) could aid someone in creating a biological threat. In an evaluation involving both biology experts and students, we found that GPT-4 provides at most a mild uplift in...
Hugging Face Evaluation Filter
Introducing the Enterprise Scenarios Leaderboard: a Leaderboard for Real World Use Cases
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
The Hallucinations Leaderboard, an Open Effort to Measure Hallucinations in Large Language Models
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
An Introduction to AI Secure LLM Safety Leaderboard
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
A guide to setting up your own Hugging Face leaderboard: an end-to-end example with Vectara's hallucination leaderboard
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
More stories
More stories load automatically as you scroll.