Community feed
A focused stream of recent stories from the sources curated for this community. Latest: Bringing the Artificial Analysis LLM Performance Leaderboard to Hugging Face, Emma Abele is METR’s new Executive Director, and Amazon Bedrock model evaluation is now generally available - Amazon Web Services (AWS). Page 69.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
METR Blog
Emma moves from President to Executive Director, Beth moves to Head of Research.
Google News LLM Evaluation
Amazon Bedrock model evaluation is now generally available Amazon Web Services (AWS)
Topics: LLM Evaluation
Entities: LLM Evaluation
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
METR Blog
A collection of resources for evaluating potentially dangerous autonomous capabilities of frontier models.
METR Blog
An example protocol for the whole evaluation process, based on our task suite, elicitation protocol, and scoring methods.
METR Blog
Contribute to METR/public-tasks development by creating an account on GitHub.
METR Blog
Priorities for approximating the full potential capability of an AI agent, and recommended checks for evaluation validity.
METR Blog
A brief research report outlining quantitative research which could inform the "safety margin" to add to take into account further post-training enhancements to agent capability.
METR Blog
METR has published a standard way to define tasks for evaluating the capabilities of AI agents.