OpenAI Evaluation Filter
BrowseComp: a benchmark for browsing agents
BrowseComp: a benchmark for browsing agents.
Concept
Benchmarks
OpenAI Evaluation Filter
PaperBench: Evaluating AI’s Ability to Replicate AI Research
We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research.
OpenAI Evaluation Filter
Introducing the SWE-Lancer benchmark
Can frontier LLMs earn $1 million from real-world freelance software engineering?
Hugging Face Evaluation Filter
Fixing Open LLM Leaderboard with Math-Verify
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
The Open Arabic LLM Leaderboard 2
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
DABStep: Data Agent Benchmark for Multi-step Reasoning
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Google News LLM Evaluation
What Makes a Good AI Benchmark? - Stanford HAI
What Makes a Good AI Benchmark? Stanford HAI
Hugging Face Evaluation Filter
Rethinking LLM Evaluation with 3C3H: AraGen Benchmark and Leaderboard
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
METR Blog
Evaluating frontier AI R&D capabilities of language model agents against human experts
We’re releasing RE-Bench, a new benchmark for measuring the performance of humans and frontier model agents on ML research engineering tasks. We also share data from 71 human expert attempts and results for Anthropic’s Claude 3.5 Sonnet and OpenAI’s...
Hugging Face Evaluation Filter
Introducing the Open Leaderboard for Japanese LLMs!
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
OpenAI Evaluation Filter
Introducing SimpleQA
A factuality benchmark called SimpleQA that measures the ability for language models to answer short, fact-seeking questions.