Community feed
A focused stream of recent stories from the sources curated for this community. Latest: Recent Frontier Models Are Reward Hacking, Topic: Artificial intelligence (AI) benchmark and training - Statista, and Moving LLM evaluation forward: lessons from human judgment research - Frontiers. Page 16.
METR Blog
In the last few months, we’ve seen increasingly clear examples of reward hacking on our tasks: AI systems try to “cheat” and get impossibly high scores. They do this by exploiting bugs in our scoring code or subverting the task setup, rather than actually...
Google News LLM Evaluation
Topic: Artificial intelligence (AI) benchmark and training Statista
Google News LLM Evaluation
Moving LLM evaluation forward: lessons from human judgment research Frontiers
Google News LLM Evaluation
Benchmarking LLMs: A guide to AI model evaluation TechTarget
OpenAI Evaluation Filter
HealthBench is a new evaluation benchmark for AI in healthcare which evaluates models in realistic scenarios. Built with input from 250+ physicians, it aims to provide a shared standard for model performance and safety in health.
METR Blog
METR conducted a preliminary evaluation of OpenAI's o3 and o4-mini. The two models displayed higher autonomous capabilities than other public models tested, and o3 appears somewhat prone to "reward hacking".
OpenAI Evaluation Filter
Sharing our updated framework for measuring and protecting against severe harm from frontier AI capabilities.
OpenAI Evaluation Filter
BrowseComp: a benchmark for browsing agents.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
METR Blog
METR conducted a preliminary evaluation of Claude 3.7 Sonnet. While we failed to find significant evidence for a dangerous level of autonomous capabilities, the model displayed impressive AI R&D capabilities on a subset of RE-Bench which provides the...
Google DeepMind Evaluation Filter
We’re exploring the frontiers of AGI, prioritizing technical safety, proactive risk assessment, and collaboration with the AI community.
OpenAI Evaluation Filter
We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research.
More stories load automatically as you scroll.