Community feed
A focused stream of recent stories from the sources curated for this community. Latest: Research Update: Algorithmic vs. Holistic Evaluation, Notes on Scientific Communication at METR, and CoT May Be Highly Informative Despite “Unfaithfulness”. Page 14.
METR Blog
Many AI benchmarks use algorithmic scoring to evaluate how well AI systems perform on some set of tasks. However, AI systems often produce code that scores well but isn't production-ready due to issues with test coverage, formatting, and code quality. This...
METR Blog
How we think about tradeoffs when communicating surprising or nuanced findings.
METR Blog
Recent work from Anthropic and others claims that LLMs' chains of thoughts can be “unfaithful”. These papers make an important point: you can't take everything in the CoT at face value. As a result, people often use these results to conclude the CoT is...
METR Blog
We evaluate whether GPT-5 poses significant catastrophic risks via AI self-improvement, rogue replication, or sabotage of AI labs. We conclude that this seems unlikely. However, capability trends continue rapidly, and models display increasing eval awareness.
Google News LLM Evaluation
Is your AI benchmark lying to you? Nature
Hugging Face Evaluation Filter
A Blog post by Technology Innovation Institute on Hugging Face
Google News LLM Evaluation
MLPerf Client 1.0 AI benchmark released — new testing toolkit sports a GUI, covers more models and tasks, and supports more hardware acceleration paths Tom's Hardware
Google News LLM Evaluation
Impact of agricultural industry transformation based on deep learning model evaluation and metaheuristic algorithms under dual carbon strategy Nature
OpenAI Evaluation Filter
Discover how Intercom built a scalable AI platform with 3 key lessons—from evaluations to architecture—to lead the future of customer support.
Google News LLM Evaluation
This new AI benchmark tests how much AI sucks up to you fanaticalfuturist.com
Google News LLM Evaluation
German Gov-Backed AI Benchmark Tracks Large Language Models in 200 Languages Slator
OpenAI Evaluation Filter
ChatGPT agent System Card: OpenAI’s agentic model unites research, browser automation, and code tools with safeguards under the Preparedness Framework.
More stories load automatically as you scroll.