evald.ai

METR Blog · October 28, 2025 09:47

Review of the Anthropic Summer 2025 Pilot Sabotage Risk Report

External review from METR of Anthropic's Summer 2025 Sabotage Risk Report

Safety Evals

Anthropic Safety Evals

OpenAI Evaluation Filter · October 27, 2025 10:00

Addendum to GPT-5 System Card: Sensitive conversations

This system card details GPT-5’s improvements in handling sensitive conversations, including new benchmarks for emotional reliance, mental health, and jailbreak resistance.

Benchmarks

Benchmarks

Google DeepMind Evaluation Filter · October 23, 2025 23:44

Google DeepMind strengthens the Frontier Safety Framework

Today, we’re publishing the third iteration of our Frontier Safety Framework (FSF) — our most comprehensive approach yet to identifying and mitigating severe risks from advanced AI models. This updat…

Google Google DeepMind

Google DeepMind Evaluation Filter · October 23, 2025 18:52

Rethinking how we measure AI intelligence

Kaggle Game Arena is a new platform where AI models compete head-to-head in complex strategic games.

METR Blog · October 23, 2025 07:00

Summary of our gpt-oss methodology review

Details on external recommendations from METR for gpt-oss Preparedness experiments and follow-up from OpenAI.

Safety Evals

Safety Evals OpenAI

METR Blog · October 14, 2025 07:00

MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity

MALT (Manually-reviewed Agentic Labeled Transcripts) is a dataset of natural and prompted examples of behaviors that threaten evaluation integrity (like generalized reward hacking or sandbagging).

LLM Evaluation

LLM Evaluation

METR Blog · October 06, 2025 07:00

Early Results on Monitorability in QA Settings

Vincent Cheng, Thomas Kwa, and Neev Parikh share research on how AI agents can hide secondary task-solving from monitors, finding that harder tasks are more detectable and small models can learn to evade larger monitors.

Hacker News LLM Evaluation · October 05, 2025 15:55

Understanding the 4 Main Approaches to LLM Evaluation (From Scratch)

Multiple-Choice Benchmarks, Verifiers, Leaderboards, and LLM Judges with Code Examples

Benchmarks LLM Evaluation

Benchmarks LLM Evaluation

Hugging Face Evaluation Filter · October 01, 2025 00:00

Introducing RTEB: A New Standard for Retrieval Evaluation

We’re on a journey to advance and democratize artificial intelligence through open source and open science.