evald.ai

METR Blog · September 27, 2026 07:00

Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals

Reilly Haskins, Rif A. Saurous, Nate Rush, Neev Parikh, and Beth Barnes lay out the claims that would need to hold for their per-action blocking monitor to be effective, the evidence for each claim, and the gaps that remain.

Mitchell Bryson AI Reliability Articles · September 27, 2026 00:00

AI products need to leave room for doubt - Mitchell Bryson

Wrong AI advice boosting confidence, GPT-6 Astra spotting IKEA assembly errors and Eclatira launching conversational video agents suggest that convincing assistance can outpace a user's ability to judge it. The editorial argues that builders should make...

MLCommons Evaluation Filter · September 24, 2026 14:50

MLPerf Training v6.1: First LLM Post-Training Benchmark

MLPerf Training v6.1 adds an agentic RL benchmark: teach a 397B-parameter model to repair real software. Reference run: 256 Blackwell Ultra GPUs.

Topics: Benchmarks

Entities: Benchmarks

MLCommons Evaluation Filter · September 23, 2026 14:11

MLCommons Joins EU-Funded AIRIS Project to Build and Benchmark Next-Generation Biomedical AI - MLCommons

MLCommons joins the EU-funded AIRIS consortium to develop open benchmarks for mechanism-informed generative AI in biomedical research.

Topics: Benchmarks

Entities: Benchmarks