Safety Evals

Mitchell Bryson AI Reliability Articles · August 10, 2026 00:00

The permission prompt is giving way to the blast wall - Mitchell Bryson

Claude Code is making auto mode the default, Docker is shipping disposable sandboxes for agents, Wardline is auto-blocking compromised agents, and TechCrunch is warning that AI safety tests can become safety risks. The shared story is that builders are...

Topics: Safety Evals

Entities: Safety EvalsClaudeClaude Code

Mitchell Bryson AI Reliability Articles · July 31, 2026 00:00

The AI test environment is becoming the new production risk - Mitchell Bryson

Anthropic saying its own AI models breached three companies, TechCrunch’s analysis of the Hugging Face breach, Google saying AI fixed more Chrome bugs in June than over the past two years, and Okta buying Permiso for about $200M all point to the same shift:...

Topics: Safety Evals

Entities: AnthropicSafety EvalsGoogle

Mitchell Bryson AI Reliability Articles · July 29, 2026 00:00

The security layer is becoming the agent stack - Mitchell Bryson

Today’s headlines suggest that the next AI platform battle is less about making agents more autonomous than making them safe enough to admit into real companies. Cyera’s $1B Oasis deal, Spur’s $200M raise, Runlayer’s MCP lawsuit and Visa’s open-sourced...

Topics: Safety Evals

Entities: Safety EvalsMythos

METR Blog · May 08, 2026 07:00

Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)

External review from METR of the "Risks from automated R&D" section in Anthropic's February 2026 Risk Report

Topics: Safety Evals

Entities: AnthropicSafety Evals

MLCommons Evaluation Filter · April 20, 2026 22:10

Fresh Benchmarks, Reliable Scores: Introducing Continuous Prompt Stewardship for AI Risk Evaluation - MLCommons

MLCommons introduces Continuous Prompt Stewardship to keep the AILuminate AI safety benchmark fresh and reliable as frontier models evolve.

Topics: BenchmarksSafety Evals

Entities: Safety EvalsBenchmarks

Google DeepMind Evaluation Filter · March 25, 2026 16:46

Protecting People from Harmful Manipulation

Google DeepMind releases new findings and an evaluation framework to measure AI's potential for harmful manipulation in areas like finance and health, with the goal of enhancing AI safety.

Topics: Safety EvalsTesting Tools

Entities: Safety EvalsTesting ToolsGoogleGoogle DeepMind

MLCommons Evaluation Filter · March 13, 2026 16:57

Global Standards, Local Ground Truths: Piloting Multilingual, Multimodal AI Safety Understanding in APAC - MLCommons

MLCommons is developing the AILuminate Culturally-Specific Multimodal Benchmark to close the AI performance and representation gap across APAC cultures, languages, and real-world use cases.

Topics: BenchmarksSafety Evals

Entities: Safety EvalsBenchmarks