evald.ai

METR Blog · March 10, 2026 07:00

Many SWE-bench-Passing PRs Would Not Be Merged into Main

We find that roughly half of test-passing SWE-bench Verified PRs written by recent AI agents would not be merged into main by repo maintainers. A naive interpretation of benchmark scores may lead one to overestimate how useful agents are without more...

Benchmarks

Benchmarks

Mitchell Bryson AI Reliability Articles · March 10, 2026 00:00

AI finally learns to secure the code it writes - Mitchell Bryson

OpenAI shipping Codex Security, Anthropic's Claude finding 22 CVEs in Firefox in two weeks, and Microsoft treating AI agents as governed security principals all point to the same inflection: the industry is racing to close the security gap that AI coding...

Anthropic Claude OpenAI Microsoft

OpenAI Evaluation Filter · March 06, 2026 00:00

How Balyasny Asset Management built an AI research engine

By combining rigorous model evaluation, full-platform use of OpenAI, and agent workflows, Balyasny is reinventing investment research.

LLM Evaluation

LLM Evaluation OpenAI

Mitchell Bryson AI Reliability Articles · March 04, 2026 00:00

The security bill is coming due for AI's agent era - Mitchell Bryson

AI agents are gaining deeper access to enterprise systems and developer environments faster than anyone is securing them. Three stories from a single news cycle show the attack surface widening in real time.

METR Blog · March 03, 2026 08:00

Observations from two CLI game reimplementation runs with Opus 4.6

Nikola Jurkovic describes observations from tasking Opus 4.6 with reimplementing Slay the Spire and Balatro in the CLI.

Mitchell Bryson AI Reliability Articles · March 03, 2026 00:00

The Pentagon values auction: AI safety gets its market test - Mitchell Bryson

OpenAI amends its Pentagon deal after Altman admits it looked 'opportunistic and sloppy', while Claude surges to number one on the App Store and hundreds of employees publicly back Anthropic's stance.

Safety Evals

Anthropic Safety Evals Claude OpenAI

Mitchell Bryson AI Reliability Articles · March 02, 2026 00:00

The agent security reckoning nobody is ready for - Mitchell Bryson

Three separate security disclosures this week exposed a pattern: we are deploying agentic AI infrastructure faster than we can secure it, from MCP servers to coding assistants.