evald.ai

MLCommons Evaluation Filter · March 10, 2026 14:17

Bringing Text-to-Video to MLPerf Inference v6.0 - MLCommons

MLCommons introduces the new Text-to-Video benchmark in MLPerf Inference v6.0, based on the Wan2.2-T2V-A14B-Diffusers model and validated using the VBench framework. Learn about the key architectural decisions, including the adoption of the SingleStream...

Topics: Benchmarks

Entities: Benchmarks

METR Blog · March 10, 2026 07:00

Many SWE-bench-Passing PRs Would Not Be Merged into Main

We find that roughly half of test-passing SWE-bench Verified PRs written by recent AI agents would not be merged into main by repo maintainers. A naive interpretation of benchmark scores may lead one to overestimate how useful agents are without more...

Topics: Benchmarks

Entities: Benchmarks

Mitchell Bryson AI Reliability Articles · March 10, 2026 00:00

AI finally learns to secure the code it writes - Mitchell Bryson

OpenAI shipping Codex Security, Anthropic's Claude finding 22 CVEs in Firefox in two weeks, and Microsoft treating AI agents as governed security principals all point to the same inflection: the industry is racing to close the security gap that AI coding...

Entities: AnthropicClaudeOpenAIMicrosoft

OpenAI Evaluation Filter · March 06, 2026 00:00

How Balyasny Asset Management built an AI research engine

By combining rigorous model evaluation, full-platform use of OpenAI, and agent workflows, Balyasny is reinventing investment research.

Topics: LLM Evaluation

Entities: LLM EvaluationOpenAI

Mitchell Bryson AI Reliability Articles · March 04, 2026 00:00

The security bill is coming due for AI's agent era - Mitchell Bryson

AI agents are gaining deeper access to enterprise systems and developer environments faster than anyone is securing them. Three stories from a single news cycle show the attack surface widening in real time.

METR Blog · March 03, 2026 08:00

Observations from two CLI game reimplementation runs with Opus 4.6

Nikola Jurkovic describes observations from tasking Opus 4.6 with reimplementing Slay the Spire and Balatro in the CLI.

Mitchell Bryson AI Reliability Articles · March 03, 2026 00:00

The Pentagon values auction: AI safety gets its market test - Mitchell Bryson

OpenAI amends its Pentagon deal after Altman admits it looked 'opportunistic and sloppy', while Claude surges to number one on the App Store and hundreds of employees publicly back Anthropic's stance.

Topics: Safety Evals

Entities: AnthropicSafety EvalsClaudeOpenAI