Community feed
A focused stream of recent stories from the sources curated for this community. Latest: ChatGPT agent System Card, How Does Time Horizon Vary Across Domains?, and Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. Page 62.
OpenAI Evaluation Filter
ChatGPT agent System Card: OpenAI’s agentic model unites research, browser automation, and code tools with safeguards under the Preparedness Framework.
Topics: Safety Evals
Entities: Safety EvalsOpenAIChatGPT
METR Blog
We build on our time-horizon work and analyze 9 benchmarks for scientific reasoning, math, robotics, computer use, and self-driving in terms of time-horizon trends; we observe generally similar rates of improvement to the 7-month doubling time in our...
Topics: Benchmarks
Entities: Benchmarks
METR Blog
We conduct a randomized controlled trial to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer...
Google News LLM Evaluation
Elon Musk’s xAI sets AI benchmark records with new reasoning-optimized Grok 4 model SiliconANGLE
Topics: Benchmarks
Entities: BenchmarksxAI
Google News LLM Evaluation
Former Intel CEO’s New AI Benchmark Focuses on Human Flourishing The New Stack
Topics: Benchmarks
Entities: Benchmarks
Google News LLM Evaluation
Effective cross-lingual LLM evaluation with Amazon Bedrock Amazon Web Services (AWS)
Topics: LLM Evaluation
Entities: LLM Evaluation
Google News LLM Evaluation
Comparing traditional natural language processing and large language models for mental health status classification: a multi-model evaluation Nature
Topics: LLM Evaluation
Entities: LLM Evaluation
Hugging Face Evaluation Filter
A Blog post by Technology Innovation Institute on Hugging Face
Google News LLM Evaluation
Getting Started with MLFlow for LLM Evaluation MarkTechPost
Topics: LLM Evaluation
Entities: LLM Evaluation
METR Blog
METR conducted preliminary evaluations of a set of DeepSeek and Qwen models. We found that the level of autonomous capabilities of mid-2025 DeepSeek models is similar to the level of capabilities of frontier models from late 2024.
METR Blog
Current views on information relevant for visibility into frontier AI risk.
Topics: Safety Evals
Entities: Safety Evals
