Community feed
A focused stream of recent stories from the sources curated for this community. Latest: Details about METR's preliminary evaluation of DeepSeek-R1, METR’s GPT-4.5 pre-deployment evaluations, and Deep research System Card. Page 21.
METR Blog
We evaluated DeepSeek-R1 for dangerous autonomous capabilities and found no evidence of dangerous capabilities beyond those of existing models such as Claude 3.5 Sonnet and GPT-4o. Interestingly, we found that it did not do substantially better than...
METR Blog
Additional details about our evaluations of GPT-4.5, and some discussion about the limitations of pre-deployment evaluations and current evaluation methodologies.
OpenAI Evaluation Filter
This report outlines the safety work carried out prior to releasing deep research including external red teaming, frontier risk evaluations according to our Preparedness Framework, and an overview of the mitigations we built in to address key risk areas.
OpenAI Evaluation Filter
Can frontier LLMs earn $1 million from real-world freelance software engineering?
METR Blog
We measured the performance of frontier models at writing GPU kernels. With a small amount of scaffolding, we found that the best model can provide an average speedup on KernelBench of 1.8x.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Google News LLM Evaluation
LLM-as-a-judge on Amazon Bedrock Model Evaluation | Amazon Web Services Amazon Web Services (AWS)
METR Blog
We evaluated DeepSeek-V3 for dangerous autonomous capabilities and found no evidence of dangerous capabilities beyond those of existing models such as Claude 3.5 Sonnet and GPT-4o. We also confirmed that its performance on GPQA is not due to training data...
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
OpenAI Evaluation Filter
This report outlines the safety work carried out for the OpenAI o3-mini model, including safety evaluations, external red teaming, and Preparedness Framework evaluations.
More stories load automatically as you scroll.