Community feed
A focused stream of recent stories from the sources curated for this community. Latest: Details about METR's preliminary evaluation of OpenAI o1-preview, ERROR: The request could not be satisfied, and A review of model evaluation metrics for machine learning in genetics and genomics - Frontiers. Page 26.
METR Blog
We measured the performance of OpenAI's o1-mini and o1-preview models on our autonomy and AI R&D task suites, and found they did not exceed the capabilities of the best existing public model we've evaluated, though we could not confidently upper-bound...
METR Blog
Suggestions for expanded guidance on capability elicitation and robust model safeguards in the U.S. AI Safety Institute’s draft document “Managing Misuse Risk for Dual-Use Foundation Models” (NIST AI 800-1).
Google News LLM Evaluation
A review of model evaluation metrics for machine learning in genetics and genomics Frontiers
METR Blog
Vivaria is METR's tool for running evaluations and conducting agent elicitation research. Vivaria is a web application with which users can interact using a web UI and a command-line interface.
METR Blog
We measured the performance of GPT-4o given a simple agent scaffolding on 77 tasks across 30 task families testing autonomous capabilities.
METR Blog
More tasks, human baselines, and preliminary results for GPT-4 and Claude.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
OpenAI Evaluation Filter
OpenAI and Los Alamos National Laboratory are working to develop safety evaluations to assess and measure biological capabilities and risks associated with frontier models.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
OpenAI Evaluation Filter
Consistency models are a nascent family of generative models that can sample high quality data in one step without the need for adversarial training.
Hugging Face Evaluation Filter
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
More stories load automatically as you scroll.