METR Blog
An update on our general capability evaluations
More tasks, human baselines, and preliminary results for GPT-4 and Claude.
Entities: Claude
Community feed
evald.ai
A focused stream of recent stories from the sources curated for this community. Latest: An update on our general capability evaluations, LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?, and OpenAI and Los Alamos National Laboratory announce research partnership. Page 68.
Search
Hugging Face Evaluation Filter
LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
OpenAI Evaluation Filter
OpenAI and Los Alamos National Laboratory announce research partnership
OpenAI and Los Alamos National Laboratory are working to develop safety evaluations to assess and measure biological capabilities and risks associated with frontier models.
Entities: OpenAI
Hugging Face Evaluation Filter
Our Transformers Code Agent beats the GAIA benchmark 🏅
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
Hugging Face Evaluation Filter
Ethics and Society Newsletter #6: Building Better AI: The Importance of Data Quality
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: LLM Evaluation
Entities: LLM Evaluation
OpenAI Evaluation Filter
Improved Techniques for Training Consistency Models
Consistency models are a nascent family of generative models that can sample high quality data in one step without the need for adversarial training.
Topics: LLM Evaluation
Entities: LLM Evaluation
Hugging Face Evaluation Filter
Launching the Artificial Analysis Text to Image Leaderboard & Arena
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
METR Blog
ERROR: The request could not be satisfied
Comments on NIST’s draft document “AI Risk Management Framework: Generative AI Profile.”
Topics: Safety Evals
Entities: Safety Evals
Hugging Face Evaluation Filter
CyberSecEval 2 - A Comprehensive Evaluation Framework for Cybersecurity Risks and Capabilities of Large Language Models
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Testing Tools
Entities: Testing Tools
METR Blog
ML Engineers Needed for New AI R&D Evals Project
METR is hiring ML engineers and researchers.
Hugging Face Evaluation Filter
Introducing the Open Arabic LLM Leaderboard
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks
Hugging Face Evaluation Filter
Introducing the Open Leaderboard for Hebrew LLMs!
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics: Benchmarks
Entities: Benchmarks