evald.ai

METR Blog · July 31, 2023 20:00

New report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

We have just released our first public report. It introduces methodology for assessing the capacity of LLM agents to acquire resources, create copies of themselves, and adapt to novel challenges they encounter in the wild.

Hugging Face Evaluation Filter · June 23, 2023 00:00

What's going on with the Open LLM Leaderboard?

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Benchmarks

Benchmarks

METR Blog · March 17, 2023 15:22

Update on ARC's recent eval efforts

More information about ARC's evaluations of GPT-4 and Claude

LLM Evaluation

Claude LLM Evaluation

OpenAI Evaluation Filter · March 14, 2023 07:00

GPT-4

We’ve created GPT-4, the latest milestone in OpenAI’s effort in scaling up deep learning. GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits...

Benchmarks

Benchmarks OpenAI

Hugging Face Evaluation Filter · October 19, 2022 00:00

MTEB: Massive Text Embedding Benchmark

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Benchmarks

Benchmarks

Hugging Face Evaluation Filter · June 28, 2022 00:00

Announcing Evaluation on the Hub

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

OpenAI Evaluation Filter · June 10, 2021 07:00

Improving language model behavior by training on a curated dataset

Our latest research finds we can improve language model behavior with respect to specific behavioral values by fine-tuning on a small, curated dataset.

OpenAI Evaluation Filter · January 05, 2021 08:00

CLIP: Connecting text and images

We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized,...

Benchmarks

Benchmarks

OpenAI Evaluation Filter · June 20, 2020 07:00

Procgen and MineRL Competitions

We’re excited to announce that OpenAI is co-organizing two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind, using Procgen Benchmark and MineRL.

Benchmarks

Benchmarks OpenAI

OpenAI Evaluation Filter · June 17, 2020 07:00

Image GPT

We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and...

LLM Evaluation

LLM Evaluation

OpenAI Evaluation Filter · December 03, 2019 08:00

Procgen Benchmark

We’re releasing Procgen Benchmark, 16 simple-to-use procedurally-generated environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills.

Benchmarks

Benchmarks