Company
OpenAI Evaluation Filter
ChatGPT agent System Card: OpenAI’s agentic model unites research, browser automation, and code tools with safeguards under the Preparedness Framework.
Topics: Safety Evals
Entities: Safety EvalsOpenAIChatGPT
METR Blog
METR conducted a preliminary evaluation of OpenAI's o3 and o4-mini. The two models displayed higher autonomous capabilities than other public models tested, and o3 appears somewhat prone to "reward hacking".
Entities: OpenAI
OpenAI Evaluation Filter
This report outlines the safety work carried out for the OpenAI o3-mini model, including safety evaluations, external red teaming, and Preparedness Framework evaluations.
Topics: Safety Evals
Entities: Safety EvalsOpenAI
OpenAI Evaluation Filter
Drawing from OpenAI’s established safety frameworks, this document highlights our multi-layered approach, including model and product mitigations we’ve implemented to protect against prompt engineering and jailbreaks, protect privacy and security, as well...
Entities: OpenAI
OpenAI Evaluation Filter
This report outlines the safety work carried out prior to releasing OpenAI o1 and o1-mini, including external red teaming and frontier risk evaluations according to our Preparedness Framework.
Topics: Safety Evals
Entities: Safety EvalsOpenAI
METR Blog
We’re releasing RE-Bench, a new benchmark for measuring the performance of humans and frontier model agents on ML research engineering tasks. We also share data from 71 human expert attempts and results for Anthropic’s Claude 3.5 Sonnet and OpenAI’s...
Topics: Benchmarks
Entities: AnthropicClaudeBenchmarksOpenAI
METR Blog
We measured the performance of OpenAI's o1-mini and o1-preview models on our autonomy and AI R&D task suites, and found they did not exceed the capabilities of the best existing public model we've evaluated, though we could not confidently upper-bound their...
Entities: OpenAI
OpenAI Evaluation Filter
OpenAI and Los Alamos National Laboratory are working to develop safety evaluations to assess and measure biological capabilities and risks associated with frontier models.
Entities: OpenAI
OpenAI Evaluation Filter
We’ve created GPT-4, the latest milestone in OpenAI’s effort in scaling up deep learning. GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits...
Topics: Benchmarks
Entities: BenchmarksOpenAI
OpenAI Evaluation Filter
We’re excited to announce that OpenAI is co-organizing two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind, using Procgen Benchmark and MineRL.
Topics: Benchmarks
Entities: BenchmarksOpenAI
OpenAI Evaluation Filter
Yesterday, OpenAI Five won a best-of-three against a team of 99.95th percentile Dota players: Blitz, Cap, Fogged, Merlini, and MoonMeander—four of whom have played Dota professionally—in front of a live audience and 100,000 concurrent livestream viewers.
Topics: Benchmarks
Entities: BenchmarksOpenAI
OpenAI Evaluation Filter
The OpenAI Five Benchmark match is now over!
Topics: Benchmarks
Entities: BenchmarksOpenAI