OpenAI Evaluation Filter
Consistency models are a nascent family of generative models that can sample high quality data in one step without the need for adversarial training.
Topics: LLM Evaluation
Entities: LLM Evaluation
OpenAI Evaluation Filter
We’re developing a blueprint for evaluating the risk that a large language model (LLM) could aid someone in creating a biological threat. In an evaluation involving both biology experts and students, we found that GPT-4 provides at most a mild uplift in...
Topics: Safety Evals
Entities: Safety Evals
OpenAI Evaluation Filter
To support the safety of highly-capable AI systems, we are developing our approach to catastrophic risk preparedness, including building a Preparedness team and launching a challenge.
Topics: Safety Evals
Entities: Safety Evals
OpenAI Evaluation Filter
We’ve created GPT-4, the latest milestone in OpenAI’s effort in scaling up deep learning. GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits...
Topics: Benchmarks
Entities: BenchmarksOpenAI
OpenAI Evaluation Filter
Our latest research finds we can improve language model behavior with respect to specific behavioral values by fine-tuning on a small, curated dataset.
OpenAI Evaluation Filter
We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized,...
Topics: Benchmarks
Entities: Benchmarks
OpenAI Evaluation Filter
We’re excited to announce that OpenAI is co-organizing two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind, using Procgen Benchmark and MineRL.
Topics: Benchmarks
Entities: BenchmarksOpenAI
OpenAI Evaluation Filter
We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and...
Topics: LLM Evaluation
Entities: LLM Evaluation
OpenAI Evaluation Filter
We’re releasing Procgen Benchmark, 16 simple-to-use procedurally-generated environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills.
Topics: Benchmarks
Entities: Benchmarks
OpenAI Evaluation Filter
We’ve made progress towards stable and scalable training of energy-based models (EBMs) resulting in better sample quality and generalization ability than existing models. Generation in EBMs spends more compute to continually refine its answers and doing so...
Topics: LLM Evaluation
Entities: LLM Evaluation
OpenAI Evaluation Filter
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question...
Topics: Benchmarks
Entities: Benchmarks
OpenAI Evaluation Filter
Yesterday, OpenAI Five won a best-of-three against a team of 99.95th percentile Dota players: Blitz, Cap, Fogged, Merlini, and MoonMeander—four of whom have played Dota professionally—in front of a live audience and 100,000 concurrent livestream viewers.
Topics: Benchmarks
Entities: BenchmarksOpenAI