Article List
Explore latest news, discover interesting content, and dive deep into topics that interest you
EleutherAI Second Retrospective: The long version
What we've been up to for the past year EleutherAI....
The View from 30,000 Feet: Preface to the Second EleutherAI…
(Some of) what we've been up to for the past year-and-a-half at EleutherAI....
Announcing GPT-NeoX-20B
Announcing GPT-NeoX-20B, a 20 billion parameter model trained in collaboration with CoreWeave....
A Preliminary Exploration into Factored Cognition with Lang…
We perform a series of experiments using GPT-3 with decomposition to perform complex toy tasks that it is otherwise unable to solve. The goal of these...
Multiple Choice Normalization in LM Evaluation
There are multiple ways of evaluating multiple choice tasks on autoregressive LMs like GPT-3/Neo/J. This post lays out the current prevalent normaliza...
Downstream Evaluations of Rotary Position Embeddings
A comparison of Rotary Position Embedding against GPT-style learned position embeddings....
What A Long, Strange Trip It's Been: EleutherAI One Year Re…
A look back at the first year of EleutherAI....
Why Release a Large Language Model?
We believe the creation and open source release of a large language model is a net good to AI safety. We explain why....
On the Sizes of OpenAI API Models
Using eval harness, we can deduce the sizes of OpenAI API models from their performance....
Evaluating Different Fewshot Description Prompts on GPT-3
We evaluate different fewshot prompts on GPT-3 to see how it changes performance....
Finetuning Models on Downstream Tasks
We tuned GPT-Neo on eval harness tasks to see how it would change its performance....
Activation Function Ablation
An ablation of activation functions in GPT-like autoregressive language models....
Rotary Embeddings: A Relative Revolution
Rotary Positional Embedding (RoPE) is a new type of position encoding that unifies absolute and relative approaches. We put it to the test....