How Language Models Work

The world has changed considerably since our last "think week" five months ago—and so has Every. We’ve added new business units, launched new products, and brought on new teammates. So we’re taking this week to come up with new ideas and products that can help us improve how we do our work and, more importantly, your experience as a member of our community. In the meantime, we’re re-upping four pieces by Dan Shipper that cover basic, powerful questions about AI. (Dan hasn’t been publishing at his regular cadence because he’s working on a longer piece. Look out for that in Q2.) First up is his piece from last May that explains how language models work.—Kate Lee

Was this newsletter forwarded to you? Sign up to get it in your inbox.

If we want to wield language models in our work and still call the results creative, we’ll have to understand how they work—at least at a high level. There are plenty of excellent guides about the internal mechanisms of language models, but they’re all quite technical. (One notable exception is Nir Zicherman’s piece in Every about LLMs as food.) That’s a shame because there are only a handful of simple ideas you need to understand in order to get a basic understanding of what’s going on under the hood.

I decided to write those ideas out for you—and for me—in as jargon-free a way as possible. The explanation below is deliberately simplified, but it should give you a good intuition for how things work. (If you want to go beyond the simplifications, I suggest putting this article into ChatGPT or Claude.)

Ready? Let’s go.

Become a paid subscriber to Every to unlock this piece and learn about:

How language models transform words into rich contextual "Super Words"
The mathematical mapping of meaning in continuous space
Why context creates predictive power
The art of wielding LLMs as creative tools

Upgrade to paid

Click here to read the full post

Want the full text of all articles in RSS? Become a subscriber, or learn more.