To turn the magic into machinery
Most people treat a language model like a light switch. You type something in, an answer comes out, and everything in between stays a mystery. cutaway opens that box. You train a real model on a small body of text, then look inside it at every step. By the end the thing that felt like magic feels like machinery, because you watched it run with your own eyes.
The chip on the right stands for roughly 10 million numbers, the size of the model you explore in the later lessons. The model you train live in Lesson 2 is smaller still, around 800,000, so it can finish learning in minutes while you watch. Same design, less of it.
Start at the top
Four things, by watching the machine run
- How it reads. Your text is not letters to the model, it is numbers. You watch a line become the exact IDs it actually works on.
- How it learns. It begins by writing pure nonsense, and a few thousand tiny corrections later it writes like Shakespeare. You watch the entire climb happen live.
- What it pays attention to. When it guesses the next letter, it leans harder on some earlier letters than others. You see exactly which ones, and why that is the heart of how these models work.
- How it writes. One dial decides whether it plays it safe or takes wild risks. You turn that dial yourself and watch the writing change in front of you.
Standing on Karpathy's shoulders
This follows a path Andrej Karpathy laid down. In his talk Let's build GPT: from scratch, in code, spelled out, he builds a small character-level model and trains it on a tiny slice of Shakespeare. That is the exact setup running under these lessons. The difference: his version is code on a screen that you watch him write, and this one you operate yourself, in plain English. When you want the deeper, more technical version, his free course is where to go next.
- The talk: Let's build GPT, from scratch · the 2-hour walkthrough cutaway is built on
- The course: Neural Networks, Zero to Hero · his full free series, for going deeper
- The code: nanoGPT · the original repository (since carried forward into his nanochat); cutaway uses its proven settings
Two papers sit underneath all of it. You do not need to read them to do any of this. They are here because this is where the ideas actually came from.
- Attention Is All You Need (2017) · the paper that introduced the Transformer, the design every modern model is built on. The attention you meet in Lesson 3 comes straight from here.
- Language Models are Few-Shot Learners (2020) · OpenAI's GPT-3 paper. It showed what happens when you take the same Transformer design and scale it to 175 billion numbers. The model in this guide shares that same core architecture, just tiny.