Step 1 · Make a guess first
Before you press start
As it trains, one line tracks how wrong the model is when it guesses the next character of text. Lower means better. Zero would mean perfect.
Call it before you see it. What does that line do over time?
See the real code, line by line
Training Loop From: teach_server.py · run_training()
Why it mattersThese five lines are the entire act of learning. Every model, including the ones you use daily, learns by running this.
Key conceptGuess the next character, measure how wrong, nudge the numbers to be less wrong, and repeat thousands of times.
What to look for"loss" is the number we drive down; "loss.backward" and "opt.step" are the nudge that makes the next guess better.
for step in range(max_steps + 1):
xb, yb = get_batch(train_data, cfg.block_size, batch_size, device)
_, loss = model(xb, yb)
opt.zero_grad(set_to_none=True)
loss.backward()
opt.step()
Hover or tap any line above. Its plain-English explanation appears right here.
Five lines. Bigger models and larger datasets run this same loop faster and on more text. The loop itself does not change.