Skip to slide
Chapter 2 · Scaling Laws and Chinchilla, How Big Should a Model Be?
18 / 74

CHAPTER 02 · Scaling Laws and Chinchilla, How Big Should a Model Be? · 5 / 5

The one-sentence takeaway

Model quality improves in a smooth, predictable way as you add parameters, data, and compute, and the smartest way to spend a compute budget is to grow the model and its training data together, not to just make the model enormous.

Next: Chapter 3, Alignment with RLHF and DPO, where a raw text predictor learns to become a helpful assistant.

← → arrow keys work too