Why a raw model is not enough

The model has soaked up enormous knowledge, but it has no sense of what makes a response good. Good is subjective and human. It involves being helpful, honest, harmless, following instructions, and matching the tone people expect. You cannot write down those rules as a formula. But humans can recognize a good answer when they see one. The trick is to teach the model from human judgment.