Instruction Tuning (SFT)
A base model predicts the next token. That's it. It doesn't follow instructions, answer questions, or refuse harmful requests. SFT is the bridge between a token predictor and a useful assistant. Every model you've ever talked to -- Claude, GPT, Llama Chat -- went through this step. Implement supervised fine-tuning (SFT) that converts a base language model into an instruction-following assistant. Format training data using chat templates with system, user, and assistant roles, and mask loss on non-assistant tokens. Explain why SFT is necessary: base models continue text rather than answer questions. Evaluate SFT quality by comparing base model vs fine-tuned model responses on a held-out instruction set. You trained a model in Lesson 04. It can predict the next token given a sequence. Feed it "The transformer architecture" and it might continue with "has revolutionized natural language processing." That's impressive for a next-token predictor. Now try this: feed it "What is the capital of France?" A base model doesn't answer "Paris." It continues the pattern. It might produce "What is the capital of Germany? What is the capital of Spain?" because it learned from documents that contain lists of questions. Or it might produce "is a question that many people ask" because that's a plausible next-token continuation. The model has no concept of answering. It only knows continuing. This…
Instruction Tuning (SFT): A base model predicts the next token. That's it. It doesn't follow instructions, answer questions, or refuse harmful requests. SFT…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.