Capstone Lesson 39: Instruction Tuning by Supervised Fine-Tuning
A pretrained base model can extend a sequence but cannot follow an instruction. Supervised fine-tuning is the smallest change that fixes this: feed the model paired examples of an instruction and a desired response, and train the body to predict the response tokens. The trick is that you only want the loss to count the response, not the instruction. This lesson builds an Alpaca-style SFT loop with a custom collate function that masks instruction tokens with ignoreindex=-100, trains on 200 instruction-response pairs, and evaluates on a held-out split using exact-match. Format paired instruction-response data into a single causal sequence with explicit boundary tokens. Build a collate function that masks instruction tokens so cross-entropy only counts response tokens. Train a tiny transformer body under the SFT objective and watch the eval metric move. Implement greedy and temperature-sampled generation that respects the response-start boundary. Compute held-out exact-match on generated completions. A base model trained on next-token prediction has no idea what an instruction is. Show it the string "What is the capital of France?" and it will continue the question or invent a new sentence. The model has the language but not the format contract. The SFT contract is a string template. Every training example becomes a single sequence with three regions: The boundary tokens are special tokens reserved at training time.…
Capstone Lesson 39: Instruction Tuning by Supervised Fine-Tuning: A pretrained base model can extend a sequence but cannot follow an instruction. Supervised…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.