Few-Shot, Chain-of-Thought, Tree-of-Thought
Telling a model what to do is prompting. Showing it how to think is engineering. The gap between 78% and 91% accuracy on the same model, same task, same data is not a better model. It is a better reasoning strategy. Implement few-shot prompting by selecting and formatting example demonstrations that maximize task accuracy. Apply chain-of-thought (CoT) reasoning to improve accuracy on multi-step problems like math word problems. Build a tree-of-thought prompt that explores multiple reasoning paths and selects the best one. Measure the accuracy improvement from zero-shot vs few-shot vs CoT on a standard benchmark. You build a math tutoring app. Your prompt says: "Solve this word problem." GPT-5 gets it right 94% of the time on GSM8K, the standard grade-school math benchmark. You think you already peaked. You do not — chain-of-thought still adds 3-4 points. Add five words -- "Let's think step by step" -- and accuracy jumps to 91%. Add a few worked examples and it reaches 95%. Same model. Same temperature. Same API cost. The only difference is that you gave the model scratch paper. This is not a hack. It is how reasoning works. Humans do not solve multi-step problems in one mental leap. Neither do transformers. When you force a model to generate intermediate tokens, those tokens become part of the context for…
Few-Shot, Chain-of-Thought, Tree-of-Thought: Telling a model what to do is prompting. Showing it how to think is engineering. The gap between 78% and 91%…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.