Prompt Engineering

Write the prompt, add examples and reasoning, vote, and budget the context window. Python from the lessons.

What goes in the window#

Task	Code
Build a prompt from a pattern	`build_prompt("persona", <variables>)`
Build a multi-turn message list	`build_multi_turn(<pattern>, <turns>)`
Score one output	`score_response(<text>, <criteria>)`
Few-shot CoT prompt	`build_cot_prompt(question, examples, num_examples=3)`
Majority vote over samples	`self_consistency_solve(question, examples, client, model, n_samples=5)`
CoT, then vote, then ToT	`solve_with_escalation(question, examples, client, model)`
Budget the window	`ContextBudget(max_tokens=128000, generation_reserve=4000)`
Prune tools by intent	`select_tools(query, token_budget=2000)`

system, user = build_cot_prompt(question, examples)3 worked examples

answers = []

for _ in range(n_samples):n_samples=5

text = call_llm(client, model, system, user, temperature=0.7)

answer = extract_answer(text)reads "The answer is <n>"

if answer is not None:

answers.append(answer)

vote_counts = Counter(answers)

best_answer = vote_counts.most_common(1)[0][0]majority answer

confidence = vote_counts[best_answer] / len(answers)

if confidence >= 0.8 and single_answer == sc_answer:

...accept the vote

tree_of_thought_solve(question, client, model, breadth=3, depth=2)else

Setting	Default	Note
Temperature, extraction and code	0.0	Top-p 1.0. Also for prompt tests
Temperature, summarization	0.3	Top-p 0.9
Temperature, general Q&A	0.7	Top-p 0.95
Few-shot examples	3-5	One per label for many labels
`n_samples`	5	At temperature 0.7. N=3 is the minimum vote
Vote confidence to accept	0.8	Below it, escalate to Tree-of-Thought
`generation_reserve`	4000	Tokens kept free for the answer
System prompt size	under 500 tokens	For most tasks

Command	What it does
Zero-shot	Simple tasks. 1 call, no overhead
Few-shot	Format matching. 200-500 extra tokens
Zero-shot CoT	Add "Let's think step by step". 50-200 tokens
Few-shot CoT	Best single-call accuracy. 300-600 tokens
Self-consistency, N=5	High-stakes reasoning. 5x token cost
Tree-of-Thought	Search and planning. 10-40+ calls
ReAct	Knowledge-grounded reasoning. 3-10+ calls
Prompt chaining	Complex multi-step tasks. 2-5 calls

System message: identity, rules and constraints. Persists across turns.
User message: the task. Changes every turn.
Assistant prefill: a partial response such as { that sets the format.
Role: a specific role narrows the output distribution.
Constraints: negative "Do NOT", positive "Always", conditional "If X then Y".
Delimiters: XML tags such as <analysis> keep sections apart.
Clarity: state format, length, audience, include and exclude, one example.
Lost in the middle: 85-90% accuracy first or last, 60-70% in the middle.

Symptom	Cause	Fix
Generic, median-quality output	Vague task, generic role	Give a specific role, format, length, audience
Confident nonsense	Role too rare in training data	Use a less narrow role
Model follows one rule, drops another	Contradictory instructions	Audit the prompt for conflicts
Task quality drops	2,000 words of rules	Keep the system prompt under 500 tokens
User text overrides the rules	Prompt injection	Validate input, use delimiters, filter output
All votes identical	Samples at temperature 0.0	Sample at 0.7
Votes cannot be compared	No fixed answer format	End with "The answer is [number]"
Key fact ignored	Fact in the middle of a long context	Put it first or last, or repeat it at the end

Why does CoT help?: Each reasoning token is context for the next. It extends computation depth.
When does CoT hurt?: Simple recall and single-step classification. It adds 50-200 tokens.
CoT on a reasoning model?: No. It reasons internally. The phrase is redundant.
Why test at temperature 0 first?: It separates prompt quality from sampling noise.
Temperature and top-p together?: No. Use one. They interact unpredictably.

Pick examples similar to the input. Cover every label.
Put critical instructions first and the query last.
Drop retrieved chunks below a relevance threshold. Bad retrieval is worse than none.
Include only the tools the intent needs. Tool tokens drop 60-80%.
Summarize history past a threshold such as 5,000 tokens.
Put stable parts at the top and variable parts at the bottom for prompt caching.