AI Engineering from Scratch
Cheat sheets
One poster for each topic. Each poster holds the tasks, commands, code, diagrams, defaults and fixes that you use while you work. Every fact comes from a lesson or a manual of this course.
Topics
- A2A ProtocolDiscover an agent, send it work, follow the task, read its errors. Pinned to A2A 1.0.1.
- Agent Harness EngineeringThe code around the model. Loop contract, hooks, budgets, tools, gates, sandbox, traces and workbench files.
- AI Safety and Red TeamingName the attack, detect it, measure refusals, gate the output and audit alignment. Phase 18 and 19 lessons.
- Autonomous Agents and PermissionsRun an agent unattended with limits. Permission modes, budgets, kill switches, durable execution, approvals, rollback.
- Classical Machine LearningFit, validate, tune and fix regression, classification and clustering models with scikit-learn.
- Computer VisionPreprocess, convolve, classify, fine-tune, detect, segment and patchify images. PyTorch and NumPy from the lessons.
- Training Neural NetworksPick the optimizer, init, normalization, regularization and schedule. Then debug the run when the loss misbehaves.
- Diffusion and Generative ModelsTrain and sample a VAE, a GAN, DDPM, latent diffusion and flow matching, then measure them. Toy Python from the lessons.
- Docker Sandboxes for AgentsRun an agent in a microVM, control its network and keys, get its commits back. sbx 0.47.0, docker-agent 1.149.0.
- Embeddings and Vector SearchEncode text, pick a metric, size the vectors, choose an index and fix bad recall. Python, NumPy and FAISS.
- Inference Optimization: KV Cache, Quantization, BatchingSize the KV cache, batch continuously, pick a quantization format, and measure what users feel.
- Evaluating LLMsSpecify tasks, score them, judge with a rubric, calibrate, rank with intervals and test long context.
- Fine-Tuning with LoRAAdd LoRA adapters, load a 4-bit base for QLoRA, mask SFT labels, train, merge and debug. PyTorch, PEFT and bitsandbytes.
- Math for Machine LearningGradients, probability, information, optimizers, decompositions and float safety. NumPy, SciPy and PyTorch.
- Model Context Protocol (MCP)Build, call and serve stateless MCP 2026-07-28 servers over stdio and Streamable HTTP.
- Multi-Agent SystemsAgents, handoffs, shared state and orchestrators. Pick a pattern, vote, checkpoint and fix the known failures.
- Prompt EngineeringWrite the prompt, add examples and reasoning, vote, and budget the context window. Python from the lessons.
- PyTorch EssentialsTensors, autograd, nn.Module, the training loop, DataLoader and devices, from the lessons.
- Retrieval-Augmented Generation (RAG)Chunk, index, retrieve, fuse, rerank, generate with citations and grade each stage. Python from the lessons.
- Reinforcement LearningMDPs, value methods, policy gradients, PPO, RLHF and GRPO. Formulas, update lines and diagnostics from Phase 9.
- Reverse Engineering with reaRead a compiled program and prove what you found. Pinned to rea-agents 6.5.0 with Ghidra 12.1.4.
- LLM Serving with SGLangStart, query, size, measure and fix an SGLang server. Pinned to SGLang 0.5.22.
- Tokenization and BPETrain, encode and decode a byte-level BPE tokenizer, then slice the id stream into training windows.
- Tool Calling and Structured OutputsDeclare tools, read calls, return results, run calls in parallel, stream arguments, and force JSON that validates.
- Transformers and AttentionCompute attention, mask it, split heads, add positions and stack the block. PyTorch and NumPy from the lessons.
By phase
- Phase 0: Setup & ToolingPyTorch Essentials
- Phase 1: Math FoundationsEmbeddings and Vector Search, Math for Machine Learning
- Phase 2: ML FundamentalsClassical Machine Learning, Embeddings and Vector Search
- Phase 3: Deep Learning CoreTraining Neural Networks, PyTorch Essentials
- Phase 4: Computer VisionComputer Vision, Embeddings and Vector Search
- Phase 5: NLP, Foundations to AdvancedEmbeddings and Vector Search, Evaluating LLMs, Retrieval-Augmented Generation (RAG), Tokenization and BPE, Transformers and Attention
- Phase 7: Transformers Deep DiveInference Optimization: KV Cache, Quantization, Batching, Transformers and Attention
- Phase 8: Generative AIDiffusion and Generative Models
- Phase 9: Reinforcement LearningReinforcement Learning
- Phase 10: LLMs from ScratchInference Optimization: KV Cache, Quantization, Batching, Evaluating LLMs, Fine-Tuning with LoRA, Tokenization and BPE, Transformers and Attention
- Phase 11: LLM EngineeringEmbeddings and Vector Search, Evaluating LLMs, Fine-Tuning with LoRA, Prompt Engineering, Retrieval-Augmented Generation (RAG), Tool Calling and Structured Outputs
- Phase 13: Tools & ProtocolsModel Context Protocol (MCP), Reverse Engineering with rea, Tool Calling and Structured Outputs
- Phase 14: Agent EngineeringAgent Harness Engineering
- Phase 15: Autonomous SystemsAgent Harness Engineering, Autonomous Agents and Permissions
- Phase 16: Multi-Agent & SwarmsMulti-Agent Systems
- Phase 17: Infrastructure & ProductionInference Optimization: KV Cache, Quantization, Batching
- Phase 18: Ethics, Safety & AlignmentAI Safety and Red Teaming
- Phase 19: Capstone ProjectsAgent Harness Engineering, AI Safety and Red Teaming, Embeddings and Vector Search, Evaluating LLMs, Fine-Tuning with LoRA, Retrieval-Augmented Generation (RAG), Tokenization and BPE, Tool Calling and Structured Outputs, Transformers and Attention
How to use them
- Each colour means one kind of block. Blue blocks are tasks and commands. Red blocks are fixes. The key is at the top of each poster.
- Click a command to copy it. Replace each value in angle brackets, such as
<model>. - Each block names the lesson or manual section that its facts come from. Open it when you need the reason.
- Print a poster on one or two landscape pages. It fits A4 and Letter paper.