AI Engineering from Scratch

Interviews

Questions and measured incidents for AI engineering interviews. Try each question first, then open the answer. Every answer links to the lesson that teaches it, and every number comes from a run that you can read in the repository.

questions
42
scenario
1
cards
23
role views
7

Foundations

The math, the networks and the transformer that every later answer builds on.

  • FoundationsComing soon

    Math and classical ML

    Linear algebra, probability, optimization, and the classical models that still win on tabular data.

    0 of 15 planned
  • FoundationsComing soon

    Deep learning

    Backpropagation, losses, optimizers, initialization and how to debug a network that does not learn.

    0 of 15 planned
  • FoundationsComing soon

    Transformers

    Attention, positions, the transformer block, the KV cache and the cost of long context.

    0 of 15 planned
  • FoundationsComing soon

    Tokenization, sampling and decoding

    How text becomes tokens, how a model picks the next token, and how speculation makes decoding faster.

    0 of 15 planned

Models

How a model is tuned after training, served to users, and fit into GPU memory.

  • ModelsComing soon

    Post-training

    Supervised tuning, preference tuning, LoRA and reinforcement learning on a trained model.

    0 of 15 planned
  • Models32 items

    Inference and serving systems

    Batching, the KV cache, prefix caching, schedulers and how to measure a serving engine.

    32 of 30 planned
  • Models10 items

    GPU memory, quantization and hardware

    Where GPU memory goes, how quantization frees it, and how work splits across GPUs.

    10 of 15 planned

Applications

Retrieval, prompts, structured output, evaluation and models that see and hear.

  • ApplicationsComing soon

    RAG

    Chunking, hybrid retrieval, reranking and how to measure a retrieval pipeline.

    0 of 15 planned
  • ApplicationsComing soon

    Prompting and structured output

    Prompt design, context engineering, schemas, constrained decoding and function calls.

    0 of 15 planned
  • ApplicationsComing soon

    Evaluation

    Metrics, eval harnesses, judges, calibration and how to trust a score.

    0 of 15 planned
  • ApplicationsComing soon

    Multimodal

    How vision and language models join images and text, and how they are trained.

    0 of 15 planned

Agents

Agent loops, protocols, teams of agents, sandboxes and evidence from binaries.

  • AgentsComing soon

    Agents

    The agent loop, tools, scope contracts, verification gates and why agents fail.

    0 of 15 planned
  • AgentsComing soon

    Protocols: MCP, A2A and skills

    How agents call tools and other agents, and how those calls stay safe.

    0 of 15 planned
  • AgentsComing soon

    Multi-agent systems

    Supervisors, handoffs, shared memory, consensus and the failure modes of agent teams.

    0 of 15 planned
  • AgentsComing soon

    Sandboxes

    Isolation for agents that run code or drive a computer, and the policy around it.

    0 of 15 planned
  • AgentsComing soon

    Evidence and reverse engineering

    Reading binaries, recovering intent, and stating confidence with the evidence for it.

    0 of 15 planned

Production

Operating a model service: releases, observability, cost, capacity and safety.

  • ProductionComing soon

    Production and MLOps

    Observability, progressive delivery, A/B tests, chaos tests and on-call for model services.

    0 of 15 planned
  • ProductionComing soon

    Cost and capacity

    Caching, routing, batch APIs, cold starts and the cost of each request.

    0 of 15 planned
  • ProductionComing soon

    Safety

    Reward hacking, prompt injection, jailbreaks, refusals and safety gates.

    0 of 15 planned

Rounds

Whole interview rounds: system design, incidents, coding and behavioral questions.

  • RoundsComing soon

    System design

    Design a model service end to end: requirements, data, model, serving, evaluation and cost.

    0 of 15 planned
  • Rounds1 item

    Incidents and scenarios

    Measured incidents from our manuals. Read the evidence one listing at a time and find the cause.

    1 of 30 planned
  • RoundsComing soon

    Coding

    Write the code that the lessons build. Each item links to a practice task with checks.

    0 of 15 planned
  • RoundsComing soon

    Behavioral

    Framing work, planning from evidence, delegation and turning feedback into a system.

    0 of 15 planned

All items

Each item has a prompt, a short answer, a full answer, follow-ups and the wrong answers that interviewers hear most.

  1. The two phases of one requestL1ScreenConceptInference and serving systems6 min
  2. Why 255 tokens per second is a sloganL1ScreenConceptInference and serving systems6 min
  3. Cached tokens at zero with a stable templateL2WorkingDebugInference and serving systems8 min
  4. Choose tensor, pipeline or data parallelismL2WorkingDesignInference and serving systems10 min
  5. How many users fit in the KV poolL2WorkingEstimateInference and serving systems10 min
  6. How speculative decoding keeps the output the sameL2WorkingConceptInference and serving systems8 min
  7. One token budget instead of two phasesL2WorkingConceptInference and serving systems8 min
  8. Read the GPU clock before the throughputL2WorkingDebugInference and serving systems8 min
  9. Read three scheduler log linesL2WorkingCode readingInference and serving systems8 min
  10. Sleep, wake and weight updates for an RL loopL2WorkingConceptInference and serving systems8 min
  11. Temperature zero gave two different answersL2WorkingConceptInference and serving systems8 min
  12. The knee that the prefix cache hidL2WorkingDebugInference and serving systems8 min
  13. Two clients, two numbers, one serverL2WorkingConceptInference and serving systems8 min
  14. What an FP8 KV cache buys and costsL2WorkingTrade-offInference and serving systems10 min
  15. What many LoRA adapters cost one serverL2WorkingDesignInference and serving systems8 min
  16. Why one ID at the top of a prompt kills the cacheL2WorkingDebugInference and serving systems8 min
  17. Why the KV cache is a pool of 16-token blocksL2WorkingConceptInference and serving systems8 min
  18. Why the shared part of the prompt goes firstL2WorkingConceptInference and serving systems8 min
  19. A larger token budget at low loadL3SeniorTrade-offInference and serving systems10 min
  20. Acceptance by draft position, tokens per passL3SeniorEstimateInference and serving systems8 min
  21. An admission cap decides who waitsL3SeniorTrade-offInference and serving systems10 min
  22. Pick a concurrency cap when throughput keeps risingL3SeniorTrade-offInference and serving systems12 min
  23. Pick the EAGLE3 draft depth for your concurrencyL3SeniorTrade-offInference and serving systems10 min
  24. Should the prefill chunk be smallerL3SeniorTrade-offInference and serving systems10 min
  25. Speculation won the benchmark and lost in productionL3SeniorDebugInference and serving systems10 min
  26. Two data-parallel copies, less output than one cardL3SeniorDebugInference and serving systems10 min
  27. What determinism costs a live serverL3SeniorTrade-offInference and serving systems10 min
  28. When an FP8 checkpoint paysL3SeniorTrade-offInference and serving systems10 min
  29. When prefill-decode disaggregation paysL3SeniorDesignInference and serving systems12 min
  30. A split-server benchmark that counted empty answersL4StaffCode readingInference and serving systems10 min
  31. Design a chat service on a fleet of L4 cardsL4StaffDesignInference and serving systems15 min
  32. Same throughput, two choices at a full poolL4StaffTrade-offInference and serving systems12 min
  33. The model is 35 GB now, so it fitsL1ScreenConceptGPU memory, quantization and hardware5 min
  34. Count the KV bytes per token from the configL2WorkingEstimateGPU memory, quantization and hardware8 min
  35. Pick a 4-bit format for a 14B model on 24 GBL2WorkingTrade-offGPU memory, quantization and hardware10 min
  36. Why an 8B model fails to start on a 24 GB cardL2WorkingEstimateGPU memory, quantization and hardware8 min
  37. Pick the memory fraction on unified memoryL3SeniorEstimateGPU memory, quantization and hardware10 min
  38. Split the model or copy it across two cardsL3SeniorDesignGPU memory, quantization and hardware10 min
  39. The FP8 KV scales that never loadedL3SeniorDebugGPU memory, quantization and hardware8 min
  40. The pool was 95 percent empty and the server diedL3SeniorDebugGPU memory, quantization and hardware10 min
  41. Why FP8 grew the pool 4.77 times and not 2L3SeniorEstimateGPU memory, quantization and hardware8 min
  42. Eight copies of the same KV cacheL4StaffConceptGPU memory, quantization and hardware10 min
  43. More load gave a faster first tokenL3SeniorIncidentIncidents and scenarios25 min

How to use the bank

  1. Read the prompt and answer it aloud or on paper before you open anything.
  2. Open the short answer, then the full answer. Compare the reasoning, not only the result.
  3. Answer the follow-ups one at a time. They rise in level.
  4. Read the wrong answers. Each one is a mistake that sounds right.
  5. In a scenario, read the evidence one listing at a time and commit to a cause before you open the resolution.

To practise with a coding agent, clone the repository: the files are in interviews/.