Rounds
System design
Design a model service end to end: requirements, data, model, serving, evaluation and cost.
22 of 15 planned
L1 3L2 9L3 7L4 3
RolesLLM Product EngineeringAgent Systems EngineeringAI Evaluation and ReliabilityAI Data SystemsCustomer AI DeploymentDeveloper Experience and EducationInference Engineering
- A churn model with a good AUC that nobody uses
- The signals that a demo has become a product
- When to say no to a model
- A spam classifier from 800 messages and a pretrained body
- Batch or online serving for a two-hour freshness limit
- Design a feature store that keeps training honest
- Design monitoring for a payment risk model
- Design the churn system, not the churn model
- Keep a chat service answering when the provider does not
- Project the monthly bill of a chat feature
- Put a model demo behind a public link safely
- Roll out a new prompt without betting the product on it
- A payment fraud check inside a hundred milliseconds
- Accuracy fell and every input monitor stayed green
- Design a chat API for ten thousand daily users
- Design cited search over ten repositories of code
- Design the gate that decides whether a model ships
- Offline accuracy was fine and live accuracy is not
- One command from raw data to a served tuned model
- Keep clean data when the model changes its own future
- Review an ML design document and name the blockers
- Write the rollback plan before the training run starts
No item matches this level.
Taught in
The answers on this card come from these lessons.
- What Is Machine Learning
- ML System Design
- Monitoring Models in Production
- Feature Stores and Training-Serving Skew
- Building a Complete LLM Pipeline
- Building a Production LLM Application
- Production Runtimes: Queue, Event, Cron
- Model Demo Interfaces
- Capstone 02
- Capstone 07
- Capstone Lesson 38: Classifier Fine-Tuning by Head Swap
- Capstone Lesson 39: Instruction Tuning by Supervised Fine-Tuning
- Capstone Lesson 40: Direct Preference Optimization from Scratch