The signals that a demo has become a product
RolesLLM Product EngineeringDeveloper Experience and EducationCustomer AI Deployment
The question
You built a small web demo of a model for reviewers. Three months later a colleague asks whether it is "production". How do you tell, and what changes when the answer is yes?
Answer it first, aloud or on paper. Then open the answer.
The answer
Show short answer
A demo serves known people in short sessions from one process. It has become a product when people outside the team use it every week, when someone asks for an uptime target or reports an outage, when users want to sign in or see history, when the weekly model cost is a budget line, or when the input is covered by a contract or a law. At that point the demo moves onto the production stack with authentication, fallback models, tracing, and cost tracking per user.
Show full answer
The five signals
- People outside your team use it every week.
- Someone asks for an uptime target, or reports an outage.
- Users ask to sign in, save results, or see their history.
- The weekly model cost is a line in a budget.
- The input includes data that a contract or a law covers.
What changes
| Property | Demo | Product |
|---|---|---|
| Users | Known people, short sessions | Unknown people, daily use |
| State | None | Accounts, history, saved results |
| Availability | Best effort, one process | A written target, replicas, an on-call owner |
| Cost control | One daily usage cap | Per-user quotas and billing |
| Observability | One log record per request | Traces, metrics, and alerts |
What the demo already had
A good demo already had the small versions of these controls: an input limit, a per-client rate limit, a daily cap, and a log that keeps the input length and not the text. The product stack adds real authentication at the gateway, a fallback model chain for provider failures, traces across the components, and cost accounting per request and per user. The input limit and the log privacy carry over unchanged.
Why the question matters
The dangerous state is the middle: a demo with product traffic and demo controls. One daily cap does not stop one tenant from using the whole budget, and one process has no replica to take over when it dies.
Follow-ups
L1ScreenWhich single signal is the strongest?
Show expected answer
an outage report, because it means someone depends on the demo and expects availability.
L2WorkingThe weekly model cost appears in a budget. What control replaces the daily cap?
Show expected answer
per-user quotas and billing, so the total is attributed and limited per tenant.
L2WorkingRegulated data starts arriving in the input. What must the log do?
Show expected answer
keep no input text and no client address, with a retention period and access control on the records.
Wrong answers
Show the wrong answers that interviewers hear
“It is production when it runs on a server instead of a laptop.”
Where it runs does not decide; who depends on it does.
“Add a disclaimer that it is a demo.”
A disclaimer does not stop traffic, cost or outages.
“Keep the demo and build the product separately later.”
The signals mean the demo already carries product risk now.
Taught in
The model demo interfaces lesson lists the signals and the demo-to-product table, and the production LLM application lesson describes the stack the demo moves onto.