DMR, Compose models, and the docker/mcp-gateway service

Every model call in Part 5 and in this part ran with no API key and no cloud account. The model was ai/qwen3:4b on Docker Model Runner, on the same Mac as the agent.

When you finish this section, you can reach the runner from the host, a container, and a sandbox, and give it to a Compose service.

Docker Model Runner on port 12434

Docker Model Runner (DMR): the Docker component that pulls models from Docker Hub, an OCI registry, or Hugging Face. It serves them over OpenAI, Anthropic, and Ollama compatible APIs. Since Desktop 4.71.0, Docker Model Runner is off by default, and you turn it on in Settings. docker desktop enable model-runner --tcp <port> turns on host TCP, and the capture used port 12434.

{
  "running": true,
  "backends": {
…
    "llama.cpp": "Running: llama.cpp b9879-metal (sha256:<digest>) 72874f5",
…
  },
  "kind": "Docker Desktop",
  "endpoint": "http://model-runner.docker.internal/v1/",
  "endpointHost": "http://localhost/exp/vDD4.40/v1/"
}

The Model Runner API has no authentication. Any client that reaches the port can pull, load, and run models. The base URL depends on where the caller runs, and figure 6.5 maps the five that this manual met:

After the base URL come path prefixes and whole endpoints. The OpenAI paths sit under /engines/v1/, and /engines/llama.cpp/v1/ names the engine. The Anthropic table lists /anthropic/v1/messages, while its examples call /v1/messages (conflict C79). No capture tested either path. Ollama clients use /api/. The capture also met the /v1/ root, which status reports as endpoint.

The 8B tag ai/qwen3:latest failed to pull on the recording Mac, as docker-agent run, new, and doctor explains (the capture). With only the 4B tag pulled, doctor resolves auto to dmr/docker.io/ai/qwen3:4b (the capture). By default, Model Runner unloads an idle model after a few minutes, and provider_opts.keep_alive changes that time. provider_opts.context_size sets the context window through the runner's _configure endpoint. No capture ran docker model configure, so conflict C77 stays open.

An agent file can also name the runner in a providers: block. fixtures/agents/dmr.yaml sets base_url: http://localhost:12434/engines/llama.cpp/v1, and its run answered Hello (the capture). An explicit base_url bypasses the --record proxy, so that run is live on every capture (the capture).

Compose models

name: m101

models:
  llm:
    model: ai/qwen3:4b

services:
  mcp-gateway:
    image: docker/mcp-gateway
    use_api_socket: true
    command: ["--transport=streaming", "--port=8811"]

  printer:
    image: alpine:3.22
    depends_on: [mcp-gateway]
    models:
      llm:
        endpoint_var: LLM_URL
        model_var: LLM_MODEL
…

models: the top-level Compose element that declares a model, which a service binds by name. The short syntax derives LLM_URL and LLM_MODEL from the name llm, and endpoint_var and model_var choose the names. Compose v2.38 or later is required (our source record). docker compose up pulled and configured the model before it started a container:

 llm Pulling
 llm Pulled
 llm Configuring
 llm Configured
…
printer-1 | LLM_MODEL=ai/qwen3:4b
printer-1 | LLM_URL=http://model-runner.docker.internal/v1/
printer-1 | {"object":"list","data":[{"id":"docker.io/ai/qwen3:4b","object":"model","created":0,"owned_by":"docker","dmr":{}}]}
printer-1 | wget: server returned error: HTTP/1.1 401 Unauthorized

Compose injected the /v1/ root, where the plan, read from the model-runner source, expected /engines/v1/. GET ${LLM_URL}models answered from inside the container all the same. Compose cannot start sandboxes (conflict C96), and no capture ran models: on the private engine inside a sandbox (conflict C95).

The docker/mcp-gateway service

The mcp-gateway service runs the open source MCP gateway with the host's Docker API socket, so that it can start MCP servers as containers. In the capture it found no profile and no server, listed 0 tools, and added its own management tools such as mcp-find and mcp-add. It printed Gateway URL: http://localhost:8811/mcp and a bearer token, and the request of printer without that token got 401 (the capture). Figure 6.6 draws the stack.

Four things named gateway

The Toolkit gateway runs each MCP server in its own container. Each MCP tool container has a limit of 2 GB, and each one gets 1 CPU. A docker-agent toolset with ref: docker:<name> takes that route, and runs the MCP server as a Docker container behind the MCP Gateway. The sbx MCP gateway is sbx mcp add, sbx mcp load, and --static-mcp, and the MCP gateways lesson covers the pattern.