Promote gradually. Keep the teacher in reach.

The rollout ladder

1
Shadow
Score or compare the student without returning its answer to users.
2
Percent
Route a small, explicit percentage while monitoring health and quality.
3
Full
Promote only after the percentage cohort is stable.
4
Retire
Stop routing to the deployment while preserving its audit trail.

Create a shadow deployment

curlexample

curl -X POST "https://<your-lizzy-host>/v1/distill/deployments" \
+  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
+  -H "Content-Type: application/json" \
+  -H "Idempotency-Key: deploy-shadow-001" \
+  -d '{"student_model":"<student_model_id>","alias":"support-v1","mode":"shadow","rollout_percent":0,"teacher_fallback":true,"fallback_source":"<source_id>","fallback_model":"<teacher-model>"}'

Pythonexample

import os, requests

deployment = requests.post(
    "https://<your-lizzy-host>/v1/distill/deployments",
    headers={"Authorization": f"Bearer {os.environ['LIZZY_API_TOKEN']}", "Idempotency-Key": "deploy-shadow-001"},
    json={
        "student_model": "<student_model_id>", "alias": "support-v1",
        "mode": "shadow", "rollout_percent": 0,
        "teacher_fallback": True, "fallback_source": "<source_id>",
        "fallback_model": "<teacher-model>",
    },
).json()

TypeScriptexample

const response = await fetch("https://<your-lizzy-host>/v1/distill/deployments", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.LIZZY_API_TOKEN}`, "Content-Type": "application/json", "Idempotency-Key": "deploy-shadow-001" },
  body: JSON.stringify({
    student_model: "<student_model_id>", alias: "support-v1",
    mode: "shadow", rollout_percent: 0, teacher_fallback: true,
    fallback_source: "<source_id>", fallback_model: "<teacher-model>",
  }),
});
if (!response.ok) throw await response.json();
const deployment = await response.json();

Promote by changing one boundary

For a percentage rollout, set mode: "percent" and an explicit rollout_percent from 1 to 100. Do not bundle a model swap, fallback change, and traffic increase into one approval.

PATCH/v1/distill/deployments/{deployment_id}

Propose {"mode":"percent","rollout_percent":10}, observe, then request the next boundary.

Fallback and recovery

When teacher_fallback is enabled, a degraded deployment keeps a known-good upstream path available. If quality or health drops, reduce the rollout or return to shadow before investigating.

To call the deployment, keep the same OpenAI-compatible proxy and use model: "lz:support-v1".