Models & Routing
Models are replaceable reasoning engines. Routing starts with privacy: some tasks must remain local, while approved cloud tasks receive only a minimum package. Capability, tool support, context size, cost, latency, and reliability narrow the candidates. A live test proves availability; output verification proves fitness.
Identify reasoning, coding, vision, media, or low-cost routine work.
Decide local-only, approved-cloud, or no-AI before selecting a provider.
Measure structure, tool use, latency, and output quality.
Verify exact model IDs and provider routing explicitly.
Keep results and limitations; never infer success from configuration alone.
Routing matrix
| Dimension | Question | Evidence |
|---|---|---|
| Privacy | May this context leave the device? | Approved data package |
| Capability | Can it perform the task and tool calls? | Focused evaluation |
| Context | Does the task fit safely? | Token/input measurement |
| Cost | What ceiling applies? | Observed usage or local budget |
| Fallback | What happens on failure? | Tested alternate route |
What breaks—and how to recover.
Test with explicit provider and exact model identifier.
Apply the same privacy ceiling before failover.
Run task-specific evals instead of a simple connectivity prompt.
Copyable implementation prompt
Create a model-routing policy for [TASK CLASSES]. Define privacy ceiling, required capabilities, tool support, context limit, latency and cost target, preferred route, fallback, and evaluation.
Use placeholders for paths, accounts, endpoints, identities, and credentials. Never publish secret values, private records, or a live control surface.
Verification gate
- Every route respects privacy
- Exact provider/model route is tested
- Fallback is independently tested
- Quality is evaluated on representative tasks