https://devopstales.github.io/tags/latency/ Recent content in Latency on DevOpsTales Hugo -- gohugo.io en-US Thu, 01 Oct 2026 00:00:00 +0000 https://devopstales.github.io/ai/model-routing-strategies/ Thu, 01 Oct 2026 00:00:00 +0000 https://devopstales.github.io/ai/model-routing-strategies/ A 70B model summarizing an email burns money; a 3B model reviewing production code courts disaster. Routing matches task complexity to model capability across four strategies — capability, cost, latency, hybrid — each optimizing what hurts most. Production systems converge on hybrid; start there only after one model genuinely fails.