Stories about Splitwise

As disaggregated LLM serving decouples prefill and decode phases, routing mechanisms become critical bottlenecks determining cluster throughput and latency. By dynamically calibrating routing policies using prompt lengths, KV cache pressure, and SLO classes, cluster effective throughput under bursty traffic can be significantly improved, indicating that the optimization focus of LLM infrastructure is extending from mere hardware separation to end-to-end intelligent scheduling.