#latency

Cutting agent latency from 30s to 8s without model swap

A founder's chat product had a problem: users were abandoning the conversation before the AI agent finished responding. The p95 latency was 31 seconds. The team's instinct was to switch to a smaller model. The actual fix was four changes th