- The end-to-end latency budget across the cascaded pipeline (STT finalization, LLM prefill and generation, TTS time-to-first-byte), with a target of sub-1-second end-to-end response time
- Diagnosing and fixing cross-cutting bottlenecks that touch multiple pipeline legs at once, for example, prompt size and cache-hit rate degrading prefill time, network payload, and throughput contention simultaneously
- Evaluating and integrating STT/LLM/TTS providers (including in-region India options) against the latency and quality bar, and building the abstraction layer that keeps us model-agnostic and provider-swappable
- Turn detection, interruption and barge-in handling, filler audio, and retry logic, the mechanics that make a bot feel like it is actually listening
- Instrumentation and tracing across the full call lifecycle so every latency regression is diagnosable from data, not guesswork
- Making the build vs. buy call on pipeline components as the product scales beyond the current reference customer
Non-negotiable
You have hands-on, production experience optimizing a cascaded voice pipeline or voice harness, meaning STT, LLM orchestration, and TTS stitched together into a live conversational system, not a single-model or text-only pipeline. You can talk concretely about where latency hides in each leg, what trade-offs you made between accuracy and speed, and what broke when you pushed the system harder. If your voice AI experience is limited to calling a hosted conversational API (Twilio Voice, a managed voicebot platform) without touching the pipeline internals, this is not a fit, regardless of years of experience elsewhere.