Cekura builds voice agents that test other voice agents
Cekura is the testing and observability platform voice-agent teams use to ship without guessing. They run thousands of synthetic conversations before go-live, red-team for jailbreaks and off-script behavior, monitor production drift, and benchmark the same scenarios across stacks (Vapi, Retell, LiveKit, Pipecat, and others).
That product only works if the models inside the loop are fast and reliable. If the tester voice agent is flaky, false negatives get flagged and teams waste time debugging something that was not broken. Every simulation, every adversarial turn, every bake-off is an inference workload.
Existing providers for their preferred model were unreliable and laggy
Cekura tests the latest models for their use case and zeroed in on Gemma 4 26B — an open-weight model with the right balance of intelligence, instruction-following, and latency.
They tried the obvious serverless options first: DeepInfra Turbo and Google Vertex. On paper both serve Gemma. For Cekura's voice-shaped workloads, two things broke:
- Reliability — ~15% of requests to the other providers returned 429 errors, causing blocked checks and failed runs that poison a benchmark suite
- p95 / tail TTFT — median latency looked okay, but the tail did not
For a voice QA platform, a model that takes up to 3.680 seconds at p95 is out of the question. The counterpart agent times out by then.
Enter Autoloops, with an optimized inference engine for their workload
We served Gemma 4 26B-A4B through Hanoi — Autoloops' purpose-built inference engine, optimized for time-to-first-token on voice-call shapes, that holds up at p99 as well as at p50.
Why a custom engine
Generic frameworks optimize for aggregate throughput across arbitrary workloads. Hanoi specializes for:
- One model: Gemma 4 26B-A4B, using the specific features of the model architecture
- One shape: voice calls — ~2K-token prompts, short generations, prefix cache across turns
- One goal: minimize TTFT under concurrency
Built from first principles in C++/CUDA (not a vLLM/SGLang wrapper): custom tokenizer, weight format, kernels, scheduler, and HTTP server — using low-level matmul primitives (cuBLASLt / CUTLASS) underneath.
Read more about the Hanoi engine here.
Results
We hosted Gemma 4 26B on Autoloops' servers within a week and gave them a drop-in OpenAI-compatible API endpoint.
Comparison against their existing providers:
| Provider | Mean | p50 | p90 | p95 | p99 |
|---|---|---|---|---|---|
| Autoloops Gemma 4 26B | 0.574s | 0.510s | 0.850s | 0.986s | 1.379s |
| Deep Infra Gemma 4 26B | 1.290s | 1.150s | 2.032s | 2.464s | 3.452s |
| Vertex Gemma 4 26B | 2.079s | 1.880s | 3.009s | 3.680s | 4.856s |
Green marks the best result in each column.
How to read this
- Lower variance between p50 and p99. The gap from median to p99 is 0.87s on Autoloops versus 2.30s on DeepInfra Turbo and 2.98s on Vertex — 62% and 71% lower, respectively.
Why voice agent builders should notice
LLMs are the brain of any voice agent built on a cascading pipeline, but also the most brittle part of it. Closed APIs and generic serverless hosts optimize for average traffic. Voice QA and voice agents fail on the tail — p95, p99, and the run that never returns.
Autoloops exists to solve this: serverless or VPC deployments for open models (Gemma, Qwen, and others) for those enabling realtime human–machine interaction.