Benchmark · Coval stt-v3 · measured 24 Aug 2026

Qwen3-ASR-1.7Bv/sDeepgram Nova-3

Streaming speech-to-text on Coval stt-v3 — 897 conversational clips. Same audio, same runner, same metrics.

LOWER WER

Qwen3-ASR-1.7B · WER

2.34%

Deepgram Nova-3 · WER

3.33%

0.99 pp lower mean word error rate across the full set.

Accuracy

Metric	Qwen3-ASR-1.7B	Nova-3
WER mean	2.34%	3.33%
WER p90	8.3%	10.0%
WER max	93.5%	100%

Latency

Metric	Qwen3-ASR-1.7B	Nova-3
Time to first token (mean)	1,290 ms	1,672 ms
Time to first token (median)	1,261 ms	1,932 ms
Time to forced final (mean)	100 ms	68 ms
Time to forced final (median)	97 ms	57 ms
Real-time factor (mean)	1.048	1.010

Green is better (lower). TTFT is time to first partial; TTFS is time from end-of-speech to the forced Finalize result.

Clip-level WER

How many of the 897 files each model won. A tie is usually both at 0%.

Qwen3-ASR-1.7B lower

250

Tie

563

Nova-3 lower

Setup

Dataset is Coval stt-v3: 897 spontaneous voice-agent turns from pipecat's STT benchmark data. Both models were scored with the official coval-bench streaming path (16 kHz mono PCM16, Deepgram v1/listen protocol, endpointing=false, then Finalize / CloseStream). WER uses Coval's Whisper-normalizer pipeline.

Qwen3-ASR-1.7B was served via the autoloops API with no model weights changed. Nova-3 was Deepgram's hosted listen API. Clips were identical and run independently; 19 Nova-3 handshake/503 failures were retried and succeeded.

Try Qwen3-ASR in the console →$0.003 / audio min · streaming

Source: local coval-bench run, 24 August 2026. Concurrency 8 (Nova-3 retries at 2).