Heartbeat thinking enables realtime voice agents to perform speculative reasoning and execute tools while listening.

Heartbeat Thinking Helps Voice Agents Prepare Tools During Speech

Meta description: Heartbeat Thinking lets voice agents prepare tool calls during speech, improving FDBench v3 coverage by 16 points and cutting latency by 36.4 percent.

Tags: voice agents, tool use, speech AI, Heartbeat Thinking, latency

Featured image: abstract, friendly illustration style showing sound waves forming a checklist beside a clock and connected tool icons, with no real people, logos, or text.

For builders of tool-using voice agents, the main problem is not only understanding speech, but starting useful work before the speaker finishes. Heartbeat Thinking is a method that keeps speculative reasoning and dispatched tools running while updated transcript fragments arrive. Instead of restarting or truncating a call every time the speech input changes, it uses new Thinker invocations to read the latest partial transcript and a shared record of completed reasoning and tool results. In reported tests, this approach raised tool coverage by 16 percentage points on FDBench v3 and lowered mean final-response latency by 36.4% compared with a SHANKS-style windowed baseline.

Why waiting still costs users

A voice agent that uses tools can receive actionable information before the user stops speaking. Speculative methods try to begin work during speech. But if the system is chunk-bounded or interrupts generation whenever the transcript updates, a tool call can be cut off before its arguments are complete. That creates a bad coupling: computation time is tied to speech updates, so less useful work is completed in advance. The user hears silence, then waits again after the speech ends.

How the heartbeat idea separates input updates from ongoing work

Heartbeat Thinking handles updated input through new Thinker invocations. At the same time, ongoing invocations and dispatched tools are allowed to finish. Periodic invocations read the latest partial transcript and a shared record of completed reasoning and tool results. Those records can help resolve missing arguments and initiate dependent calls. The effect is that preparation can progress across speech updates instead of being discarded by each update.

What the benchmarks show

The reported evaluation spans SpeechFCEval, FDBench v3, and AudioCRAG. Across those benchmarks, Heartbeat improves aggregate speculative tool coverage or retrieval reuse over a SHANKS-style windowed baseline. The results also show benchmark-dependent effects on final task quality, so the main advance is not a blanket claim that every task becomes better. The clearest reported gain is on FDBench v3, where tool coverage increases by 16 percentage points and mean final-response latency falls by 36.4%.

What this means for voice-agent design

For users, the practical promise is less dead time: listening can become useful preparation. For developers, the design shift is to treat partial speech as an input stream that can be sampled without killing work already in progress. The method also suggests that shared records of reasoning and tool results can serve as a bridge between earlier speculation and later corrections. The caveat is that final task quality still depends on the benchmark and task type, so teams should test Heartbeat-style scheduling against their own speech, tool-calling, and retrieval workloads before adopting it widely.