Skip to content
Hussh
Connect MCP

Real-time Voice Transport — Provider-Neutral Protocol

Public concept and protocol design for One's low-latency, full-duplex realtime voice transport, including client/server requirements, provider-neutral voice transport contract, and animated waveform visual feedback.

Concept

TL;DR: Design and requirements for One's low-latency, full-duplex realtime voice transport protocol, including a provider-neutral voice transport contract, smooth animated waveform feedback, and instant client-interruption handling.

Status as of 2026-07-01: see body.

Relations

The One platform communicates with users via highly conversational, responsive, and natural interfaces. Real-time Voice Transport establishes a standard, low-latency protocol and visual system that enables full-duplex conversational voice interactions across a wide variety of hardware devices and AI models.

By isolating the audio capture, transport serialization, and visual synthesis layers, the voice runtime ensures a continuous stream of user intent reaches the reasoning models while providing smooth, immersive feedback.

The Three Core Pillars

  1. Provider-Neutral Voice Transport Contract: Enforces a standard interface for real-time audio chunk chunking, synchronization, interrupt-handling, and stream state. Reasoning runtimes must conform to this contract rather than coupling to provider-specific WebSockets (e.g., Gemini Live API vs. OpenAI Realtime).
  2. Smooth Visual Waveform Feedback: The user interface displays a continuous, physics-modeled animated waveform embedded inside the Agent voice bar. This waveform acts as the heartbeat of the connection, reacting instantly to incoming audio metadata, decibel levels, and connection state.
  3. Full-Duplex Interruption Model: Implements client-side and server-side fast interruption detection. When the user speaks while the model is responding, the transport layer sends a deterministic cancel frame, immediately cutting the outgoing audio stream and resetting the model's response pipeline without lag or overlap.

Protocol Mechanics

The voice transport operates over low-overhead WebSockets or WebRTC data channels:

  • Upstream (Client-to-Agent): Audio is chunked (PCM 16-bit, 24kHz or 16kHz mono) and packed into binary frames carrying a millisecond timestamp and stream ID.
  • Downstream (Agent-to-Client): The server-side multiplexer forwards generated audio chunks along with volume envelope metadata (amplitude and frequency buckets) to drive the visual waveform on the client device.
  • Interruption Signal: A high-priority control message ("interrupt") with the active stream's target timestamp ensures both client and server immediately flush their playback buffers.

Sources