Alibaba Cloud has added Qwen-Audio 3.1 Realtime to Model Studio, an end-to-end voice model that brings tool calling, semantic turn detection, and full-duplex control to its real-time audio stack. It ships as qwen-audio-3.1-realtime-plus, alongside the older qwen-audio-3.0-realtime-plus and qwen-audio-3.0-realtime-flash, which remain available.

The model holds a persistent full-duplex connection open, streaming microphone audio in while returning speech and text at the same time. Beyond WebSocket it supports AOQ and WebRTC; Alibaba recommends AOQ for client integrations that need stable latency, resilience on weak networks, and built-in full-duplex noise suppression and echo cancellation. The API is event-driven: the client sends session.update and input_audio_buffer.append, and the server answers with response.audio.delta and response.done, with no polling.

The documentation lists a Singapore WebSocket endpoint, DASHSCOPE_API_KEY authentication, and Python quick-start samples for native WebSocket and push-to-talk. One operational detail stands out: the turn-detection mode can only be set before the first audio is sent, so switching between server_vad, smart_turn, and push-to-talk mid-session requires a new session.update and, in practice, a fresh connection. The technical report adds that both the full-duplex decision model and the speech-to-text model share an audio-encoder-plus-LLM architecture, diverging mainly in how they are trained.

A technical report submitted on 21 September 2026 describes the work in three layers: Think, Act, and Speak and Coordinate. Think combines Core-Cocktail supervised fine-tuning with two on-policy distillation paths that carry text reasoning into the audio model; Act trains tool use in self-evolving simulated environments using dialogue-, milestone-, and turn-level rollouts; and Speak and Coordinate governs when, how, and whether the assistant speaks or acts. Alibaba reports that overall task success on a half-duplex, speech-to-text adaptation of the τ-Voice tool-use benchmark rises from 78.4% to 82.0%, and that the response rate to background speech on Full-Duplex-Bench v1.5 falls from 73.0% to 13.0%.

The gains are not uniform. The same report shows the EVA-A Mean slipping from 70.50% to 66.26%, MMAU easing from 82.21 to 81.60, and the in-house long-context VoiceChat-L score essentially flat at 4.58 against 4.61.

Alibaba also reports safety movement in the same evaluation set: the HalluQA no-hallucination pass rate rises from 68.44% to 71.33%, TruthfulQA from 78.73% to 82.28%, and the multi-turn attack success rate falls by 54.50 percentage points in Chinese and 40.00 points in English. A separate Voice Harness prototype, built on Qwen-Audio 3.0 Realtime as its foreground model, extends spoken interaction to persistent tasks through foreground–background coordination and memory.

Confirmed

  • Model ID and availability: qwen-audio-3.1-realtime-plus, with qwen-audio-3.0-realtime-plus and qwen-audio-3.0-realtime-flash still listed.
  • Protocols: WebSocket, AOQ, and WebRTC. Alibaba recommends AOQ for stable latency, weak-network resilience, and built-in full-duplex noise suppression and echo cancellation.
  • Three interaction modes: acoustic VAD (server_vad), semantic turn detection (smart_turn), and manual push-to-talk.
  • Function Calling and web search are both supported, but cannot be enabled in the same session.
  • Conversation context can be managed directly through client events: create, retrieve, and delete conversation items.
  • Audio format: 16 kHz, 16-bit mono PCM input; 24 kHz, 16-bit mono PCM output.
  • Languages: German, English, Spanish, French, Indonesian, Italian, Japanese, Korean, Portuguese, Russian, and Chinese, plus 21 Chinese varieties.
  • Context capacity: up to 50 audio turns or 300 seconds of cumulative audio, with 20 turns as the default.
  • In smart_turn mode, speaker enhancement can lock onto a target speaker from a pre-recorded sample, blocking other voices and background noise.
  • Voices: system and cloned voices; the voice can only be set in the first session.update.

Unknown

  • Pricing per minute or per token for real-time audio sessions.
  • SLA or latency guarantees for production workloads.
  • Independent replication of the reported 82.0% τ-Voice task success and 13.0% background-speech response rate.
  • Rollout beyond the Singapore endpoint shown in the documentation; other regions are not enumerated.
  • Whether the mutual exclusion of function calling and web search is a temporary or architectural limitation.

Our take

Alibaba is framing Qwen-Audio 3.1 as a voice-native agent platform rather than a speech-to-text pipeline with text-to-speech attached. The background-speech result — 73% down to 13% — is the number that matters for noisy deployments, and the report's WebSearch1K figures point the same way: mean search queries per turn drop from 4.37 to 1.05 while F1 gives up only 2.26 points, a trade that favors execution economy over exhaustiveness. The constraint developers will hit first is the one-tool-per-session rule, which forces a choice between retrieval and function calling.

Sources