ByteDance has officially launched SeedRealtime, a native audio-visual full-duplex large language model that unifies audio, video, and text within a single end-to-end architecture. The system enables real-time interaction over continuous multimodal streams, allowing users to watch, listen, and speak simultaneously in what the company describes as a key step toward omni-modal interaction. SeedRealtime has already been fully rolled out, marking what ByteDance calls the industry's first large-scale deployment of audio-visual full-duplex technology.

The model addresses fundamental limitations of cascaded systems that chain separate automatic speech recognition, vision-language, and text-to-speech modules. By integrating perception, understanding, decision-making, and response generation within one model, SeedRealtime reduces information loss and error accumulation while natively supporting full-duplex interaction that continuously models conversational state and timing. End-to-end human evaluations show that compared with cascaded models, the system reduces conversational pacing issues by half, significantly decreases interruptions, latency, and false triggers, and markedly improves the usability rate of single-turn conversations in terms of smoothness and completeness.

What's New / Specs

SeedRealtime introduces three core breakthroughs that distinguish it from both cascaded pipelines and earlier end-to-end approaches:

  • Joint Audio-Visual Understanding: The model natively fuses visual, audio, and temporal information. It uses live visual context to resolve homophones and ambiguous speech, continuously tracks visual changes and references, and aligns what is seen, heard, and said for more accurate intent understanding. In a demonstrated dinner-party scenario, the model matched names to faces by appearance, recognized individual speakers by voice, kept each person's voice tied to their identity throughout overlapping dialogue, and synthesized a group travel plan that accounted for each participant's stated preferences and constraints.
  • Proactive Interaction: Combining persistent environmental awareness with the ability to initiate communication, SeedRealtime can detect scene changes — such as the appearance of a key object — and respond proactively without waiting for a user prompt. It can also weave tool calls into its responses, shifting interaction from passive reaction to active collaboration. In a museum-tour demo, the user asked to be reminded when a specific artifact appeared; as the camera panned, the model recognized the target object, spoke up with a reminder, and provided detailed historical explanation of the piece and related exhibits.
  • Natural Conversational Timing: The model perceives conversational state and rhythm in real time, deciding when to join in, pause, or respond. It distinguishes side conversations and background noise to avoid false triggers, maintaining smooth and coherent communication. The system handles overlapping speech, backchannels, and pauses without relying on an external voice activity detection module, because turn-taking logic is embedded in the unified model.

On the engineering side, ByteDance optimizes for low-latency real-time interaction through chunked audio-visual input and streaming generation, while improving serving efficiency with efficient quantization and inference optimization. The unified framework eliminates the need for an external voice activity detection module to manage turns, enabling true full-duplex operation where perception, understanding, and expression run in parallel over continuous streams. The company reports that the model has been fully rolled out, indicating that the serving infrastructure can sustain the compute demands of continuous multimodal inference at production scale.

Why It Matters

The launch represents a significant shift in how multimodal AI systems handle real-world interaction complexity. Traditional cascaded systems introduce latency and information loss between discrete stages — audio transcription, visual analysis, reasoning, and speech synthesis each operate in sequence. Even end-to-end models have often relied on external turn-taking mechanisms, remaining essentially half-duplex question-answer systems. SeedRealtime's unified architecture addresses the core tension between continuous perception and timely expression.

In real-world settings, backgrounds are noisy, voices overlap, scenes change continuously, and users may pause, add information, or interrupt at any moment. The model's ability to jointly model audio, visual, and temporal streams means that when a user says "how do I do this," the system combines the current scene, gestures, gaze, and prior actions to determine the referent — all while maintaining conversational flow. The supplementary blog post demonstrates seven concrete scenarios: a dinner party where the model identifies multiple speakers by voice and appearance, tracks overlapping conversations, and synthesizes a group travel plan; a restaurant setting where it translates a Chinese-only menu in real time, explains cultural context for dishes such as "fish-fragrant pork" (which contains no fish) and century eggs, and translates server comments contextually; and a museum tour where it proactively reminds the user when a specific artifact appears and provides detailed historical explanation of casting, gold-and-silver inlay, and welding techniques.

These capabilities suggest applications in accessibility, education, professional collaboration, and consumer assistance where continuous multimodal awareness matters. The proactive interaction layer — where the model initiates based on environmental changes rather than waiting for prompts — moves the paradigm from reactive assistants to collaborative agents that maintain situational awareness. The company's outlook emphasizes further reductions in end-to-end delay, more robust performance in complex multi-party scenarios, and connecting tool use to real-world actions such as search, booking, and task completion, turning real-time multimodal understanding into concrete action.

Our Take

ByteDance's deployment of a native audio-visual full-duplex model at scale is a notable milestone for omni-modal AI. The company has moved beyond research demonstrations to what it describes as full rollout, which implies the engineering challenges of serving a unified audio-visual full-duplex model at production scale have been substantially solved. The reported 50% reduction in pacing issues versus cascaded systems is a meaningful quality metric, though independent benchmarking will be needed to validate claims across diverse languages, accents, acoustic conditions, and visual environments.

The architectural choice to unify all modalities in a single model — rather than the decoupled interaction-and-thinking layers proposed in contemporaneous research such as DuplexOmni — reflects a different trade-off. ByteDance's approach prioritizes end-to-end latency and tight audio-visual alignment, while the decoupled approach aims to preserve deep reasoning capacity by offloading complex cognition to a separate thinking layer. Both strategies are attempting to solve the same fundamental problem: real-time interaction should not stall while the model thinks. The DuplexOmni paper, published in June 2026, describes a method that separates an interaction layer (handling streaming perception and expression) from a pluggable thinking layer (providing complex reasoning and tool use), allowing asynchronous collaboration. SeedRealtime's single-model design may simplify deployment but could face limits on reasoning depth for very complex tasks.

Several questions remain. The company has not disclosed model size, training data composition, compute requirements, or whether the deployed version matches the full capability described in the announcement. The outlook section acknowledges ongoing work on lower latency, more robust multi-party scenarios, and connecting tool use to real-world actions — suggesting the current deployment is a foundation rather than a finished product. Pricing, API access, and developer tooling details have not been announced. For now, SeedRealtime represents the most ambitious production deployment of native audio-visual full-duplex interaction to date, and its real-world performance at scale will inform the next generation of omni-modal systems.

FAQ

What makes SeedRealtime different from existing voice assistants?

SeedRealtime is a native audio-visual full-duplex LLM that processes continuous audio and video streams simultaneously within a single unified model. Unlike cascaded systems that chain separate ASR, VLM, and TTS modules, or end-to-end models that still rely on external voice activity detection for turn-taking, SeedRealtime jointly models perception, understanding, decision-making, and expression in real time without discrete stages.

How does the model handle noisy environments and overlapping speech?

The model uses joint audio-visual understanding to distinguish speakers by combining voice characteristics with visual identity tracking. In the demonstrated dinner party scenario, it matched names to faces by appearance, kept each person's voice tied to their identity throughout the conversation, and could tell which line came from whom in overlapping dialogue. It also distinguishes side conversations and background noise to avoid false triggers.

What does "proactive interaction" mean in practice?

Proactive interaction means the model continuously monitors the visual environment and can initiate communication when it detects relevant changes, without waiting for a user prompt. In the museum demo, the user asked to be reminded when a specific artifact appeared; as the camera panned, the model recognized the target object and spoke up with a reminder and detailed explanation, maintaining the task context across the visual stream.

Is SeedRealtime available via API for developers?

ByteDance has stated that SeedRealtime has been "fully rolled out" but has not yet published API documentation, pricing, or developer access details. The announcement focuses on the model's capabilities and deployment milestone rather than commercial availability. Interested developers should monitor the company's official channels for future announcements.

How does SeedRealtime compare to the DuplexOmni approach from academic research?

DuplexOmni, presented in a June 2026 arXiv paper, proposes a decoupled architecture with separate interaction and thinking layers that collaborate asynchronously. SeedRealtime takes a different approach: a single unified end-to-end model handling all modalities and interaction timing. ByteDance's strategy prioritizes tight audio-visual alignment and minimal latency, while DuplexOmni aims to preserve deep reasoning capacity by offloading complex cognition to a pluggable thinking layer.

Sources