On August 26, 2026, Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125-billion-parameter multimodal mixture-of-experts model with 6 billion parameters activated per token and an additional 51 billion N-gram embedding parameters. The model is available now on Hugging Face and ModelScope under the qwen-community-1.0 license, with a technical report and citation details published on GitHub.
The release serves as an early architectural preview of Qwen4. Qwen3.8-Flash-Next introduces a hybrid Gated DeltaNet plus Qwen Sparse Attention (QSA) design, Gated Residual streams with four branches, N-gram embeddings indexed by bigrams and trigrams at layer 2, and a tailored Muon/AdamW optimizer recipe that eliminates traditional batch-size warmups. Native context length is 262,144 tokens, extensible to 1 million with YaRN.
What's new
- Architecture: Hybrid GDN + QSA attention; Gated Residual (4 branches, bottleneck rank 320); N-gram Embedding (20M entries, offloadable to host memory); Muon optimizer with refitted scaling laws.
- Scale: 125B total parameters, 6B activated per token, 51B N-gram embeddings, 4B MTP parameters, 48 layers, hidden dimension 2560.
- MoE configuration: 512 experts, 10 routed + 1 shared activated, intermediate dimension 640.
- Context: 262,144 tokens native, extensible to 1,000,000.
- Deployment: Transformers, vLLM, SGLang, TokenSpeed, llama.cpp (GGUF), Unsloth; OpenAI-compatible APIs via Qwen Cloud.
- Pricing (production Qwen3.8-Flash via Qwen Cloud): $0.16 per 1M input tokens, $0.47 per 1M output tokens.
Benchmarks and capabilities
Qwen reports that Qwen3.8-Flash-Next was trained at roughly one-ninth the cost of Qwen3.7-Plus while surpassing it across the board, with particular strength in coding and office-task benchmarks. Highlights from the technical report include 58.7 on DeepSWE 1.1 (best on mini-SWE-agent), 62.5 on SWE-bench Pro, 81.0 on SWE-bench Multilingual, 73.9 on CoWorkBench for long-horizon office work, 55.7 on JobBench for professional tasks, 73.5 on Toolathlon Verified for real-world tool use, 91.7 on GPQA Diamond for scientific reasoning, and 91.9 on LiveCodeBench v6 for competitive coding. On multimodal tasks, the model scores 95.7 on MathVision with chain-of-thought and 84.5 on AndroidWorld.
Why it matters
The release gives the open-weight community a concrete, runnable instance of the hybrid attention and residual designs that will underpin Qwen4, months before that family arrives. Because the model activates only 6B parameters per token despite a 125B backbone, it reduces inference cost dramatically — a pattern that could shift deployment economics for agents and long-context workloads. The N-gram embedding approach also offers a parameter-scaling lever that is less compute-intensive than traditional MoE and more amenable to memory offloading, which matters for developers targeting consumer GPUs or constrained cloud instances.
Our take
Qwen3.8-Flash-Next is a rare case where an architecture preview ships as a usable, benchmarked model rather than a paper alone. The 1/9 training-cost claim versus Qwen3.7-Plus is striking, but independent replication on diverse hardware and workloads will determine whether the efficiency gains hold outside Alibaba's training stack. The production API pricing ($0.16/$0.47 per 1M tokens) positions Qwen3.8-Flash aggressively against closed-model vendors for high-volume agentic workloads.
