5 results for “sparse attention”
NVIDIA validated Alibaba's Qwen3.8-Flash-Next 176B MoE model on its GB300 NVL72 rack, achieving over 16K tokens per second per GPU. The hybrid GDN/QSA architecture keeps memory and compute bounded for million-token agentic coding workloads.
Z.ai released GLM-5.3-Flash, a 320B-parameter multimodal model with only 18B active parameters using hybrid sparse-linear attention. The MIT-licensed model approaches Claude Opus 4.8 on coding benchmarks at one-tenth the inference cost of its predecessor.
Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125B-parameter multimodal MoE model with 6B activated parameters and 51B N-gram embeddings. The open-weight preview of Qwen4's architecture cuts training cost to one-ninth of its predecessor while leading coding and office-task benchmarks.
Z.ai released GLM-5.2 on June 13 with 1M context, MIT weights, and Coding Plan access before benchmarks. Developers can evaluate it in agents today while GLM-5.5 remains entirely unconfirmed.
DeepSeek V4 Flash is positioned as a fast, lower-cost open-weight AI model for developers who need long-context reasoning without paying frontier-model. The model is described as a Mixture-of-Experts system with 284 billion total.