11 results for “speculative decoding”
Liquid AI released an experimental DSpark draft model for its LFM2.5-VL-3B vision-language model, delivering up to 3.13× faster decoding on Apple silicon. It ships with day-one llama.cpp, MLX-VLM, and SGLang support.
NVIDIA released Nemotron 3.5 Lightning, a 30B MoE model with 3B active parameters built for high-volume agent execution. The open model cuts inference cost for always-on agents by handling routine tool calls while frontier models focus on planning.
NVIDIA says TensorRT Edge-LLM ran Qwen3.6-27B through the MLPerf Edge Agentic workload in 24 minutes 36 seconds on one Jetson AGX Thor. The 6.4x gap over llama.cpp puts long-context agent inference on edge.
Z.ai opened GLM-5.3-FlashX as a faster API SKU of Flash — claimed ~200 tokens/s, about 2.5× Flash list price, same multimodal stack. Coding Plan still Flash-only for now.
NVIDIA's NIM 2.0.12 stack boosts Nemotron 3 Ultra throughput to 1,997 tokens per second on four B200 GPUs, a 2.5x gain at 50 TPS per user. Enterprises can replay this profile using AIPerf.
Tencent has released AngelSpec, a unified training framework designed to tackle the persistent inference efficiency challenges that accompany large. Inference cost remains one of the most significant barriers to widespread LLM adoption.
DeepSeek open-sourced its experimental V4-Flash-Vision-Exp multimodal model on Hugging Face under MIT license on August 31, 2026. The release gives developers local access to the 305B-parameter vision-language model that previously ran only on DeepSeek's API platform.
Mistral AI has released Shieldstral, a 3-billion-parameter open-weight content moderation model that the company says matches the safety performance. Content moderation has traditionally forced a trade-off between coverage and cost.
Tencent open-sourced Hy4 preview, a 770B-parameter MoE model with 49B activated weights and a 1M-token context window. The Apache 2.0 release puts a frontier-class model within reach of self-hosted inference stacks.
Meta's Superintelligence Labs has released Muse Glimmer, a 30-billion-parameter open-weight model built specifically for always-on local agent workflows. Local agentic models change the economics of AI deployment.
Z.ai released GLM-5.2 on June 13 with 1M context, MIT weights, and Coding Plan access before benchmarks. Developers can evaluate it in agents today while GLM-5.5 remains entirely unconfirmed.