6 results for “TensorRT”
NVIDIA released TensorRT Model Connect in public preview, turning Hugging Face checkpoints into native C++ TensorRT bundles with two commands and no ONNX step. The aarch64-only wheels target robotics and edge teams on ARM-based NVIDIA hardware.
NVIDIA validated Alibaba's Qwen3.8-Flash-Next 176B MoE model on its GB300 NVL72 rack, achieving over 16K tokens per second per GPU. The hybrid GDN/QSA architecture keeps memory and compute bounded for million-token agentic coding workloads.
NVIDIA Dynamo now offers a preview shadow engine recovery feature that cuts LLM failover from 283 seconds to 7.3 seconds on B200 GPUs. The GPU Memory Service keeps weights resident across process crashes, letting a pre-warmed standby take over almost instantly.
NVIDIA released Nemotron 3.5 Lightning, a 30B MoE model with 3B active parameters built for high-volume agent execution. The open model cuts inference cost for always-on agents by handling routine tool calls while frontier models focus on planning.
Mistral AI has released Shieldstral, a 3-billion-parameter open-weight content moderation model that the company says matches the safety performance. Content moderation has traditionally forced a trade-off between coverage and cost.
Tencent has released AngelSpec, a unified training framework designed to tackle the persistent inference efficiency challenges that accompany large. Inference cost remains one of the most significant barriers to widespread LLM adoption.