6 results for “inference optimization”
Tencent has released AngelSpec, a unified training framework designed to tackle the persistent inference efficiency challenges that accompany large. Inference cost remains one of the most significant barriers to widespread LLM adoption.
NVIDIA's NIM 2.0.12 stack boosts Nemotron 3 Ultra throughput to 1,997 tokens per second on four B200 GPUs, a 2.5x gain at 50 TPS per user. Enterprises can replay this profile using AIPerf.
Tencent open-sourced an extreme quantized GGUF build of its 770B Hunyuan Hy4 model, compressing 1.5TB weights down to 214GB. By varying precision from 1.31-bit on non-critical layers to 2+ bits on sensitive layers, the model fits on accessible multi-GPU nodes.
Z.ai opened GLM-5.3-FlashX as a faster API SKU of Flash — claimed ~200 tokens/s, about 2.5× Flash list price, same multimodal stack. Coding Plan still Flash-only for now.
Tencent open-sourced Hy4 preview, a 770B-parameter MoE model with 49B activated weights and a 1M-token context window. The Apache 2.0 release puts a frontier-class model within reach of self-hosted inference stacks.
ByteDance has officially launched SeedRealtime, a native audio-visual full-duplex large language model that unifies audio, video, and text within. The launch represents a significant shift in how multimodal AI systems handle real-world.