3 results for “LLM serving”
Z.ai published a blog post describing custom inference infrastructure for its GLM models, framed as a step toward recursive self-improvement. No performance data or hardware details were disclosed.
NVIDIA Dynamo now offers a preview shadow engine recovery feature that cuts LLM failover from 283 seconds to 7.3 seconds on B200 GPUs. The GPU Memory Service keeps weights resident across process crashes, letting a pre-warmed standby take over almost instantly.
NVIDIA hand-delivered its first Vera CPUs to Anthropic, OpenAI, SpaceXAI, and Oracle Cloud Infrastructure in mid-May 2026, correcting earlier reports of late August shipments. The 88-core Olympus processor targets agentic AI workloads where CPU orchestration gates GPU utilization.