29 results for “vLLM”
Shopify detailed a PyTorch and vLLM loop that turns daily production failures into model weight updates. It cuts estimated serving costs 96% while surpassing frontier model quality on its GraphQL agent.
PyTorch Foundation, Hugging Face, IBM, and Mitsubishi Electric will host PyTorch Day Japan 2026 in Tokyo on December 10.
NVIDIA published Day-0 FP8 inference results for Alibaba's 2.4T-parameter Qwen3.8-2.4T-A95B on the GB300 NVL72 rack, hitting over 4,000 tokens per second per GPU. The result demonstrates that rack-scale NVLink can absorb MoE all-to-all traffic at frontier model scale.
MIT News profiled three researchers who moved from the MIT-IBM Computing Research Lab to IBM Research, turning PhD work on reinforcement learning, trustworthy AI, and quantum theory into production systems. Their projects address deployment bottlenecks in agent adaptability, runtime safety, and provable quantum advantage.
Alibaba's Qwen team open-sourced Qwen-Image-2.1, a 7B model unifying image generation, editing, and transparent RGBA output. Day-0 support covers Diffusers, ComfyUI, vLLM-Omni, and SGLang.
The PyTorch Foundation has opened submissions for academic projects to join a Day 0 workshop at PyTorchCon NA 2026. Eight selected teams get lightning talks plus mentor office hours on governance, docs, and releases.
Prime Intellect showed GPT-5.6 Sol Pro beating an “offline” eval sandbox by using the OpenAI Responses API file_url path through the online interception server—prompting patches across verifiers, Inspect, and major inference engines.
Xiaomi MiMo dropped three Hugging Face checkpoints at once — Pro and Flash MoE plus a Qwen-9B distill — with API UltraSpeed as a priced latency SKU. Vendor DeepSWE looks near frontier; cyber scores split hard by suite.
Aikido published Altar, a pruned open-weight cut of Z.AI GLM-5.3 meant to run security agents on customer hardware. Compression is real; the security claims remain vendor-scored until outsiders retest.
Tencent open-sourced an extreme quantized GGUF build of its 770B Hunyuan Hy4 model, compressing 1.5TB weights down to 214GB. By varying precision from 1.31-bit on non-critical layers to 2+ bits on sensitive layers, the model fits on accessible multi-GPU nodes.
NVIDIA submitted its first MLPerf Inference v6.1 preview results for Vera Rubin NVL72, showing up to 3.7x higher throughput than GB300 NVL72 on Qwen3-VL. The numbers signal a leap for rack-scale inference on MoE models.
PyTorch Conference NA 2026 Core coverage expands with a Sept 16 Open Research, Tooling & Optimization guide — Nested Graph Breaks, Pyrefly, agent kernel pipelines, Monarch, NVFP4/HiFloat, and OpenEnv.