7 results for “llama.cpp”
Hugging Face added GGUF support to Transformers, letting developers load llama.cpp checkpoints directly on Apple Silicon Macs. The integration reuses ggml Metal kernels and keeps the generation loop in PyTorch.
NVIDIA says TensorRT Edge-LLM ran Qwen3.6-27B through the MLPerf Edge Agentic workload in 24 minutes 36 seconds on one Jetson AGX Thor. The 6.4x gap over llama.cpp puts long-context agent inference on edge.
Tencent open-sourced an extreme quantized GGUF build of its 770B Hunyuan Hy4 model, compressing 1.5TB weights down to 214GB. By varying precision from 1.31-bit on non-critical layers to 2+ bits on sensitive layers, the model fits on accessible multi-GPU nodes.
DeepSeek open-sourced its experimental V4-Flash-Vision-Exp multimodal model on Hugging Face under MIT license on August 31, 2026. The release gives developers local access to the 305B-parameter vision-language model that previously ran only on DeepSeek's API platform.
Tencent open-sourced Hy-MT1.5-1.8B packs that shrink a 3.3 GB translation model to as little as about 440 MB for offline use on ordinary phones. The 1.25-bit and 2-bit releases keep 33-language coverage while targeting on-device privacy and real-time translation without a network.
Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125B-parameter multimodal MoE model with 6B activated parameters and 51B N-gram embeddings. The open-weight preview of Qwen4's architecture cuts training cost to one-ninth of its predecessor while leading coding and office-task benchmarks.
Liquid AI has released the LFM2.5 model family, a comprehensive suite of open-weight models designed to run entirely on edge devices without cloud. The LFM2.5 releases address three critical constraints that have limited on-device AI.