llama.cpp

Evidence (17)

• 6Hacker News·9/30/2026product_launch
  Launch HN: Magnitude, a self-optimizing local inference engine for agents

  Magnitude is a newly launched open source inference engine for local agents that tunes itself on the target hardware to improve speed and memory efficiency. The team claims it can run on Mac, Linux, and Windows and reports benchmark gains over llama.cpp for both decode and prefill on tested hardware.

• 7Hugging Face Blog·9/24/2026model_release
  LiquidAI releases DSpark draft model for LFM2.5-VL-3B

  LiquidAI released an experimental DSpark drafter for its vision-language model LFM2.5-VL-3B, adding speculative decoding to speed up inference without changing output quality. The company reports decode speedups of up to 3.13x on-device and 2.66x on H100, with end-to-end gains up to 2.62x and 2.27x, and day-one support in llama.cpp, MLX-VLM, and SGLang.

• 2Hacker News·9/23/2026open_source
  Parody post shows a 25-line Python Jev classifier

  This Hacker News item presents a parody implementation of “Jev” as a small local Python script that loads a GGUF model and classifies an email into labels such as Legitimate, Spam, or Phishing. The post argues that the approach is simple, fast, and privacy-preserving because it runs locally without sending data to an external API.

• 2Hacker News·9/17/2026product_launch
  Show HN: A community for sharing AI setups

  A Hacker News post introduces a community site for people to share and compare their AI development setups, including tools, workflows, and agent configurations. The creator says the goal is to make it easier to learn from others and keep one’s own setup up to date.

• 6Hacker News·9/8/2026research
  Qwen3.8 27B 4-bit quantization keeps benchmark quality

  A benchmark comparison of Qwen3.8 27B quantizations found that the 4-bit Q4_K_M model performs essentially the same as BF16 on GPQA Diamond, IFBench, and Terminal-Bench 2.1 while using far less GPU memory. Performance drops sharply at very aggressive compression, with 1-bit quantization collapsing to near-random results on GPQA Diamond and degrading reasoning further on longer tasks.

• 8NVIDIA GenAI·9/3/2026product_launch
  NVIDIA Brings Local AI, Faster Inference, and New RTX Spark PCs to IFA 2026

  NVIDIA announced new local AI features and hardware partnerships at IFA 2026, including simplified setup for agent apps on NVIDIA GPUs, faster inference for popular local stacks, and a new personal AI routing tool. The company also said compact RTX Spark Windows PCs from Lenovo and Acer will arrive in October, alongside support for several newly launched models and video-generation tools optimized for NVIDIA hardware.

• 7Hugging Face Blog·8/20/2026model_release
  LFM2.5 DSpark Draft Models Deliver Faster Inference

  Hugging Face and Liquid AI released DSpark draft checkpoints for three LFM2.5 models, adding speculative decoding to improve inference speed without changing output quality. The company reports up to 3.18x higher GPU throughput, up to 2.87x on-device speedups, and a 57% average reduction in function-calling latency for LFM2.5-2.6B, with day-one support in llama.cpp and SGLang.

• 7Hacker News·8/19/2026model_release
  Unsloth releases Dynamic v3.0 GGUF quants for Qwen3.8-27B

  Unsloth released Dynamic v3.0 GGUF quantizations for Qwen3.8-27B, claiming better top-1% accuracy than other providers at the same model size. The update improves calibration, layer selection, and post-training quantization methods, and is compatible with inference engines such as llama.cpp and Unsloth Desktop.

• 6Hugging Face Blog·8/19/2026model_release
  Liquid AI releases QAD Q4_0 checkpoints for LFM2.5

  Liquid AI released updated Q4_0 GGUF checkpoints for four LFM2.5 models using quantization-aware distillation. The company says the checkpoints preserve Q4_0 memory and speed while recovering most of the BF16 accuracy lost in quantization, and they are available for use with llama.cpp and other GGUF-compatible runtimes.

• 6Hacker News·8/12/2026open_source
  llama.cpp promotes fully local AI on personal hardware

  llama.cpp is presented as an open-source, privacy-focused way to run frontier AI models entirely on local hardware, without API keys, telemetry, or cloud dependency. The post also highlights installation options, local coding-agent integration via pi-llama and Pi, and broad support across CPUs and GPUs from laptops to clusters.

• 6Hacker News·8/11/2026open_source
  Apple Silicon macOS VMs Speed Up LLM Inference

  The article describes using GPU passthrough on macOS virtual machines running on Apple Silicon to improve local large language model inference performance. It highlights llama.cpp as the inference engine and shows that access to the host GPU can significantly reduce latency compared with a fully virtualized setup.

• 8Simon Willison·8/10/2026model_release
  Meta releases Muse Glimmer, a local multimodal open-source model

  Meta has released Muse Glimmer, a 30B-parameter multimodal model distilled for local, agentic use cases and licensed under Apache 2.0. Hugging Face highlights day-one support across transformers, llama.cpp, vLLM, and Inference Endpoints, along with benchmarks and architecture details showing its focus on privacy-aware coding, document analysis, and personal assistant workflows.

• 8Hugging Face Blog·8/4/2026model_release
  Hugging Face introduces LFM2.5-2.6B for on-device agents

  Hugging Face announced LFM2.5-2.6B, a 2.6B-parameter model designed to run capable agents entirely on-device with tool calling and multi-step workflows. The model is positioned for laptops and phones, emphasizing privacy, low memory use, and strong benchmark performance relative to much larger models.

• 4Hacker News·7/13/2026research
  Benchmarking Old Tesla GPUs on Modern AI and Rendering Workloads

  A hardware enthusiast benchmarked 15 retired NVIDIA Tesla enterprise GPUs, including K80, P100, and V100 cards, across modern workloads such as ResNet50, vision transformers, Blender, and llama.cpp. The article argues that these low-cost, end-of-life GPUs can still be useful in homelab settings if users accept higher power consumption and older software stacks.

• 8Hacker News·7/3/2026model_release
  Leanstral 1.5 boosts Lean 4 proof engineering

  Leanstral 1.5 is a new free Apache-2.0 licensed model from Mistral AI focused on formal verification and proof engineering in Lean 4. It reports major benchmark gains, including saturating miniF2F, solving 587/672 PutnamBench problems, and setting new results on FATE-H and FATE-X, while also finding bugs in open-source codebases.

• 6Hacker News·6/29/2026model_release
  Qwen 3.6 27B as a strong local development model

  The article argues that Qwen 3.6 27B is the most compelling local model the author has tried, outperforming previous local options on constrained writing and coding tasks. It also describes how to run the model locally with llama.cpp and a GGUF quantized build from Hugging Face for interactive chat and agent workflows.

• 6Hugging Face Blog·5/27/2026product_launch
  Reachy Mini can now run fully locally

  Hugging Face announced that Reachy Mini’s conversation stack can now run entirely on local hardware instead of sending audio to a server. The setup uses a cascaded speech-to-speech pipeline with llama.cpp, Silero VAD, Parakeet-TDT 0.6B v3 for STT, and Qwen3-TTS, enabling private offline conversations with the robot.