Mixture of Experts
AdoptTechniques
A sparse neural network architecture that routes each token to a small subset of expert sub-networks.
Why it's here
Placed in Adopt: 20 article(s) of evidence from 4 source(s), led by model releases, with 5 in the last 30 days. Confidence 88%.
Evidence (15)
- 8Hacker News·10/5/2026model_releaseReflection unveils Beam, a 501B open-weight coding model
Reflection introduced Beam, a sparse Mixture-of-Experts open-weight model with 501 billion total parameters and 23 billion active parameters. The company says it is optimized for coding, reasoning, and agentic workloads, with training that combined large-scale pretraining and high-compute reinforcement learning. The weights, technical report, model card, and developer artifacts are scheduled for release later this month.
- 7Hacker News·10/3/2026model_releaseAleph Alpha’s Kolibri: How the Sovereign German LLM Works
Kolibri is an open-weight mixture-of-experts language model from Aleph Alpha, built for German and English with 78.1 billion total parameters but only 3.46 billion active per token. The model was trained from scratch in Germany and Finland, released under Apache 2.0, and designed with sovereignty, compliance, and long-context use cases in mind.
- 4Hacker News·10/2/2026open_sourceds4 Lets Users Run LLMs Locally
The item introduces ds4, a local inference setup for running large language models on user machines. It highlights asymmetric 2-bit quantization and routed-MoE compression to reduce model size while preserving accuracy on critical shared paths.
- 7Hugging Face Blog·10/1/2026framework_updateAi2 releases Olmo-core 3 for scalable MoE training
Ai2 has released Olmo-core 3, an upgraded training framework for large language models with a redesigned open mixture-of-experts (MoE) system. The new stack is built to scale toward trillion-parameter MoE models while improving throughput, including benchmarks showing higher token processing rates than the earlier implementation.
- 8Hugging Face Blog·9/28/2026model_releaseHugging Face launches Holo4 generalist computer-use agents
Hugging Face introduced Holo4, a new agentic model series for computer-use workflows, in 27B dense and 35B-A3B Mixture-of-Experts variants. The models are designed to work across GUIs, code, MCP, and APIs, with accompanying releases including Holotron4 Nano, trajectories, and API access.
- 7Hacker News·8/5/2026product_launchDiscovery Loop launches to automate scientific and engineering discovery
Discovery Loop describes a system for automating experimental loops, using frontier AI models and large-scale compute to propose, run, and evaluate experiments in parallel. The company says it will first focus on machine learning research and engineering, then expand toward broader science and engineering problems.
- 8Hacker News·8/3/2026model_releaseAlibaba unveils Qwen3.8-Max for long-running agentic tasks
Alibaba has launched Qwen3.8-Max, a multimodal model with 2.4 trillion total parameters designed for complex tasks that can run for days. The company says the model can be accessed through QwenCloud and Alibaba Cloud Model Studio, with downloadable weights planned for release next week and public GitHub traces from a 16-day coding demo. Alibaba has not yet provided enough detail for independent verification of its long-running benchmark results.
- 7Hacker News·7/23/2026open_sourcePetals lets people run large language models at home
Petals is a distributed inference and fine-tuning system that lets users run parts of large language models on consumer GPUs and share the rest over a peer network, BitTorrent-style. It supports models such as Llama 3.1, Mixtral, Falcon, and BLOOM, with API-like usability and access to lower-level model behavior for customization.
- 7Hacker News·7/21/2026model_releasePoolside releases Laguna S 2.1 coding model
Poolside has released Laguna S 2.1, a 118B-parameter Mixture-of-Experts coding model with 8B active parameters per token and support for up to a 1M-token context window. The company says it is optimized for longer-horizon agentic coding work and reports strong results on benchmarks such as Terminal-Bench 2.1, SWE-Bench Multilingual, SWE-Bench Pro, DeepSWE, SWE Atlas, and Toolathlon Verified.
- 8The New Stack·7/16/2026model_releaseKimi K3 launches as an open 3T-class model
Kimi has introduced Kimi K3, a 2.8-trillion-parameter open model with native vision support and a 1-million-token context window. The company says it is optimized for long-horizon coding, reasoning, and knowledge work, with full weights planned for release on July 27, 2026.
- 8Hacker News·7/15/2026model_releaseInkling Open-Weights Multimodal Model Released
Thinking Machines has released Inkling, an open-weights Mixture-of-Experts model trained from scratch with full weights available for customization. The model supports up to 1M tokens of context, processes text, images, and audio natively, and is available for fine-tuning on Tinker, alongside a smaller Inkling-Small preview.
- 6Hacker News·7/9/2026open_sourceColibrì runs GLM 5.2 on a slow laptop by streaming MoE experts from disk
A Hacker News post introduces Colibrì, a single-file C engine designed to run the GLM 5.2 mixture-of-experts model on modest hardware. The author says the system keeps the dense model parts in RAM at int4 precision and streams routed experts from disk, enabling use on a 32 GB laptop without running out of memory.
- 8Hugging Face Blog·7/8/2026framework_updateTransformers backend in vLLM reaches native speed
Hugging Face says the transformers modeling backend in vLLM now matches or exceeds the throughput of hand-written native vLLM implementations for several Qwen3 models. The update lets model authors run Hugging Face Transformers models in vLLM with a single flag while keeping standard parallelism and serving workflows intact.
- 8Simon Willison·7/6/2026model_releaseTencent releases Hy3 open model
Tencent has released Hy3, an Apache 2.0-licensed Mixture-of-Experts model with 295B total parameters and 21B active parameters. The company says it was improved with feedback from more than 50 products and now offers strong performance on utility and productivity tasks, with free access on OpenRouter until July 21.
- 7Hacker News·6/30/2026model_releaseLongCat-2.0: 1.6T MoE model with 48B active parameters
LongCat-2.0 is a large-scale mixture-of-experts language model announced with 1.6 trillion total parameters and 48 billion active parameters per forward pass. The release highlights a design aimed at scaling model capacity while keeping inference compute lower than a dense model of comparable size.