Mixture of Experts
AdoptTechniques
A sparse neural network architecture that routes each token to a small subset of expert sub-networks.
Why it's here
Placed in Adopt: 15 article(s) of evidence from 4 source(s), led by model releases, with 6 in the last 30 days. Confidence 83%.
Evidence (15)
- 7Hacker News·8/5/2026product_launchDiscovery Loop launches to automate scientific and engineering discovery
Discovery Loop describes a system for automating experimental loops, using frontier AI models and large-scale compute to propose, run, and evaluate experiments in parallel. The company says it will first focus on machine learning research and engineering, then expand toward broader science and engineering problems.
- 8The New Stack·8/3/2026model_releaseAlibaba unveils Qwen3.8-Max for long-running agentic tasks
Alibaba has launched Qwen3.8-Max, a multimodal model with 2.4 trillion total parameters designed for complex tasks that can run for days. The company says the model can be accessed through QwenCloud and Alibaba Cloud Model Studio, with downloadable weights planned for release next week and public GitHub traces from a 16-day coding demo. Alibaba has not yet provided enough detail for independent verification of its long-running benchmark results.
- 7Hacker News·7/23/2026open_sourcePetals lets people run large language models at home
Petals is a distributed inference and fine-tuning system that lets users run parts of large language models on consumer GPUs and share the rest over a peer network, BitTorrent-style. It supports models such as Llama 3.1, Mixtral, Falcon, and BLOOM, with API-like usability and access to lower-level model behavior for customization.
- 7Hacker News·7/21/2026model_releasePoolside releases Laguna S 2.1 coding model
Poolside has released Laguna S 2.1, a 118B-parameter Mixture-of-Experts coding model with 8B active parameters per token and support for up to a 1M-token context window. The company says it is optimized for longer-horizon agentic coding work and reports strong results on benchmarks such as Terminal-Bench 2.1, SWE-Bench Multilingual, SWE-Bench Pro, DeepSWE, SWE Atlas, and Toolathlon Verified.
- 8Simon Willison·7/16/2026model_releaseKimi K3 launches as an open 3T-class model
Kimi has introduced Kimi K3, a 2.8-trillion-parameter open model with native vision support and a 1-million-token context window. The company says it is optimized for long-horizon coding, reasoning, and knowledge work, with full weights planned for release on July 27, 2026.
- 8Hugging Face Blog·7/15/2026model_releaseInkling Open-Weights Multimodal Model Released
Thinking Machines has released Inkling, an open-weights Mixture-of-Experts model trained from scratch with full weights available for customization. The model supports up to 1M tokens of context, processes text, images, and audio natively, and is available for fine-tuning on Tinker, alongside a smaller Inkling-Small preview.
- 6Hacker News·7/9/2026open_sourceColibrì runs GLM 5.2 on a slow laptop by streaming MoE experts from disk
A Hacker News post introduces Colibrì, a single-file C engine designed to run the GLM 5.2 mixture-of-experts model on modest hardware. The author says the system keeps the dense model parts in RAM at int4 precision and streams routed experts from disk, enabling use on a 32 GB laptop without running out of memory.
- 8Hugging Face Blog·7/8/2026framework_updateTransformers backend in vLLM reaches native speed
Hugging Face says the transformers modeling backend in vLLM now matches or exceeds the throughput of hand-written native vLLM implementations for several Qwen3 models. The update lets model authors run Hugging Face Transformers models in vLLM with a single flag while keeping standard parallelism and serving workflows intact.
- 8Simon Willison·7/6/2026model_releaseTencent releases Hy3 open model
Tencent has released Hy3, an Apache 2.0-licensed Mixture-of-Experts model with 295B total parameters and 21B active parameters. The company says it was improved with feedback from more than 50 products and now offers strong performance on utility and productivity tasks, with free access on OpenRouter until July 21.
- 7Hacker News·6/30/2026model_releaseLongCat-2.0: 1.6T MoE model with 48B active parameters
LongCat-2.0 is a large-scale mixture-of-experts language model announced with 1.6 trillion total parameters and 48 billion active parameters per forward pass. The release highlights a design aimed at scaling model capacity while keeping inference compute lower than a dense model of comparable size.
- 7Hugging Face Blog·6/24/2026framework_updateNVIDIA NeMo AutoModel boosts MoE fine-tuning in Transformers v5
Hugging Face describes how NVIDIA NeMo AutoModel builds on Transformers v5 to improve fine-tuning performance for Mixture-of-Experts models. Using the same from_pretrained() API, it adds Expert Parallelism, DeepEP all-to-all dispatch, and TransformerEngine kernels, delivering higher throughput and lower GPU memory use.
- 8Simon Willison·6/17/2026model_releaseGLM-5.2 open weights model lands with 1M context
Z.ai has released GLM-5.2 as open weights under an MIT license after an earlier limited rollout to coding subscribers. The text-only Mixture-of-Experts model has a 1 million token context window and is being reported as the leading open weights model on independent benchmarks, though it is relatively token-hungry.
- 6Hugging Face Blog·6/1/2026model_releaseJetBrains introduces Mellum2, a 12B MoE model
JetBrains has released Mellum2, an open 12B-parameter mixture-of-experts model trained from scratch on natural language and code. It activates only 2.5B parameters per token, is designed for low-latency, high-throughput inference, and is released under the Apache 2.0 license. The company says it is suitable for routing, RAG, summarization, sub-agents, coding workflows, and private deployment.
- 7Hugging Face Blog·2/26/2026researchMixture of Experts in Transformers
Hugging Face explains how Mixture of Experts (MoE) architectures replace dense feed-forward layers with sparse expert routing to improve compute efficiency. The post highlights why MoEs can deliver dense-model quality with fewer active parameters at inference time, and notes increasing adoption in recent open models and the Transformers ecosystem.
- 6Hugging Face Blog·1/27/2026researchChina’s Open-Source AI Ecosystem Moves Beyond DeepSeek
Hugging Face’s article surveys how China’s open-source AI community has shifted in 2025 toward Mixture-of-Experts architectures, broader multimodal systems, and smaller models that are easier to deploy and fine-tune. It also notes increasing use of Chinese hardware and a focus on reusable engineering assets such as inference stacks, datasets, evaluation tools, and agent workflows.