sglang
AssessTools
An inference serving framework for large language models with support for advanced decoding features.
Why it's here
Placed in Assess: 4 article(s) of evidence from 3 source(s), led by research-stage coverage, with 2 in the last 30 days. Confidence 55%.
Evidence (4)
- 8Hacker News·8/12/2026model_releaseQwen3.8 Open-Model Release
Qwen has released Qwen3.8-2.4T-A95B, a post-trained open model available in Hugging Face Transformers format and compatible with tools such as vLLM and SGLang. The model is positioned as the most capable Qwen open-model generation to date, with improvements in coding, agentic task execution, long-context handling, and flexible reasoning control.
- 7Hacker News·8/3/2026researchCloudflare scales Kimi and GLM with KV cache quantization and weight compression
Cloudflare says it is serving Moonshot's Kimi K-series and Z.ai's GLM more efficiently on Workers AI by combining KV cache quantization, weight compression, and cache protection techniques. The company reports that FP8 KV cache and INT4 weights cut GPU memory use and lower costs while preserving model accuracy, using SGLang as the serving framework.
- 7Hugging Face Blog·7/7/2026product_launchHugging Face models arrive on Microsoft Foundry Managed Compute
Microsoft announced that curated Hugging Face open-weight models are now available on Foundry Managed Compute, with weekly refreshes and one-click deployment. The offering adds enterprise controls such as security scanning, governance, observability, billing, and support for managed GPU deployment of open-source and custom models.
- 6Hacker News·7/3/2026researchWafer reports faster GLM5.2 inference on AMD MI355X
Wafer says it ran GLM5.2 on AMD MI355X with 2626 tokens per second per node at 2.4 RPS, while claiming the setup cost is more than 2x lower than Blackwell-based alternatives. The post also describes using MXFP4 quantization with AMD Quark and serving the model with sglang on ROCm after fixing speculative decoding support.