int4 quantization
AssessTechniques
A 4-bit quantization method used to reduce model memory usage.
Why it's here
Placed in Assess: 3 article(s) of evidence from 3 source(s), led by open-source activity, with 1 in the last 30 days. Confidence 54%.
Evidence (3)
- 6The New Stack·9/30/2026model_releaseCohere's Embed 5 fast query model slightly lowers retrieval quality
Cohere released Embed 5, letting teams index data with Embed 5 Pro and query the same vectors with the faster Embed 5 Fast without re-embedding. In Cohere's tests, Fast queries against a Pro index reached 98.4% of the Pro-to-Pro retrieval baseline, while using Fast for both indexing and queries scored 96.6%.
- 7Hacker News·8/3/2026researchCloudflare scales Kimi and GLM with KV cache quantization and weight compression
Cloudflare says it is serving Moonshot's Kimi K-series and Z.ai's GLM more efficiently on Workers AI by combining KV cache quantization, weight compression, and cache protection techniques. The company reports that FP8 KV cache and INT4 weights cut GPU memory use and lower costs while preserving model accuracy, using SGLang as the serving framework.
- 6Hacker News·7/9/2026open_sourceColibrì runs GLM 5.2 on a slow laptop by streaming MoE experts from disk
A Hacker News post introduces Colibrì, a single-file C engine designed to run the GLM 5.2 mixture-of-experts model on modest hardware. The author says the system keeps the dense model parts in RAM at int4 precision and streams routed experts from disk, enabling use on a 32 GB laptop without running out of memory.