Model Serving
AssessTechniques
The process of deploying machine learning models so they can respond to requests in production.
Why it's here
Placed in Assess: 2 article(s) of evidence from 2 source(s), led by research-stage coverage, with 1 in the last 30 days. Confidence 40%.
Evidence (2)
- 6Hacker News·9/17/2026researchGLM’s custom AI inference infrastructure
The article describes how GLM built its own inference infrastructure to serve its models at scale, rather than relying entirely on external or off-the-shelf systems. It focuses on the engineering choices behind latency, throughput, and operational control for model serving. The piece is notable as an inside look at AI infrastructure design from a model provider.
- 4InfoQ·8/17/2026researchPractical Patterns for LLM-Based Selection Systems
Jendrik Jördening outlines engineering practices for using LLMs in production selection pipelines. He emphasizes reducing non-determinism, constraining output schemas, separating semantic extraction from deterministic logic, and validating results with discriminator models to improve reliability and data integrity.