Rotary Positional Embedding
AssessTechniques
A positional encoding method that injects token order information through rotary transformations.
Why it's here
Placed in Assess: 3 article(s) of evidence from 2 source(s), led by model releases, with 1 in the last 30 days. Confidence 50%.
Evidence (3)
- 8Hacker News·7/28/2026model_releaseKimi K3 Architecture Highlights LatentMoE and NoPE
Kimi K3 is presented as a scaled-up production version of Kimi Linear, growing from 48B to 2.8T parameters and positioning itself as the largest open-weight model to date. The architecture emphasizes inference efficiency with components such as LatentMoE, Kimi Delta Attention, attention residuals, and a full switch to NoPE, while also adding native multimodal support.
- 7Hugging Face Blog·4/29/2026model_releaseGranite 4.1 LLMs: How They Are Built
IBM’s Granite Team details how Granite 4.1 was trained, from data engineering through supervised fine-tuning and reinforcement learning. The family includes dense decoder-only models in 3B, 8B, and 30B sizes, trained on about 15 trillion tokens and extended to a 512K context window, with the 8B instruct model reportedly matching or exceeding the prior Granite 4.0-H-Small despite using fewer parameters.
- 5Hugging Face Blog·2/3/2026researchAblation Lessons for Training Text-to-Image Models
Hugging Face publishes the second part of Photoroom’s series on training efficient text-to-image foundation models from scratch. The post reviews a baseline Flow Matching setup for the PRX-1.2B model and examines which training changes improve convergence, stability, and efficiency in practice.