Subquadratic Sparse Attention
AssessTechniques
A sparse attention mechanism intended to reduce attention compute and scale closer to linearly with context length.
Why it's here
Placed in Assess: 1 article(s) of evidence from 1 source(s), led by model releases, with 0 in the last 30 days. Confidence 24%. Low accumulated evidence, so it defaults conservatively pending more signal.
Evidence (1)
- 6The New Stack·7/2/2026model_releaseSubquadratic details sparse-attention model and longer-term architecture plans
Subquadratic has released benchmarks and a model card for SubQ 1.1 Small, a sparse-attention model designed to handle very long context windows more efficiently than dense transformers. The company says the model performs strongly on long-context retrieval while using far less compute, and it is also preparing non-attention architectures for future systems.