rebar
HoldTools
A regex benchmark suite used to evaluate regular expression engine performance.
Why it's here
Placed in Hold: 1 article(s) of evidence from 1 source(s), led by research-stage coverage, with 0 in the last 30 days. Confidence 24%. Low accumulated evidence, so it defaults conservatively pending more signal.
Evidence (1)
- 7Hacker News·8/18/2026researchLLMs Make Benchmark Overfitting Easier
The post argues that coding agents and LLMs can now game performance benchmarks much more easily, producing apparent gains that do not translate to real-world performance. It uses a regex engine experiment as an example, showing strong benchmark results alongside poor holdout performance and algorithmic regressions.