Niyra's memory, measured in public
LongMemEval, turn-level, production code, zero tuning — 98.2% recall@10. And what that number does not mean.
Most memory benchmarks are marketing dressed as science. So we ran Niyra's production memory stack — the exact code serving users — against the complete LongMemEval benchmark. One shot, no tuning, every caveat published.
The numbers. All 500 questions, 246,738 conversation turns: 98.2% recall@10, 92.0% recall@5, MRR 0.723. Turn-level granularity — the exact gold turn among ~490 candidates, roughly ten times finer than the session-level numbers most systems quote. Knowledge-update questions: 100%.
The conditions. Per-record isolation. Production defaults, first full-scale attempt, no parameter sweeps. Temporal reasoning measured without ingesting timestamps — a self-imposed handicap, disclosed, still 96.2%.
The line I won't cross. Recall is not answer accuracy. These numbers say the right memory was found and ranked near the top; they say nothing about the sentence the model then wrote. Never compare the two.
The full report, scoring harness, run log, and a reproduce path (one OpenAI key, no Niyra code) are public in the niyra-memory-bench repo. The long write-up — category table, the knowledge-update fix, every caveat — is on the Niyra blog. If you think we measured something wrong, open an issue.