Reproducible scores across the benchmarks that actually measure agent memory. Open methodology. Pick your own benchmark. Submit your own score.
8 systems, 6 benchmarks. CORTEX V2.4, run 2026-04-22T22:22Z.
Atlan MCP metadata, text-to-SQL | –6 | –6 | –6 | –6 | –6 | –6 |
Cognee | – | – | – | – | – | – |
CORTEX V2.4, ATERNA | 100.0% | 93.6% | 78.5% | pending7 | 40.1%8 | pending9 |
HippoRAG 2 OSU + UIUC | – | – | – | – | – | – |
Letta formerly MemGPT | – | 74.0% | – | – | – | – |
Mastra Observational Memory | 94.9%1 | – | – | – | – | – |
Mem0 | 93.4%1 | 91.6%1 | – | no lift2 | 64.1% | decline3 |
Zep Graphiti | +18.5%4 | 58.4%5 | – | no lift2 | – | – |
Six benchmarks, six different questions. Each one below names its paper, what it tests, and why it is on the board.
500 long-conversation questions across 5 question types. Tests retrieval plus single-hop reasoning. Saturated for retrieval systems in 2026 but still the canonical entry point.
10 multi-session dialogues averaging 300 turns. Categories: single-hop, multi-hop, temporal, open-domain. Published retrieval scores are comparable across systems.
Cognitive-memory capability suite: temporal validity, reconsolidation, novelty detection, emotional recall, cross-agent transfer, compounding intelligence, procedural learning. Designed for memory systems, not LLM context.
Slack, Linear and Git event streams. Correctness, efficiency, redundancy. Zep and Mem0 do not significantly improve GPT-5 on this benchmark, per the paper.
First benchmark with single-user narrative coherence at 1M and 10M tokens. Ten abilities including contradiction resolution and event ordering. The LIGHT framework uses an explicit working-memory component.
First operation-level hallucination benchmark for memory systems. Three stages: extraction, updating, QA. Mem0, Mem0-Graph and Memobase all show marked decline on long context, per the paper.
Send raw JSON and reproduction instructions to benchmarks@aterna.ai. We list every verifiable result, including ones that beat CORTEX. This is a public scoreboard, not a marketing page.
git clone https://github.com/ATERNA-AI/cortex cd cortex && npm install npx tsx benchmarks/longmemeval/run.ts --topk 10 --dataset oracle npx tsx benchmarks/locomo/run-retrieval.ts --topk 10 npm run cogbench:run
In a 20-minute live demo we show CORTEX recalling real client context, then map what it would do for your team.