Cognitive memory, measured.

Reproducible scores across the benchmarks that actually measure agent memory. Open methodology. Pick your own benchmark. Submit your own score.

8 systems, 6 benchmarks. CORTEX V2.4, run 2026-04-22T22:22Z.

Fig. 1   The leaderboard, sorted by system name ↑
Sort by
Atlan MCP
metadata, text-to-SQL
LongMemEval oracle, R@10, saturated
–6
LoCoMo retrieval, R@10
–6
CogBench composite
–6
MEMTRACK multi-source state
–6
BEAM 1M token
–6
HaluMem op-level hallucination
–6
Cognee
LongMemEval oracle, R@10, saturated
–
LoCoMo retrieval, R@10
–
CogBench composite
–
MEMTRACK multi-source state
–
BEAM 1M token
–
HaluMem op-level hallucination
–
CORTEX
V2.4, ATERNA
LongMemEval oracle, R@10, saturated
100.0%
LoCoMo retrieval, R@10
CogBench composite
78.5%
reproduce
MEMTRACK multi-source state
pending7
BEAM 1M token
40.1%8
HaluMem op-level hallucination
pending9
HippoRAG 2
OSU + UIUC
LongMemEval oracle, R@10, saturated
–
LoCoMo retrieval, R@10
–
CogBench composite
–
MEMTRACK multi-source state
–
BEAM 1M token
–
HaluMem op-level hallucination
–
Letta
formerly MemGPT
LongMemEval oracle, R@10, saturated
–
LoCoMo retrieval, R@10
74.0%
CogBench composite
–
MEMTRACK multi-source state
–
BEAM 1M token
–
HaluMem op-level hallucination
–
Mastra
Observational Memory
LongMemEval oracle, R@10, saturated
94.9%1
LoCoMo retrieval, R@10
–
CogBench composite
–
MEMTRACK multi-source state
–
BEAM 1M token
–
HaluMem op-level hallucination
–
Mem0
LongMemEval oracle, R@10, saturated
93.4%1
LoCoMo retrieval, R@10
91.6%1
CogBench composite
–
MEMTRACK multi-source state
no lift2
BEAM 1M token
64.1%
HaluMem op-level hallucination
decline3
Zep
Graphiti
LongMemEval oracle, R@10, saturated
+18.5%4
LoCoMo retrieval, R@10
58.4%5
CogBench composite
–
MEMTRACK multi-source state
no lift2
BEAM 1M token
–
HaluMem op-level hallucination
–
  1. 1.LLM-judge accuracy metric. Not directly comparable to retrieval R@10. See the note on metrics below.
  2. 2.Per the MEMTRACK paper (Patronus AI, NeurIPS 2025): Zep and Mem0 do not significantly improve LLM performance on this benchmark. arXiv:2510.01353
  3. 3.Per the HaluMem paper (MemTensor, Nov 2025): Mem0, Mem0-Graph and Memobase all show marked decline on long-context variants. arXiv:2511.03506
  4. 4.Zep reports percentage lift over baseline only; absolute R@10 unpublished. arXiv:2501.13956
  5. 5.Zep's original 84% LoCoMo claim corrects to 58.44% per issue #5 on their public zep-papers repo. We list the corrected number.
  6. 6.Atlan's published evaluations focus on metadata-grounded text-to-SQL accuracy (reported 3x improvement over a non-Atlan baseline). No agent-memory benchmark scores published.
  7. 7.CORTEX on MEMTRACK: the adapter is complete. The score is withheld until Patronus releases the official dataset, so the comparison against Mem0 and Zep runs on the same data.
  8. 8.CORTEX on BEAM: the official BEAM-1M set (35 conversations, 700 questions). Retrieval, then Haiku 4.5 synthesis, then a Haiku 4.5 judge. The raw judged run is linked from the cell.
  9. 9.CORTEX on HaluMem: the adapter is planned and no score exists yet. The cell fills when a run has been published under /bench.

What each benchmark measures.

Six benchmarks, six different questions. Each one below names its paper, what it tests, and why it is on the board.

i.

LongMemEval oracle, R@10, saturated

Wu et al., ICLR 2025

500 long-conversation questions across 5 question types. Tests retrieval plus single-hop reasoning. Saturated for retrieval systems in 2026 but still the canonical entry point.

ii.

LoCoMo retrieval, R@10

Maharana et al., ACL 2024

10 multi-session dialogues averaging 300 turns. Categories: single-hop, multi-hop, temporal, open-domain. Published retrieval scores are comparable across systems.

iii.

CogBench composite

ATERNA, 2026

Cognitive-memory capability suite: temporal validity, reconsolidation, novelty detection, emotional recall, cross-agent transfer, compounding intelligence, procedural learning. Designed for memory systems, not LLM context.

iv.

MEMTRACK multi-source state

Patronus AI, NeurIPS 2025

Slack, Linear and Git event streams. Correctness, efficiency, redundancy. Zep and Mem0 do not significantly improve GPT-5 on this benchmark, per the paper.

v.

First benchmark with single-user narrative coherence at 1M and 10M tokens. Ten abilities including contradiction resolution and event ordering. The LIGHT framework uses an explicit working-memory component.

vi.

HaluMem op-level hallucination

MemTensor, Nov 2025

First operation-level hallucination benchmark for memory systems. Three stages: extraction, updating, QA. Mem0, Mem0-Graph and Memobase all show marked decline on long context, per the paper.

On metrics
LLM-judge scores (Mem0, Mastra, Supermemory on LongMemEval) and retrieval R@10 scores (CORTEX) are not directly comparable. Retrieval recall asks "did the system surface the right memory." LLM-judge asks "did the end-to-end answer match ground truth." CORTEX prioritizes retrieval R@10 because a memory system that cannot surface the right memory cannot be made correct by any downstream LLM. We publish both when available.
On saturation
LongMemEval oracle top-10 is saturated for 2026-era retrieval systems. That is why we are shipping BEAM, MEMTRACK and HaluMem adapters: discriminating benchmarks that test state tracking, cross-source coherence and operation-level hallucination rather than session-level retrieval accuracy.
On competitor data
We do not re-run competitor systems. Where a competitor has published a score, we cite it with a source link. Where they have not, the cell shows a dash. Corrected numbers (Zep LoCoMo) are shown with a footnote and a link to the correction thread.

Run a memory system we missed? Post a score.

Send raw JSON and reproduction instructions to benchmarks@aterna.ai. We list every verifiable result, including ones that beat CORTEX. This is a public scoreboard, not a marketing page.

Benchmark snapshot
github.com/ATERNA-AI/cortex
Archived at v2.4 so these scores stay reproducible. Current CORTEX ships inside ATERNA agents.
Reproduce locally
git clone https://github.com/ATERNA-AI/cortex
cd cortex && npm install
npx tsx benchmarks/longmemeval/run.ts --topk 10 --dataset oracle
npx tsx benchmarks/locomo/run-retrieval.ts --topk 10
npm run cogbench:run
Dataset downloads and the full steps are in BENCHMARKS.md.
Build
CORTEX V2.4
run 2026-04-22T22:22Z

See the memory behind the numbers.

In a 20-minute live demo we show CORTEX recalling real client context, then map what it would do for your team.