MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios

arXiv cs.AI / 4/17/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that existing long-term memory evaluations for LLMs are overly static, focusing on simple retrieval and short-context inference instead of capturing dynamic, interactive memory behavior.
It introduces MemGround, a gamified long-term memory benchmark built on rich interactive scenarios to evaluate three layers of memory: Surface State Memory, Temporal Associative Memory, and Reasoning-Based Memory.
MemGround includes a hierarchical, three-tier framework with specialized tasks designed to test how models track state, associate events over time, and perform hierarchical reasoning during continuous interactions.
The authors propose multiple metrics—QA Overall, Memory Fragments Unlocked (MFU), Memory Fragments with Correct Order (MFCO), and Exploration Trajectory Diagrams (ETD)—to measure both memory usage and the evolution of agent behavior.
Experiments show that even state-of-the-art LLMs and memory agents remain weak at sustained dynamic tracking, temporal event linking, and multi-step reasoning based on long-term accumulated evidence in interactive settings.

Abstract

Current evaluations of long-term memory in LLMs are fundamentally static. By fixating on simple retrieval and short-context inference, they neglect the multifaceted nature of complex memory systems, such as dynamic state tracking and hierarchical reasoning in continuous interactions. To overcome these limitations, we propose MemGround, a rigorous long-term memory benchmark natively grounded in rich, gamified interactive scenarios. To systematically assess these capabilities, MemGround introduces a three-tier hierarchical framework that evaluates Surface State Memory, Temporal Associative Memory, and Reasoning-Based Memory through specialized interactive tasks. Furthermore, to comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite comprising Question-Answer Score (QA Overall), Memory Fragments Unlocked (MFU), Memory Fragments with Correct Order (MFCO), and Exploration Trajectory Diagrams (ETD). Extensive experiments reveal that state-of-the-art LLMs and memory agents still struggle with sustained dynamic tracking, temporal event association, and complex reasoning derived from long-term accumulated evidence in interactive environments.