AI Navigate

Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

arXiv cs.LG / 3/17/2026

📰 NewsIdeas & Deep AnalysisModels & Research

Key Points

  • The work uses Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories on the N-sphere with an exponential number of stored patterns M = e^{αN}.

Abstract

Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological computation. We use Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the N-sphere with an exponential number of stored patterns M = e^{\alpha N}: a log-sum-exp (LSE) kernel and a log-sum-ReLU (LSR) kernel. Both kernels share the zero-temperature critical load \alpha_c(0)=0.5, but their finite-temperature behavior differs markedly. The LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature; for typical sharpness values this threshold approaches \alpha_c, making retrieval nearly perfect across the entire load range. We also compare the measured equilibrium alignment with analytical Boltzmann predictions within the retrieval basin.