耦合液态时间常数编码器与现代霍普菲尔德记忆
Coupling Liquid Time-Constant Encoders with Modern Hopfield Memory
把现代霍普菲尔德网络当作内容寻址记忆，与连续时间编码器并联，可作为一种通用的时序特征聚合模块，用于视频动作识别或事件相机等需要同时保留快速变化与长程上下文的视觉时序建模；其梯度收缩、降低 Hessian 迹的分析也为视觉模型优化提供思路。
原文依据 we study a simple coupling of Liquid Time-Constant Networks (LTCs) with a Modern Hopfield Network (MHN) that serves as a content-addressable memory
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论连续时间神经网络提供自适应动力学，但依赖单一隐藏状态来同时编码快速输入波动与较长程上下文。这种共享表示迫使快速变化的输入覆盖较慢的上下文信号，导致模型在新观测到来时丢失过去信息。相比之下，生物感知系统通过将持续信号与存储的联想模式整合，而非依赖单一演化状态，从而在变化的感官输入下保持稳定行为。受这一差异启发，我们研究一种简单耦合：将液态时间常数网络（LTC）与现代霍普菲尔德网络（MHN）耦合，后者充当内容寻址记忆。在每个时间步，液态状态被投影为查询，MHN 检索一个记忆向量，两个表示在读出层之前被拼接。我们在标准范数与 Lipschitz 假设下分析该耦合，并证明组合表示保持有界。我们进一步证明，检索映射会收缩记忆查询上游参数的梯度，这提供了一种降低损失曲面曲率的机制。在公开时间序列基准上，耦合的 LTC-MHN 模型相比有竞争力的循环与连续时间基线将平均准确率提升 2.3%，并将估计的 Hessian 迹相对独立 LTC 编码器降低约一个数量级，在分类任务上增益最大，在回归任务上表现相当。对训练曲线、损失曲面与潜变量嵌入的定性分析支持如下解释：霍普菲尔德检索平滑了优化，并促使类别流形更紧凑、更线性可分。代码将在论文发表后发布。
英文原文摘要 1798 字符
Continuous-time neural networks provide adaptive dynamics, but rely on a single hidden state to encode both fast input fluctuations and longer-term context. This shared representation forces rapidly changing inputs to overwrite slower contextual signals, causing the model to lose past information as new observations arrive. In contrast, biological perceptual systems maintain stable behaviour under evolving sensory input by integrating ongoing signals with stored associative patterns rather than relying on a single evolving state.Motivated by this distinction, we study a simple coupling of Liquid Time-Constant Networks (LTCs) with a Modern Hopfield Network (MHN) that serves as a content-addressable memory. At each time step, the liquid state is projected into a query, the MHN retrieves a memory vector, and the two representations are concatenated before a readout layer. We analyse this coupling under standard norm and Lipschitz assumptions and show that the combined representation remains bounded. We further show that the retrieval map contracts gradients for parameters upstream of the memory query, which provides a mechanism for reducing curvature in the loss landscape.On public time-series benchmarks, the coupled LTC-MHN model improves mean accuracy by 2.3% over competitive recurrent and continuous-time baselines and reduces the estimated Hessian trace by about an order of magnitude relative to a standalone LTC encoder, with the largest gains on classification tasks and competitive performance on a regression task. Qualitative analyses of training curves, loss landscapes, and latent embeddings support the interpretation that Hopfield retrieval smooths optimization and encourages more compact, linearly separable class manifolds. Code will be released upon publication.