HKUST Survey Maps In-Parameter Memory Methods for LLMs Across Embedding, Attention, FFN and Hybrid

A new arXiv survey led by HKUST's Haoyu Huang et al. (Oct 6) organizes in-parameter memory augmentation - encoding post-pretraining knowledge directly into weights, adapters or other parameter-like objects - along two axes: parameter placement (embedding, attention, FFN or hybrid) and acquisition time (online vs offline). The authors argue parametric memory complements in-context learning by cutting context overhead, and flag open issues around interference, safety and recursive self-improvement.