The RNN Homunculus: Learning to Gate Long-Term Memory
Summary
The RNN Homunculus: Learning to Gate Long-Term Memory
Shared on Bluesky by 2 AI experts
-
I really enjoy finding gaps in our understanding of text-book deep learning. The RNN Homunculus: Long-range input vanishes RNN gradients. Gates (GRU, LSTM) help, but gates are themselves an RNN. Ie: gates are not suffic…
View on Bluesky →
Originally reported by arxiv.org
Read the original article →Original headline: The RNN Homunculus: Learning to Gate Long-Term Memory