{"id":"src_m9dn54gbzeun","title":"Long Short-Term Memory","authorName":"Sepp Hochreiter, Jürgen Schmidhuber","abstract":"Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly due to insufficient, decaying error backflow. We briefly review Hochreiter's analysis of this problem, then address it by introducing a novel, efficient, gradient based method called Long Short-Term Memory. LSTM can learn to bridge time intervals in excess of 1000 steps even in case of noisy, incompressible input sequences, without loss of short time lag capabilities. This is achieved by an efficient, gradient based algorithm for an architecture enforcing constant error flow through internal states of special units. Key points are constant error carousels and multiplicative gate units that learn to open and close access to the constant error flow.","tags":["recurrent neural networks","lstm","sequence learning","deep learning","memory"],"priceUSDC":"0.0001"}