A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures

This review surveys recurrent neural networks and, in particular, the long short-term memory (LSTM) cell. The authors explain that RNNs built from simple sigma or tanh cells are unable to learn relevant information when the gap between inputs is large, and that introducing gate functions into the cell structure gives the LSTM its ability to handle long-term dependencies well. They note that since its introduction, nearly all the notable results based on RNNs have been achieved with the LSTM, making it a focus of deep learning.

The review examines the LSTM cell and its variants to explore its learning capacity, and organizes LSTM networks into two broad categories: LSTM-dominated networks and integrated LSTM networks. It discusses the various applications of these architectures across domains and closes by presenting future research directions for LSTM networks, serving as a consolidated reference on the cell and its architectural landscape.