Generating Sequences With Recurrent Neural Networks

The paper demonstrates that Long Short-term Memory (LSTM) recurrent neural networks can generate complex sequences containing long-range structure by learning to predict one data point at a time. The method is applied to discrete data in the form of text and to real-valued data in the form of online handwriting, and is then extended to handwriting synthesis by allowing the network to condition its predictions on a given text sequence.

The resulting system produces highly realistic cursive handwriting in a wide variety of styles, showing that a single next-step prediction approach can capture long-range structure across both discrete and continuous data domains. This established recurrent networks as a general and effective technique for modeling and synthesizing structured sequential data.