Hierarchical Attention Networks for Document Classification

The paper proposes a hierarchical attention network for document classification with two distinctive characteristics. It has a hierarchical structure that mirrors the hierarchical structure of documents, building sentence representations from words and document representations from sentences, and it applies two levels of attention mechanisms at the word and sentence levels. This design enables the model to attend differentially to more and less important content when constructing the document representation, rather than treating all words and sentences equally.

Experiments conducted on six large-scale text classification tasks demonstrate that the proposed architecture outperforms previous methods by a substantial margin. Visualization of the attention layers illustrates that the model selects qualitatively informative words and sentences, offering interpretability into which content drives the classification alongside its accuracy gains.