Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

The paper presents GNMT, Google's Neural Machine Translation system, designed to make end-to-end NMT accurate and fast enough for practical deployment despite NMT's high training and inference cost and its difficulty with rare words. The model is a deep LSTM network with 8 encoder and 8 decoder layers using attention and residual connections, and the attention mechanism connects the bottom decoder layer to the top encoder layer to improve parallelism and reduce training time. To handle rare words, GNMT divides words into a limited set of sub-word 'wordpieces' for both input and output.

For deployment, GNMT accelerates inference with low-precision arithmetic and uses a beam search with length normalization and a coverage penalty to encourage outputs that cover the entire source sentence. On the WMT'14 English-to-French and English-to-German benchmarks it achieves results competitive with the state of the art, and in a human side-by-side evaluation on isolated simple sentences it reduces translation errors by an average of 60% relative to Google's phrase-based production system, indicating NMT was ready for real-world service.