Google announces Neural Machine Translation

Google has announced an improvement to Google Translate, which uses neural network technology to improve the process of machine translation. Google Neural Machine Translation (GNMT) is an end-to-end learning system, created to overcome the weaknesses inherent in the phrase-based systems that are currently used for machine translation.

Phrase-based machine translations break an input sentence into words or phrases that are then translated individually to create the output – the same sentence in a different language. This method uses a basic algorithm to break down the input, and create the output.

However, non-continuous phrases (such as the French ne … pas) are difficult for existing systems to handle and translate correctly without human intervention, or dense programming requirements.

Neural Machine Translation instead considers the entire sentence as a whole unit for translation. While NMT systems and Recurrent Neural Networks (RNN) have been researched for some time as an alternative to phrase-based machine translations, the improvements that were made in translations were outweighed by sacrifices to speed and difficulty translating rare words among other things.

In a research paper released today, Google researchers have outlined the steps taken to overcome these obstacles in order to make GNMT a workable alternative to the phrase-based Google Translate.

First, the team built a deep Long Short Term Memory (LSTM) network using eight encoder layers, and eight decoder layers. Then they connected the bottom layer of the decoder to the top layer of the encoder, which improved parallelism and helped to decrease training time.

They also improved the overall speed of translation by using low-precision arithmetic for computations. Finally, the Google team improved machine translation of rare words by breaking them down into their component parts, or ‘sub-word’ units.

The GNMT system was found to provide both flexibility and efficiency, with an overall improvement in the accuracy of translations provided. When compared to state-of-the-art systems using English-French and English-German benchmarks, GNMT was found to have competitive results. When compared to a human translation of simple sentences, it was found to have 60% fewer errors than the previous phrase-based version of Google translate.

In addition, Google announced that effective immediately, GNMT would be used exclusively by Google Translate to execute the ‘notoriously difficult’ Chinese-to-English machine translations for Google mobile and web apps – an estimated 18 million translations per day.