Progress in Machine Translation – Backtranslation examples then and now

Ten years ago, I viewed machine translation more as a joke than as a useful translation tool. Thus I included an amusing machine translation experiment in the humour section of my website:

A Folksinger Christine Lavin had investigated the results of automatic translation of some of her song titles to and from a foreign language. The idea was to translate to another language and then retranslate the result back to English. This procedure is known as “backtranslation”, and it sometimes used by a translation client who has no knowledge of the target language as a way of guaging the quality of the translation by reading the backtranslation in their own language. In this case Lavin chose the song title “Please Don’t Make Me Too Happy” and did the backtranslation in 5 different languages using the Altavista automatic translation tool (The left-hand column in the table below).

I didn’t record the date when this was done, but my memory tells me it must have been about ten years ago. Today, August 28, 2012, I did the same exercise using Google Translate and Bing. As you can see, things have improved greatly, except for Bing in German. On the other hand, Bing did somewhat better in some cases than Google. The sample is too small as a quality estimate for any purpose other than amusement. We will come back to the question of measuring machine translation quality in a future article.