Qualities of Literary Machine Translation: A Corpus-based Case Study

Abstract

The research assesses and compares the translation performance of two popular machine translation systems, GPT-4o and Youdao AI Translate, in translating into English ten Chinese prose essays excerpted from Selected Modern Chinese Essays 2 by Zhang Peiji. The goal is to discover their linguistic features and investigate how well they can perform in this translation. Through a corpus-based analysis, the research explores the STTR and word/ sentence length of their translations and conducts both automated and human evaluations on their translation quality. It reveals that GPT-4o exhibits higher lexical variety and both the two machine translation systems tend to produce more and shorter sentences than the human translation does. Both of them perform surprisingly well in the translation, as they get relatively high BLUE scores yet low TER scores, as well as high adequacy and fluency rates. Our evaluation results also show that Youdao AI Translate displays generally better performance than GPT-4o in the translation of Chinese-to-English literary texts, and they can complement each other to achieve even better performance, though a certain amount of errors are still present in both of their translations.