Exploiting Parts of Speech in Bangla-To-English Machine Translation Evaluation

Abstract

Machine translation (MT) converts one language to another automatically. One of the major challenges of MT is evaluating the performance of the system. There are many automatic evaluation metrics available these days. But the results of automatic evaluation metrics are sometimes not reliable. In this paper, we have attempted to address this issue by considering another type of evaluation strategy, i.e., syntactic evaluation in Bangla-to-English translation. We have attempted to address the problems of automatic evaluation metric BLEU and, thereby, how syntactic evaluation could be helpful in achieving higher accuracy is discussed. In our syntactic evaluation, we have exploited the use of parts of speech (POS) during computing evaluation scores. A comparative analysis is done on different types of evaluations such as syntactic, human, and automatic on a low-resourced English–Bangla language pair. A correlation indicates syntactic evaluation score correlates more with the human evaluation score compared to the normal BLEU score.