Study of lexical automatic machine translation evaluation metrics for Indic languages

Abstract

This research paper aims to study and compare different lexical automatic machine translation evaluation metrics for Indic languages. As machine translation systems have grown in popularity, it is now crucial to assess the accuracy of the translations these systems generate. However, the existing evaluation metrics designed for English and other European languages may not be suitable for Indic languages due to their complex morphology and syntax. Therefore, this study evaluates four different metrics, namely, BLEU, METEOR, TER, and NIST to identify the most suitable evaluation metric for Indic languages. The study uses datasets for three Indic languages, namely, Hindi, Bengali, and Telugu, and evaluates the metrics on various translation models. The study advances the field of machine translation by offering guidance on appropriate metrics for evaluating languages that are Indic.