Are Automatic Metrics Robust and Reliable in Specific Machine Translation Tasks?

Por favor, use este identificador para citar o enlazar este ítem:
Información del item - Informació de l'item - Item information
Título: Are Automatic Metrics Robust and Reliable in Specific Machine Translation Tasks?
Autor/es: Chinea-Rios, Mara | Peris, Álvaro | Casacuberta, Francisco
Palabras clave: Machine Translation
Área/s de conocimiento: Lenguajes y Sistemas Informáticos
Fecha de publicación: 2018
Editor: European Association for Machine Translation
Cita bibliográfica: Chinea-Rios, Mara; Peris, Álvaro; Casacuberta, Francisco. “Are Automatic Metrics Robust and Reliable in Specific Machine Translation Tasks?”. In: Pérez-Ortiz, Juan Antonio, et al. (Eds.). Proceedings of the 21st Annual Conference of the European Association for Machine Translation: 28-30 May 2018, Universitat d'Alacant, Alacant, Spain, pp. 89-98
Resumen: We present a comparison of automatic metrics against human evaluations of translation quality in several scenarios which were unexplored up to now. Our experimentation was conducted on translation hypotheses that were problematic for the automatic metrics, as the results greatly diverged from one metric to another. We also compared three different translation technologies. Our evaluation shows that in most cases, the metrics capture the human criteria. However, we face failures of the automatic metrics when applied to some domains and systems. Interestingly, we find that automatic metrics applied to the neural machine translation hypotheses provide the most reliable results. Finally, we provide some advice when dealing with these problematic domains.
Patrocinador/es: The research leading to these results has received funding from the Generalitat Valenciana under grant PROMETEO/2018/004.
ISBN: 978-84-09-01901-4
Idioma: eng
Tipo: info:eu-repo/semantics/conferenceObject
Derechos: © 2018 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC-BY-ND.
Revisión científica: si
Versión del editor:
Aparece en las colecciones:Congresos - EAMT2018 - Proceedings

Archivos en este ítem:
Archivos en este ítem:
Archivo Descripción TamañoFormato 
ThumbnailEAMT2018-Proceedings_11.pdf1,88 MBAdobe PDFAbrir Vista previa

Este ítem está licenciado bajo Licencia Creative Commons Creative Commons