Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Diagnostic Evaluation of Machine Translation Systems Using Automatically Constructed Linguistic Check-Points

Ming Zhou, Bo Wang, Shujie Liu, Mu Li, Dongdong Zhang, and Tiejun Zhao

Abstract

We present a diagnostic evaluation platform which provides multi-factored evaluation based on automatically constructed check-points. A check-point is a linguistically motivated unit (e.g. an ambiguous word, a noun phrase, a verb~obj collocation, a prepositional phrase etc.), which are pre-defined in a linguistic taxonomy. We present a method that automatically extracts check-points from parallel sentences. By means of checkpoints, our method can monitor a MT system in translating important linguistic phenomena to provide diagnostic evaluation. The effectiveness of our approach for diagnostic evaluation is verified through experiments on various types of MT systems.

Details

Publication typeInproceedings
PublisherCOLING
> Publications > Diagnostic Evaluation of Machine Translation Systems Using Automatically Constructed Linguistic Check-Points