| Summary: | "NIST MetricsMATR is a series of research challenge events for machine translation (MT) metrology, promoting the development of innovative, even revolutionary, MT metrics that correlate highly with human assessments of MT quality. In this program, participants submit their metrics to the National Institute of Standards and Technology (NIST). NIST runs those metrics on certain held-back test data for which it has human assessments measuring quality and then calculates correlations between the automatic metric scores and the human assessments. This release contains the development data received by participants in NIST Metrics for Machine Translation 2008 Evaluation (MetricsMATR08). Specifically, this corpus is comprised of a subset of the materials used in the NIST Open MT06 evaluation and includes human reference translations, system translations, and human assessments of adequacy and preference. The source data consists of twenty-five Arabic language newswire documents with a total of 249 segments. The data in each segment includes four human reference translations in English and system translations from eight different MT06 machine translation systems. In addition to the data and reference translations, this release inlcudes software tools for evaluation and reporting and documentation describing how the human assessments were obtained and how they are represented in the data. The evaluation plan contains further information and rules on the use of this data." -- LDC catalogue.
|
|---|
