LREC 2016 Workshop: “Translation evaluation – From fragmented tools and data sets to an integrated ecosystem”

This workshop will take an in-depth look at an area of ever-increasing importance: approaches, tools and data support for the evaluation of human translation (HT) and machine translation (MT), with a focus on MT. Two clear trends have emerged over the past several years. The first trend involves standardising evaluations in research through large shared tasks in which actual translations are compared to reference translations using automatic metrics and/or human ranking. The second trend focuses on achieving high quality translations with the help of increasingly complex data sets that contain many levels of annotation based on sophisticated quality metrics – often organised in the context of smaller shared tasks. In industry, we also observe an increased interest in workflows for high quality outbound translation that combine Translation Memory (TM)/Machine Translation and post-editing. In stark contrast to this trend to quality translation (QT) and its inherent overall approach and complexity, the data and tooling landscapes remain rather heterogeneous, uncoordinated and not interoperable.