Evaluation of MT software and methods

In this paper we first discuss the wide variety of possible scenarios in which an evaluation may be carried out. On this basis, an attempt is made to pick out some general characteristics relevant to the design of evaluations and the search for a general framework for evaluation methodologies is motivated. Some resources for use in data collection for evaluation purposes are briefly described and discussed, and the work of the EEC EAGLES Evaluation and Assessment Group is summarized.