THE EXTENSION OF USERS' LITERATURE AWARENESS AS A MEASURE OF RETRIEVAL PERFORMANCE, AND ITS APPLICATION TO MEDLARS

The performance of a retrieval system with a file of only a few hundred references can be measured by assessing the relevance of each reference to each of a number of queries. A suitable measure of retrieval performance is then the Recall ratio—the fraction of the relevant references that are retrieved by the system. When the file is large this method of measuring performance is not practicable, and Recall cannot be measured although it can be estimated. A number of estimation procedures are examined and found unsatisfactory. Another measure of retrieval performance is the Extension ratio which is approximately the ratio of the quantity of known relevant references before and after putting a query to the retrieval system. The properties of this measure are examined and it is applied to 315 MEDLARS searches.