DOCUMENTATION NOTES: THE EFFECTIVENESS OF NON‐USER RELEVANCE ASSESSMENTS

In many information systems, intervening relevance assessments are an accepted operational practice; i.e. documents identified by the system as responses to a question are assessed as either relevant or non‐relevant, and only those judged relevant are forwarded to the user (questioner). It is commonly assumed that subject experts can perform such intervening relevance assessments with a high degree of accuracy. However, much of the literature on relevance indicates the relevance‐assessment process to be highly subjective, and hence a task that should be performed only by the user.