An applied ethnographic method for evaluating retrieval features

This paper reports on a study conducted to develop a framework for evaluating the retrieval features using an applied ethnographic method. Direct observation, interviews, analysis of notes, and informal social interaction were done with ten users on their application of the retrieval features and their difficulties in searching. Retrieval features evaluated were those offered by 12 database providers. Findings revealed that the proposed framework successfully gathered the data needed. Application of the features was related to users' retrieval tasks, preference and style of searching, and understanding of the features. Difficulties were related to identification of the appropriate search terms. Expected retrieval features were related to search terms, i.e. relevance feedback, list of similar terms, and assigning values to search terms. Applied ethnographic method used in this study revealed that users have a substantial amount of knowledge about the retrieval features, and that their comments were related to their subject background.