I started to write this review many times but continually failed to progress – I keep getting in the way. It would be easy just to say that this book bears the imprimatur of the American Library Association; has been funded and supported by many established bodies and individuals; is carefully compiled; excellently produced; and provides a great deal of advice to those wanting to evaluate their reference services. All very true, and as a handbook it may well become a standard. But as, once again, I am dragged to the enquiry counter to cover, yet again, for x who is off ill/y on holiday/or z who has left and not been replaced; protesting that I have a best value folder to compile; a CIPFA return to concoct; a health and safety audit to make; a resource document to study; an IT meeting to chair; even a book on reference work to review; I wonder why it is we need to evaluate reference services. Can they not hear the punters hammering on the door; see the stressed‐out staff; and wonder whatever happened to the laid‐back affable reference librarian now seen snarling at the e‐mailers? Why evaluate something when demand is so obviously high? The book, to be sure, will tell me why: if I evaluated what I did I might discover a better way of doing it; or provide evidence of why I needed more resources; or use them better; or prioritize.
A deeper reason for my malaise is a sense of déjà vu. I remember the angst caused by the unobtrusive testing farrago in the late 1970s – we even had a visit from Thomas Childers, the guru of unobtrusive testing, whose work is reported here. I am pretty sure we got tested. And I remember the irritation of being watched by a PhD student observing our non‐verbal behaviour. We saw them off, and apart from being amused that CIPFA gave up measuring reference work because the figures made no sense, things seem to have been pretty quiet since. But here we are again, this time in the guise of Jo Bell Whitlatch – “reference luminary” (blurb) – may I never be called a reference luminary! Let us see if Jo Bell can assuage my ire!
In Part I, “Planning the evaluation of reference services”, the purpose of the evaluation is covered. Reference work is defined as “personal assistance given by library staff to users seeking information”. This assistance includes answering questions, performing literature searches, instructing people in the use of reference resources, and providing advisory service to readers. It is here that I am told why my reference service needs evaluating: for economic reasons, particularly efficiency; and to assess the quality of the service, of the resources and of the products (answers and information given). We then look at evaluation goals and objectives. These might be to “train reference staff to limit initial assistance to no more than five minutes when other users are waiting”, or to “offer in‐depth reference assistance at an average cost of five dollars per question or less”. No platitudes here! Again, reference services are sub‐divided into “service provision”, “quality of resources” and “service outcomes”, a useful and practical division.
We now have to select appropriate evaluation methods and in Part II these are considered. The characteristics of good measures are outlined: they must be valid, reliable, practical and useful. Surveys and questionnaires are then described for the novice researcher. He/she is introduced to clarifying objectives, selecting a method (such as mailed questionnaires, person‐to‐person ones, telephone surveys, and focus groups – advantages and disadvantages of each method are clearly given), methods of sampling, response rates, using published surveys, design issues, pilot testing, e‐mail and on‐line surveys, and ethical considerations. A list of survey Websites is given. There is then a look at observation methods – direct observation, observers disguised as patrons, self‐observation, recording devices and, once again, there is a note on the ethical dimension. The selection of interview methods is considered, both individual interviews and focus group interviews. This part concludes with a number of case studies. Having planned the evaluation and decided on the method, we now come to Part III: “Collecting, analyzing, and communicating study data and results”. The resources needed are considered, methods of collecting and analyzing the data are outlined, and, most importantly, the evaluation report itself and its impact on reference service are discussed.
Then comes the other half of the book: 100 pages of bibliography! No mere bibliography this, but a presentation of the literature relating to reference evaluation with full annotated accounts of the books and papers noted – descriptive and evaluative. This “Reference assessment annotated bibliography” is worth dwelling on. I have long thought that we knowledge professionals are far too modest with our bibliographies. For all sorts of reasons we should give priority to the literature already “in the can”, evaluate it as it relates to our purpose, and only then, if justified, write a new book or article. Bibliographies should be a pre‐thought, not the after‐thought they so often are. Here, Jo Bell Whitlatch and her many colleagues, have performed a service of great value to their fellow professionals. This evaluative literature review, which relates closely to the structure of the book, is an impressive piece of scholarship (I spotted Kenneth Whittaker’s – UK reference libraries are right half the time – 1990 Library Review article).
This lean and practical text, the realization that reference service extends beyond front‐line counter work, a well‐thought out structure, and an understanding of the ethical issues, have, indeed, assuaged my ire. Can do – must do! An excellent book for reference and project managers’ desks.
