This is not, as might be thought from the title, a “how to do it” manual, but a detailed and comprehensive research report into current practice. It covers the third study carried out by ARL into the performance and economics of ILL operations in research, academic and government libraries in the US and Canada. Data were collected from 72 libraries, with aggregate results presented for the 59 ARL members included.
The objective of the study was to update previous work, carried out in 1996 and 1992 (Jackson, 1998; Roche, 1993), and identify characteristics of high performing libraries. Performance was measured with reference to three indicators: direct costs, fill rate and turnaround time. Traditional library‐mediated request services are contrasted with some of the available user‐initiated services.
The report gives a technical introduction, summarises and compares the performance of the seven user‐initiated services covered, and gives an overview of the performance of mediated ILL/DD operations. The performance of those libraries participating in both the 1996 and 2002 studies is compared, and the characteristics of high performing libraries are described. A weighted average indicator for total activity is briefly analysed, and the report concludes with a discussion of ways to improve local operations and a summary of findings. Several appendices give information on the participants, and the instruments used for data collection.
The study claims that “user‐initiated ILL/DD provides better service than mediated ILL/DD” (p. 10). However, this is based on data from as few as two, and no more than 11, libraries for the five user‐initiated services compared, and no indication is given of error levels or whether intra‐library effects have been controlled for in the analyses. Subsequently, the statement is qualified, noting that “actual performance varies from service to service” (p. 98).
In the analysis of mediated services, which takes up half the report, data outside the 10th and 90th percentiles are excluded. This level appears to have been set arbitrarily, rather then determined on a statistically valid basis. Data are analysed in considerable detail, resulting in small sample sizes for some tables, and comparisons are drawn with other studies reported in the literature. The impact of electronic resources is not examined in detail, but is covered in a short literature review.
A total of 44 libraries participated in the ARL studies in both 1996 and 2002, and their results for mediated services are compared for the two years. In general terms, levels of borrowing have increased, with lower unit costs, reduced turnaround times and no change in the fill rates. Unit lending costs have also fallen, after adjusting for inflation.
The report details an analysis of low cost, high‐performing mediated services, selecting the five libraries with the best performance on each of its three indicators for analysis. Again, this cut‐off appears to have been set arbitrarily, with no statistical justification presented. Interestingly, for borrowing, this method resulted in the selection of 14 different libraries, with only one having high performance in two of the three indicators considered. This would suggest that the balance between cost, turnaround time and fill rate is a delicate one, where performance in one area may be sacrificed for that in another. For lending, there was slightly more overlap, with one library appearing in the high performance group for all three indicators, and two having both low costs and fast turnaround. Detailed tables compare characteristics of the selected libraries with the ARL cohort as a whole. While there are large differences between the two groups in the average values for some measures, others are smaller, and there is no indication of error margins or the statistical significance of the results.
This was clearly a valuable study for the participating libraries, which were able to benchmark their operations against those identified as high performing in the various areas. Strategies identified as important in the 1996 study are re‐iterated, with new areas added. Key among these is increasing the use of user‐initiated services. Specific benchmarks are given as targets for services to achieve in two of the three indicators (cost is not included). Presumably these are based on the levels achieved by the “best” performing libraries (although this is not clear from the text) and it is not clear whether such levels are currently achieved simultaneously, or at reasonable cost, by any participant.
The report concludes: “Although this report presents many detailed findings, the report is not simply about the numbers. The success of this report is how librarians will apply and adapt the findings from this study. If the follow‐up from the 1996 study is any indication, ILL managers will make great progress in maximizing access while minimizing the costs associated with that access” (p. 105).
There is clearly much of value in this report, and much that will generalise beyond the context of North America.
