Using Evaluation in Training and Development
Using Evaluation in Training and Development
Leslie RaeKogan PageLondon1999240 pp.ISBN 0749428058 (Paperback)£19.99
Keywords Training, Evaluation
As a trainer and consultant who recognizes the importance of evaluation in the process of training, I am also conscious of how easy it can be to relegate the subject to one of secondary consideration. This book by Leslie Rae provides a practical model of evaluation with a number of associated tools and techniques. The book follows a clear structure based on the model presented in Chapter 2 which suggests that there are 14 key stages to the process of evaluation. While this might sound like overkill, Rae quite rightly makes the point that coverage of the subject often focuses on only a few of the key stages of the evaluation process. It is recommended that evaluation should be considered right from the start when a training need is identified and should then be considered at each stage of the training process through training to end of programme testing and medium-term and long-term follow-up. A test on evaluation would not be complete without the classic old model of Kirkpatrick(reactions, learning, behaviour and results) and this is given brief coverage.
A high level of credence is given to the UK National Vocational Qualification system and process as applied to the development of trainers. This tends to give the book a rather British bias and I would challenge some of the statements made suggesting that the majority of trainers will be seeking NVQ qualifications. Some of the best trainers I have seen are capable of consulting rather than just training and NVQs have played little if any part in their lives.
Rae provides us with the idea of the training quintet which suggests that various parties have an interest in the evaluation of training, namely the learner, the trainer, the training manager, the line manager and senior manager. He challenges the protectiveness of some trainers towards training activity and explains how this often presents barriers between the training professional and the line manager.
By about Chapter 3, I realized that the main emphasis was being placed on the evaluation of training programmes or courses rather than considering learning in the broader sense. I was wondering about the "development" aspect mentioned in the book's subtitle. As I read on, I rather hoped for some coverage of evaluation as related to the learning activities that take place within the workplace rather than simply through off-job training courses. For instance, how does one evaluate the effectiveness of increasingly popular methods such as coaching and mentoring?
Common problems are highlighted such as line managers failing to provide pre-programme briefing and post-programme discussion. There is some useful,simple and very practical coverage of the construction of tests relating to evaluation. Included here is some discussion around the observation and evaluation of behaviour. The techniques are sound though fairly traditional,such as interview analysis and group behaviour analysis, and I feel an opportunity was missed in terms of covering some of the more leading edge and more recent developments in this area. For instance, organizations are increasingly using 360-degree feedback in a variety of ways which are particularly germane to the evaluation of behaviours.
Practical guidance is given regarding the use of learning logs which can be a real aid to encourage managers to reflect on learning. I was a little surprised that there was not more discussion regarding the theory of how people learn,given the focus on evaluation.
Suitable criticisms are given regarding the limitation of the "happiness sheets" which are described here as reactionnaires. It is my experience that this sadly remains the most used method for supposedly evaluating the effectiveness of training programmes and indeed trainers.
Towards the end of the book, some specific guidance is given regarding the evaluation of open and distance learning programmes. There is also some useful discussion of approaches to costing and valuing training and to methods of presenting data regarding the evaluation of training.
In summary, then, I would suggest that we have here a solid Kogan Page "how to" text. The style is very practical with lots of checklists and questionnaires. The structure is logical working through the recommended model of evaluation in a mechanistic way. This will provide plenty of support to the trainer who has to address the subject of evaluation particularly in relation to training courses and events. My only criticisms, which no doubt reflect my own bias, are that there is surprisingly little coverage of evaluation as related to development, there is no discussion regarding the value or evaluation of action learning and I would have liked some case studies provided from specific organizations.
Richard HaleDirector of Consulting, IMC Consulting
