Ways to Evaluate Training Programs – Part IV
I’m continuing to share some riches from How Do They Know They Know: Evaluating Adult Learning by Jane Vella, Paula Berardinelli, and Jim Burrow. It’s a great book to help you assess any type of training program that you are leading or helping out with.
Results that can be evaluated
- Learning – changes in the learner’s knowledge, skill and attitudes that result from the training
- Transfer – learning from the training that is applied in the learner’s work after completing the education or training program
- Impact – improvement in the performance of the learner’s organization as a result of the learner’s work (22)
Qualitative Evaluation – examines the qualities and characteristics of a setting or experience to develop an understanding and determine meaning. Are the training goals worth achieving? Scriven recommended that evaluators use “goal free” evaluation because, by discussing the program with participants, it might be possible to uncover unintended effects as well as ways to make the program more responsive to the learners (29).
Quantitative Evaluation – are structured, precise, and focused on identifying and studying specific elements through measurement (30).
Here’s the acid test:
Are they showing other people how to do it?
I get about a 20% return on that one so far – up from 1%! Working on it!
That is true. That is a great improvement. Awesome.
This book is great in that it offers a good basic framework for tracking Learning, Transfer and Impact, but with particular emphasis on how well the participants meet the Achievement-Based Objectives (one of Vella’s unique contributions).
However, the field of learning evaluation has developed a bit further since Vella et al wrote this one. (e.g. we need to pay attention to the contextual factors that affect how learners apply what they learned (Transfer) (e.g. using Kurt Lewin’s work)
Global Learning Partners, the company that Jane Vella founded, offers a Learning Evaluation by Design course that builds on what this book talks about. (Full disclosure: I used to work for them). I’d also recommend looking at a recent version of their e-newsletter that addresses Transfer. See http://www.globalearning.com/voices/2008fall/voices.htm
Pax, dh
Dwayne,
Thanks for your helpful comments and the links on transfer. I appreciate it.