Measuring the Effectiveness of Training4 Aug, 2010 By: Teresa Hiatt, Ricoh Americas Corporation imageSource
Measuring the Effectiveness of Training
The American Society for Training and Development (www.astd.org) research tells us that top companies (including those in the office industry) will spend more than $1,500 per employee per year for education opportunities. Also, Fortune Magazine’s 100 Best Companies to Work For lists development opportunities as an important indicator to attract and retain top employees. And the latest estimate of the size of the Training Industry is about $135 billion dollars. All of this research confirms to us that leading organizations and key management think that training employees is an important part of business today. Yet despite the time, effort and money companies spend for training and development, training is rarely held accountable for results. Fortunately, the industry has five accepted level s of training evaluation that can be used to measure the effectiveness of training and ensure getting the most bang for the buck.
All five levels of training evaluation do not have to be used for every program. Sometimes, levels one and two are enough, depending on the critical nature of the training delivered. For example, training delivered to meet compliance regulations does not need to be measured as closely as sales training meant to improve revenue, because compliance training is both less expensive and less likely to have immediate bottom line impact. The more strategic the training initiative, the higher level we should use to evaluate the program.
Levels of Training Evaluation
Level One: At the end of the training, did the students (attendees) feel it was worthwhile?
Level Two: Did the students master any new skills, meaning can they pass a test or successfully perform a role-play after training?
Level Three: Did the students actually use the new skills once they returned to the office?
Level Four: When they applied the skills, was there any bottom line financial business impact?
Level Five: Was the bottom line impact enough to pay for the fully loaded costs of the training program and still leave any Return on Investment?
Let’s look a little deeper into the five levels using a Sales Training Program as an example. Let’s imagine for a moment that a vendor has delivered an intensive 5-day Sales Training workshop for 25 random sales people six months ago and you would like to know whether you should roll it out to your entire organization. What measurements should be available to determine program value?
Level One: When the training workshop was delivered, the vendor should have collected student feedback on the last day of class. For measurement purposes, the questions should have related to how the students felt about the possibility of applying what they learned to their job, as opposed to less useful questions about the food served or did they like the training. A good question on a Level One survey would be “Do you feel you will be able to make more sales as a result of this program?” Students should have felt positive about this on the evaluation sheet.
Level Two: During the design phase of the training program, the instructional designer should have asked ‘How will we know if students have mastered the exits skills of the program?” In sales training programs, this is typically done by peer-reviewed role-plays, where sales people try out the new skills and get coaching on improving their skills. It could also involve a test or other instructor assessment tool. A majority of the students should have displayed mastery of the skills.
Level Three: A provision must be made for follow-up of the training program to see if the sales people are using the skills from the class once they get back in the field, preferably within the first 90 days after training. This can be a follow-up training class, a Computer Based Training (CBT) module, or a survey to both the student and his/her manager asking about details on how the program is being implemented. Obviously, no matter how great the training program, if sales people do not apply the skills, the program is a failure. There should be clear evidence students are using what they learned.
Level Four: Many programs include measurement at the first three levels, but it is at Level Four that the cut is made between average sales training programs and good training programs. At this level, data is gathered to truly measure if the program was successful. The metrics for measurement at this level should be agreed upon before the training course is ever designed, since the bottom line impact desired determines the design of the program. Is it just raw sale revenue that needs increased, or profit margin, or pipeline opportunities or size of orders? Once the metrics are decided, there must be first a benchmark taken of the “as-is” numbers, so that the new numbers can be compared against prior performance. There must also be a method to isolate the effects of the training program from other market factors such as seasonal peaks, product launches, etc. This measurement should show positive results.
Many vendors will exclude this step, saying it is difficult to work with the business measures they do not control. But this is the “proof in the pudding” and it should be part of every training program where substantial dollars are spent to improve bottom line results. There are a variety of methods that can be used for measuring at this level and information regarding how to do this is found in the programs available at the ROI Institute http://www.roiinstitute.net/.
Level Five: Once the financial impact is known from Level Four, it’s now time to count up all the costs of training, including development costs, travel, student time, program delivery costs, evaluation costs, etc. The fully loaded costs can be subtracted from the benefits and the result divided by costs in the formula below to yield the true Return on Investment. ROI should be well in excess of 100% in a successful program. The most important thing to remember about training evaluation is to Just Do It. So many programs are developed, delivered and retired without any measurement or records kept to judge value, simply because it is neither expected nor required of the training provider. Whether your training is done internally or outsourced, make sure your next program includes as many levels of training evaluation as possible.
Teresa Hiatt is the Director of Sales Education at Ricoh Americas Corporation. She is also the current President of the Professional Society of Sales and Marketing Training (www.smt.org).