Learning Analytics

The Fifth Level of the New Kirkpatrick Model, or why and how to calculate training ROI?

a young dark-haired woman in an office spaice indicating a point on a whiteboard to a shorter dark-haired woman

Businesses are frequently looking for tangible measures to help them justify investments in training programs. One of the most popular models of training evaluation is the Kirkpatrick Model, which consists of four levels of effectiveness assessment. In many companies, however, the process closes after the first level. The thing is, even four levels are not enough sometimes. This article presents another approach: the Phillips model. Why is it a good idea to use it in your organization?

What does the New Kirkpatrick Model consist of and what are its deficiencies? 

We wrote about the New Kirkpatrick Model some time ago. The model is based on four levels of training evaluation, including reaction, learning, behavior, and results. All these levels put the employee at the center. They allow evaluating tangible progress in knowledge, skills, and behavior, which can translate into a meaningful contribution to meeting the objectives of the organization. 

Are these factors enough to evaluate training effectiveness in a comprehensive manner? This largely depends on the kind of training. However, any time you get the question “How does this influence financial results”, you can make use of the Phillips model.

Let’s level up! What is the Phillips model, when was it developed, and how does it work?

The Phillips model was created in the 1970s. Jack Phillips believed that the Kirkpatrick Model was incomplete, so he extended it in cooperation with his team. The largest modification concerned adding the fifth evaluation level, namely: return on investment. It can be calculated based on regularly collected data from each of the previous levels.

This makes it possible to compare the total cost of a program with the financial profit it brings. What is crucial here is that all the benefits of the training must be taken into consideration. At the same time, factors that are not related to the training but have an impact on the company must be isolated. These factors include seasonal effects that temporarily contribute to business improvements.

Ultimately, the Phillips model takes into account four levels of performance evaluation. These are:

1. Reaction

“Are the participants satisfied with the training?”

Measuring the participants’ reaction and satisfaction in relation to the training program. 

2. Learning

“What did they learn?”

Measuring the skills by means of, for example, simulation tests or group assessments.

3. Behavior

“What are they going to do with that?”

Measuring how the employees draw from what they’ve learned in their daily work. 

4. Business impact

“Does that have an impact on the business results?”

Checking if and to what extent the KPIs improved after the training. 

5. Return on investment (ROI)

Comparing the monetary benefits with the cost of the training program. The result is a cost-benefit ratio. 

In a broader sense, the Phillips model includes the following areas:

  • planning,
  • data collection,
  • data analysis,
  • presenting the evaluation results.

How to do the maths and what to focus on?

The ROI is calculated based on the financial benefits and the cost of the program. 

For instance: let’s assume that we’re going to measure the effectiveness of a training program on safety, organized for 50 people. We know that thanks to the program, the number of accidents has been reduced by 20 throughout the year. This has directly increased the company’s revenue by $100,000 per year. The total cost of the program was $50,000 USD

So, the benefit-cost ratio (BCR) is: $100,000/$50,000 = 2. In other words, for each dollar spent on the training program, two dollars are returned.

The ROI of the program is: ($100,000 – $50,000/$90,000) x 100%
(50,000/90,000) x 100% = 55%

This means that every dollar spent on the program is returned and an additional $0.55 is returned as a profit.

These calculations may look simple, but if you don’t account for every factor that has an impact on particular data, it is easy to bias the final results. This is why the Phillips model puts emphasis on data collection at each level of evaluation. To do that, you can use tests, focus groups, or employee progress and performance monitoring. The next step involves a detailed analysis of the data and transforming the data into monetary values based on their share in the profit (if possible, of course).

The platform that will make it easier to manage the budget of L&D of your organization. 

Learn more

You need to look for measurable factors in particular, for example: process facilitation, improved productivity, or increased revenue. This will help you assess whether the business result is actually related to the program and to find out which stage was faulty, if the result is negative. For instance, you can find out that even though the participants learned something, they were not able to use that knowledge at work.

Phillips makes a point of isolating the business effects of training from other possible factors that may influence the financial results of the company. On the other hand, intangible outcomes are also taken into account: for example, the work environment may become less stressful and the employees may become more engaged. Although these factors are difficult to be measured financially, they should be still accounted for as they constitute added value. In fact, in the case of some programs, such as leadership development or communication training, these factors may be even more important.

What you need to remember, though, is that this model may not be applicable to all types of training. Nevertheless, it helps in showing how training programs support the growth of the company and how they impact its business results. Thanks to this, you gain influence on the company’s further development as well as flexibility in deciding whether a project should continue or perhaps the resources should be spent on another project. The Phillips model can also be of use in quality assurance. It can help you find out if the program meets the expected goals, see its strengths and weaknesses, and make better decisions concerning project prioritization.

Related posts
HR and L&D TrendsLearning Analytics

Who are perennials and why do modern organizations need them?

Learning Analytics

Can employee self-development be measured?

Learning Analytics

How to increase the response rate of your post-training surveys?

Learning Analytics

KPIs in L&D, indicators for measuring training effectiveness