Often times when conducting a high-level evaluation , educators are tempted to bypass the lower levels . However , lower-level data provide important information that can be helpful in adjusting for future program implementation . For example , when the desired outcome for a program is behavior change ( level 3 ), educators will often bypass level 1 ( reaction ) and level 2 ( learning ) and focus only on measuring changes in behavior ( level 3 ). Then again , suppose that no change in behavior occurred ; since no data was gathered at levels 1 and 2 , it is difficult to determine where the chain of impact was broken . Perhaps , the chain was broken early on at level 1 , such that participants had a negative reaction to the program ( e . g ., they felt that the program was not relevant to their job , the instructor / facilitator was not knowledgeable or effective in helping them learn , the learning environment was not conducive to learning ). If this was the case , then participants are less likely to learn ( level 2 ). No changes in participants ’ knowledge , skills and / or attitudes ( level 2 ) will lead to no changes to their on-the-job behavior ( level 3 ). While positive reaction may not ensure learning , negative reaction almost certainly reduces the possibility of it occurring .
Another possibility is that the chain of impact was broken at level 3 , in that the workplace environment did not support the application of the newly acquired knowledge , skills and / or attitudes . Three of the most frequently reported barriers to the application are manager support , lack of time , and lack of opportunity to apply . Thus , collecting lower-level data will help educators diagnose where improvements can be made to a program to increase the likelihood of achieving desired outcomes .
Like Phillips , Kaufman also modified Kirkpatrick ’ s model by adding a fifth level – Society Outcomes ( Kaufman & Keller , 1994 ). With this fifth level , organizations are encouraged to measure the societal consequences and payoffs of their programs . For instance , manufacturing organizations are increasingly being called to account for unintended societal consequences such as increases in pollution , while educational communities are under more pressure to demonstrate positive benefits to society such as increasing employment opportunities and decreasing crime rates .
Figure 1 depicts the relationship between stakeholder needs ( left side ) and levels of evaluation ( right side ) based on the work of Kirkpatrick , Phillips and Kaufman . In addition to producing a balanced set of data that addresses the different needs and perspectives of stakeholders , the Kirkpatrick , Phillips and Kaufman evaluation approaches link different categories of value by demonstrating a connection between different levels of data .