...Do Reaction Scores Matter? Let's Discuss

TL:DR - ...yes, in context

With 2022 review season in full swing many of us Learning & Development pros are filling out our reviews and looking to make the long laundry list of accomplishments we did during the previous year. As we should - we've earned it!

But this process does beg a key L&D question - what impact was made? Intrinsically and inherently we all know that training makes an impact - but many struggle to quantify it in a meaningful way. And we've all been in the position where we are asked to detail how we have moved the needle - or perhaps why we need such a budget or headcount to accomplish our goals.

But alas - unbeknownst to over over-scrutinizing colleagues we have a silver bullet we pull out from beneath our cloak - the post-learning experience survey! We can show how 97% of attendees felt their experience was "Satisfactory" or "Very Satisfactory" and we can proudly share how 82% of trainees would "recommend to a friend or colleague"! And in many cases this is sufficient to earn our ratings, pat ourselves on the back and start to plan for the next year.

But as we put on our capes and make grandiose plans to keep solving the world's problems this is a great time for us to look back in the mirror and ask ourselves - are these the metrics that REALLY matter?

The short concrete, clear and to the point answer is...sort of.

I'll cut to the chase - reaction metrics (Level 1 in Kirkpatrick) in isolation are somewhat...irrelevant. If business leaders wanted everybody to enjoy themselves they could send them to the movies or organize an event at a much lower cost than hiring folks like us. To quote the band Prodigy  "It's a cold world out there, dress accordingly."

We'll spend more time on more intensive analytics later but for now this begs the question - should we even be collecting Level 1 metrics? Do they matter? In context - yes - because they help tell the story you need as a professional.

Let's play a scenario out - you've been asked to lead a learning effort to drive a particular metric and after 6 months it doesn't pan out. What do you do? This is where your post-experience surveys are vital. Understanding your audience's response and feelings about a learning experience is an essential part of that story. Even if things go well - this data helps you assess, pivot and improve as your programs evolve year over year.

With that in mind - I would encourage the following questions to be in every Level 1 survey, every time:

• CSAT and NPS - the vitals that many leaders want to know (and after you do a great job, you get all the feels)
• Was the content relevant to my job? Did your learners find it relevant? Why or why not?
• Was the content well organized? Perhaps the content was relevant but not delivered in a helpful or meaningful way. This question can help
• If you could change one thing, what would it be and why? Dig deep into reviewing these. These can offer free consultative services - especially when placed into an analyzer to find trends and threads
• Strongly consider a brief quiz - this can help isolate whether it is a knowledge retention issue, a behavior issue or a combination of the two

Thanks for reading! Do you have any thoughts or other questions that can help unlock secrets? Post them below - love to hear it. Thanks! -Kevin