Training is always an investment, no matter how big or small the effort is. eLearning, despite its reputation for being a time-saver and a cheaper alternative to classroom training still requires organizations to invest in the technology, people ( i.e. instructional designers, instructional developers, content writers, SMEs, IT) and other resources. It’s important for learning and development teams to ensure that the success of their new eLearning course is measurable, so stakeholders and project sponsors will be assured of ROI ( return on investment). But how does one measure the success of an eLearning course?
Set Goals at the Beginning
Evaluating an eLearning project is usually done at a later stage, but the basis for such evaluation needs to be set at the very start of a project: the goal setting stage. This stage is very crucial in evaluating success, because the set goals will dictate exactly what will be measured at the end of implementation. The goals must be SMART: S- Specific M- Measurable A- Attainable R- Relevant T- Tangible Goals are not supposed to change in the middle of the implementation stage, so the eLearning project team must put a lot of thought and effort in identifying what is important.
It’s not unusual for the learning and development team to have a slightly different notion about evaluation than their stakeholders. The difference lies in the team’s experience in L&D, their interest in the eLearning project, and their priorities.
It’s important for the learning and development team to meet their stakeholders halfway, in terms of evaluating the eLearning program. If compliance the most important metric due to the nature of the business, or is the learner’s improvement the top priority? Having this conversation with your stakeholders will ensure that you will measure key areas that matter to both you and the groups who invested in your eLearning project.
Use the Four Levels of Evaluation
Donald Kirkpatrick, former Professor Emeritus at the University of Wisconsin, wrote about the four levels of evaluation in his book “Evaluating Training Programs”. The four levels are: Reaction, Learning, Behavior and Results. Let’s look at how each level applies to evaluating your eLearning program.
Reaction in the four levels of evaluation refers to the learner’s satisfaction after completing your eLearning course. Plenty of courses already include a link to the questionnaire at the end of the course. A clever way to ensure the participation of all learners, is to require the completion of the survey before they are given a certificate of completion.
Here are some examples of questions that you may ask in your training evaluation survey:
● Did you feel that the course was worth your time? ● Did you think that the course was a success? ● What were the biggest advantages and disadvantages of the training? ● Did you like the course and the way it was presented in the LMS? ● Did the course accommodate your personal learning styles? ● Were the eLearning activities engaging? ● What are the three key concepts that you learned from this course? ● How do you plan to apply what you learned in your job? ● What support might you need to apply what you learned from the course?
Measuring learning focuses on whether the learning objectives of the course were met. This shows how well the course made a difference in the skills, attitudes and knowledge of the learners. In other more in-depth learning measurement, the confidence and commitment of learners are highlighted as well.
One of the simplest, but most effective ways to carry out a learning evaluation is to conduct pre and post training assessments. For instance, if you have an eLearning course on GDPR, you can ask basic GDPR questions at the start of your course, and another identical one at the end of your course. Comparing the scores will show how much knowledge the learners have gained, and will instantly show the impact the training made.
In the case of soft skills eLearning courses, you may make add scenario-based questions in every module to assess if the training made a difference in the learner’s decision-making.
Behavior is more challenging to observe for eLearning because it will require actual observation of the learner’s behavior at work. There are plenty of creative ways that a behavior evaluation can be carried out. For example, if you have an eLearning course on telephone skills, you can create a checklist of behaviors ( aligned to your content, of course) and evaluate recorded calls that learners took before and after the training. Comparing the results of both evaluation will show how well the training influenced the behavior of the target learners.
In the case of an eLearning course on customer service, you may send “secret shoppers” to the store with scripts that will place the learners in the same situations discussed in the content. Because the secret shopper presents himself as a real customer, you will be able to record and observe the learner’s real and natural response to these important customer touch points.
You can also ask additional questions like: ● Did the learners put any of their learning to use? ● Are learners able to teach their new knowledge, skills or attitudes to other people? ● Are trainees aware that there has been a shift in their own behavior?
At level four, you analyze the final results of your eLearning course. This includes outcomes that your team, your stakeholders and the organization have agreed on that are important for business and valuable for your team members. This is the time to show that your eLearning course provided good ROI.
For example, your team has created an eLearning course on safe IT practices in order to lower the number of data breach incidents. Data breach can cause a company thousands of dollars in regulatory fines, attorney fees and declining sales due to a tarnished reputation. You may compare pre and post- training numbers of data breach incidents, to see if the training helped the company avoid these additional costs.
Measuring eLearning takes plenty of preparation and collaboration with your stakeholders, but it is the most rewarding part of an eLearning development project because you will finally be able to see the fruits of your labor.