- OBM Footprints
- OBM Community
- For Students
Including OBM in learning solutions to sustain behavior change
Including OBM in learning solutions to sustain behavior change
Posted on: March 2nd, 2016
Barbara R. Bucklin
Does Training Alone Sustain Behavior Change?
The short answer is, no. As a result of a curriculum evaluation, which I’ll describe later, my colleagues at Ardent Learning and I found that performance deteriorated over time following training. This didn’t surprise us at Ardent Learning, and is expected by anyone familiar with OBM. However, these results may surprise individuals in corporate training settings, especially those who aren’t willing to spend the time, effort, or dollars to analyze their performance-improvement needs or evaluate their programs.
Stolovich (2000) made this case when he stated,
without adequate management attention, performance deteriorates rapidly following training, often to below pre-training performance levels due to non-supportive or even inhibitory factors. The result is a rapid reversion to familiar behavior patterns, (p. 10).
Who would know that deterioration is happening without (1) a detailed front-end analysis of all the factors that impact behavior, and (2) evaluation data to measure behavior and results following training? And what can be done to sustain performance after participants have left the classroom or turned off their computers? We’ll explore these questions next along with what my Ardent Learning colleagues and I have learned over the years about sustaining behavior change.
A Lesson Learned
Several years ago, a large automotive manufacturer partnered with us to develop training for its dealership parts and service personnel. This multi-year project included:
- A detailed training needs assessment (note: this was not a performance analysis, which is identifies all possible solutions to a performance problem and focuses more widely than a training needs assessment.)
- Curriculum and training recommendations based on needs assessment results
- Design and development of training curriculums and courses for 11 dealership non-technical service roles; this resulted in 60 courses including Web-based training, webinars, and instructor-led workshops
- Curriculum evaluation
As part of the evaluation, we collected data before, during, and after participants were certified in their respective curriculums. These data were used for within-participant and within-group comparisons to assess whether results improved during the training period and/or following the training period for dealership personnel who became certified. Visual inspection and paired sample t- tests were used to analyze these data (p < 0.05 was the value used to determine statistical significance). As an example, see Warranty Administrator Claim Return Rate graph in Figure 1. For this metric, an improvement was defined as a decrease in number of claims returned to the dealership for inaccuracies.
Figure 1. Warranty Administrator Claim Return Rate Improvement Measured as a Decrease in Number of Claims Returned
As illustrated in Figure 1, and for all within-group comparisons we analyzed across the 11 job roles trained, we saw performance return to baseline over three to six months after participants were certified (i.e., completed all training in the curriculum for their job role). This confirmed the need for ongoing learning and performance interventions and refreshers to truly impact behavior. Although this wasn’t surprising to us, it was a revolutionary moment for most of our client team.
Analyzing All Performance Factors to Inform Clients Up Front
The initial experience with the automotive manufacturer gave me tangible data to share with my clients and gain their buy-in for doing things the right (OBM) way. Now I conduct performance needs analyses up front to identify all the factors that impact behavior – not just training. Then I’m able to use performance analysis results to recommend solutions in addition to, or instead of, training.
Six Boxes® Analysis
Using Carl Binder’s Six Boxes® Framework (2012), I’m able to identify gaps in (1) expectations and feedback, (2) tools and resources, (3) consequences and incentives, (4) skills and knowledge, (5) selection and assignment to the job, and (6) motives and preference (attitude). Even when clients are only interested in Box 4 gaps (skills and knowledge), which are improved with training solutions, the other “boxes” need to be addressed to ensure the environment supports newly trained skills. A needs analysis report from Ardent Learning is organized across these six factors, and includes Key Performance Indicators (KPIs), Six-Box Gaps identified, and Actions to close the gaps. See Figure 2 for an example. If you’re not familiar with the Six Boxes® Model, I recommend you visit www.sixboxes.com.
Figure 2. Six Boxes® Tied to Performance Gaps
Suggestions for Maintenance
Since that early project where we saw performance return to baseline, the methods we’ve used for sustaining behavior (tied to Six Boxes® Analysis findings, of course), have included the following.
Feedback and Management Support
Without accountability for behavior change, typically in the form of feedback or management support, training participants will rarely implement new skills or knowledge on the job. For example, Stephens and Ludwig (2005) showed that goal setting and feedback following training were effective to further improve and sustain nurses’ safety behaviors.
To help managers provide the necessary support and feedback, we have implemented:
- Performance Feedback Dashboard Portals: This serves as a Performance Dashboard for individuals by allowing them to see their personal results feedback. For examples, Service Advisors can log in and see their own data for metrics such as customer satisfaction, and labor and parts sales. We’ve learned from performance needs analyses and interviews with automotive service audiences that, on average, less than 40% of Service Advisors receive performance feedback from their managers. While this type of dashboard doesn’t replace direct management feedback, it can help employees improve on their own.
- Electronic Coach’s Guides: These guides allow the “coach” (i.e., the manager) to observe an employee in role-plays and on-the-job situations, and assess, via the coach’s guide, his or her performance. These Coaching Guide data are then stored or sent to the employee’s history on their organization’s Learning Management System. It’s a way to hold managers accountable for assessing their employees’ skills and providing relevant feedback.
Performance Support Tools and Refresher Training
Short courses or information presented at intervals following initial, formal training, have been show to keep skills at or close to post-training levels. For example, researchers from Great Britain demonstrated the necessity of frequent refresher training to sustain CPR skills (Morgan, Donnelly, Lester, and Assar, 1996).
To ensure skills are continuously practiced and applied, we have implemented:
- Hot topics: Short, printable or online information on topics previously trained and/or new topics relevant to the job role.
- Practice activities: Fun, interactive online activities to keep skills fresh and provide practice opportunists for important skills that employees don’t frequently perform.
- Social, collaborative learning: An online community to allow participants to interact with others in the same job role and share best practices and lessons learned.
Training Methods Used
Although Ardent Learning has always used sound instructional design, we’ve increased the number of fluency activities and scenario-based learning interactions, which are more likely to result in transfer of training to the job, long-term retention than more traditional training.
- Fluency learning: Fluency, the combination of accuracy plus speed in responding, results in many benefits over traditional training methods, including skill retention (Binder, 1996; Bucklin, Dickinson, & Brethower, 2000). Terms, such as "automatic," "second nature" and "effortless" have been used to describe fluent performance. This type of performance is essential in many industries. For example, if sales consultants can’t speak fluently—without hesitation—about their products and services, they’re likely to miss out on sales opportunities and damage customer relationships. As an example, when a new in-vehicle technology was introduced, we created an online fluency activity requiring learners to identify facts about the technology both quickly and accurately. This was developed with a timer and scoreboard, measuring response speed and accuracy.
- Scenario-based learning: To increase opportunities for skill practice, we’ve added more scenario-based learning elements to put participants in a situation or context that exposed them to realistic challenges and asked them to apply skills in the training (classroom and online) environments.
By conducting performance analyses up front and including OBM solutions from program inception, Ardent Learning and our clients have been finding what we expected and hoped for: sustained behavior change that does not revert back to baseline post training.
This shouldn’t be surprising to those in the OBM field; however, it’s a clear example of our need as OBM practitioners to disseminate behavioral science to the masses. Organizations spend billions of dollars each year on training and development programs (“2014 training industry report,” 2014; Bersin, 2014). In fact, organizations are estimated to spend as much as $164 billion annually in the United States alone (Miller, 2013). Businesses are investing these unprecedented amounts in training with the expectation that they will lead to employee and organizational performance improvement (Salas & Cannon-Bowers, 2001). Let’s work together as an OBM community to educate corporate training departments about the need to include OBM in learning solutions to sustain behavior change and achieve expected performance improvement.
- Bersin, J. (2014, February 4). Spending on corporate training soars: Employee capabilities now a priority. Forbes. Retrieved from http://www.forbes.com/sites/joshbersin/2014/02/04/the-recovery-arrives-c...
- Binder, C. (2012). The Performance Thinking Network. Retrieved from http://www.sixboxes.com
- Binder, C. (1996). Behavioral Fluency: Evolution of a New Paradigm. The Behavior Analyst, 9(2), 163-197.
- Bucklin, B.R., Dickinson, A.M., & Brethower, D. M. (2000). A comparison of the effects of fluency training and accuracy training on application and retention. Performance Improvement Quarterly, 13(3), 140-163.
- Miller, L. (2013, November). ASTD’s 2013 state of the industry report: Workplace learning remains a key organizational investment. T+D, 40–45.
- Morgan, C.L., Donnelly, P.D, Lester, C.A., & Assar, D. H. (1996). Effectiveness of the BBC's 999 training road shows on cardiopulmonary resuscitation: Video performance of cohort of unforewarned participants at home six months afterwards. British Medical Journal, 313, 912-916.
- Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471– 499. Stephens, S. D. & Ludwig, T. D. (2005). Improving anesthesia nurse compliance with universal precautions using group goals and public feedback. Journal of Organizational Behavior Management, 25, 37-71.
- Stolovitch, H. D. (2000). Human performance technology: Research and theory to practice. Performance Improvement, 39: 7–16.
|Contributing Author:||Featured Sponsor:|
Barbara R. Bucklin
Like what you read? There are many more articles just like this in the latest issue of the OBM Networks newsletter "Network News"! Members of the OBM Network get full access to "Network News," in addition to our archives. Want to learn more about incentive systems in the workplace, OBM in healthcare, or what OBM research has been done in clinical settings? Become a member today to have a network of possibilities at your fingertips!