Stagnant Outlook for Learning Measurement (Cushing Anderson) The Forum applies the 70:20:10 Lens


Data and analytics are impacting numerous areas of businesses and decision-making and rightfully so. Using data to drive decisions is critical in accounting, sales, supply chain management, etc. Even on a personal level we are collecting more and more information about how we function, how well we interact in our environment, and the decisions we make. Just look at the booming fitness apps and wearable technology market for examples.

The processes of either business or individuals can be monitored, analysed and influenced through the collection of rich data. However when that method and technology turns to learning things get pretty muddy.

According to a recent Chief Learning Officer article, 'Stagnant Outlook for Learning Measurement' by Cushing Anderson, it was noted that "the state of measurement in learning and development is falling behind other areas of the business." Over 330 CLO's contributed, reporting that most measurement is extremely basic with the most common measurements being in areas like student tracking, course completion data and hours of training completed. This alone accounted for about 75% of measurement activity. 60% reported that learner satisfaction data is captured, which as we know there is little correlation between appreciation and effectiveness.

Finally, only about 50% reported that training output is aligned with specific learning initiatives. Even with statistics from these areas being compiled at rates over 50% what are they really telling leadership about the effectiveness of formal learning initiatives and learning in general, very little. On the job performance and business impact measuring efforts, the most critical measure, are often considered to be time consuming and take increased resources; both of which add financial and opportunity costs for an organisation.

Before looking closer at the issues faced by CLOs and organisations, it’s important to clarify a few points about “learning.” First, most organisations are not truly tracking learning as the terms learning and training are incorrectly used interchangeably. Training is more the input, where learning is the processing of that input (unique and personal) and performance is the output and often the best measure of a change in behavior (learning).

Second, and related, collecting data on the inputs of formal learning activities (time and attendance) is not measuring any level of impact to the organisational bottom line where it matters most. Third, even when the "needle moves", after a formal learning event it is difficult or almost impossible to isolate the variable of that event's impact. The context, systems, and level of support all contribute mightily to the overall effectiveness. Finally, many studies have shown that formal learning really only accounts for about 10% of all the learning that happens in an organisation, the vast majority is outside of the watchful eyes of the CLOs charged with measuring learning.

So what of the other 90% of learning? Again, studies like those informing the 70:20:10 Framework reveal that people are learning through informal methods such as in the conversations, observations and within the work they do by experimenting and improving. The problem for organisations curious of the impacts of this learning is that they cannot seem to capture and quantify it. This however is the wrong problem to be developing a solution for.  The real problem is not in measuring if learning is happening in these areas but in finding ways to ensure that the workforce can do this without friction. The biggest problems facing informal learning include access to information, experts, time for reflection, collaboration, and developing skills in networking and filtering. CLOs in the article complained that some of their problem resided in not having the right technology.

Their vision of course was in using technology such as SCORM and xAPI to again try to track learning and justify expenditures. Although technologies like these can identify the courses and assets used as well as the pathways people have taken, they don't really measure learning. The technologies needed to address the real problem are those that help extend and expand the conversations that lead to increased collaboration and cooperation. Technology that enables employees to quickly share eureka moments, solve problems, and answer questions others have posed. 

A new focus on social and informal learning doesn't discount the need to measure the effectiveness of the 10%. Formal learning still has its place especially when there is a need to learn something new or in the case of a more novice workers. But again, the most impactful measure is not if the training has taken place and who attended but if the training has been applied and how well. The 10% rarely works in isolation. Training is enhanced when the effort includes the tools and approaches used to support the other 90% such as peer reflection, coaching and performance support. It is here that what is learned in the formal events is sustained and reinforced over time. 

So how do our organisational learning leaders reduce "falling behind other areas of the business"?  For starters Chief Learning Officers need to focus less on the "Chief" and more on the "Learning". Given the focus of the CLO research, there is a whole lot of learning happening that is not being given the attention it deserves. CLOs need to shift their attention from trying to emulate other business function and instead work to redefine organisational learning as something that transcends business functions. 

READ: 'Stagnant Outlook for Learning Measurement'