Sunday, March 20, 2011

Assessing Informal Learning

For March the LCBQ is: How do you assess whether your informal learning, social learning, continuous learning, performance support initiatives have the desired impact or achieve the desired results?

My response to the question – I think we should assess informal learning but I don't think I want to asses informal learning using the same set of parameters often used for formal learning.

Need to assess
To me, the objective of all learning (including formal and informal) is to achieve the 'desired' change in behaviour and performance. Therefore, I want to continue to assess performance to see if the ‘desired results’ have been achieved. And since informal learning contributes a great deal to our performance, therefore, it is important to think about how to assess informal learning.

Difference between formal and informal learning
We are constantly learning - either by ourselves or because others want us to learn. To me, there are two important distinctions between formal and informal learning:
• WHO drives the learning
• HOW it happens

Formal learning: All formal learning is driven by an external entity - an individual, a school, a college, a company etc. The process of formal learning is governed by stakeholders who define the system and the desired outcome. Therefore, the measure of success of such training is defined by that individual, school, college or company. The parameters of measurement for formal learning range from smiley sheets to return on investments that attempt to show the dollar value of training on the other end of the continuum. Even with such a wide range within the Kirkpatrick model, I can’t say that as learning professionals, we have been successful in measuring the impact of formal learning.

Informal learning: Research points that almost 70% of the learning is on-the-job and is acquired in an informal setting. So, informal learning happens naturally and as a consequence of a person motivated and wanting to learn. In that sense, informal learning is driven by the individual herself. The individual is intrinsically motivated and wants to learn and creates or utilizes the right environment to trigger this learning. This kind of learning almost always happens when the individual wants to learn to solve a problem or become better at what she does. Examples include connecting with your colleague over a coffee conversation to identify the right choice in a particular situation, tweeting about a given challenge to seek perspectives or solutions, blogging about an issue to seek inputs and approaches etc. With a myriad of social learning options available, the range of examples is also wide! Since, informal learning is triggered by an individual, therefore, in my opinion the success of such learning can only be measured by that individual.

Assessing informal learning – the individual focus
The question is then, as an individual who is undertaking informal learning, how can I measure the success of the time and effort I spent in doing this learning. Since all learning is a continuous process and not an isolated event, it is therefore difficult to identify what was that single defining moment in the process that caused me to learn. However, it is easier to identify whether my learning objective was met each time I triggered my own learning and learnt in a social environment in an informal way. There are a couple of things that I personally look at, for my own informal learning processes:

1) Degree of achievement of learning objective – If the informal learning helped me solve the problem at hand and/or do my job better, and basically helped me meet my learning objective, it was worth it! For example, if I wrote a blog post to seek inputs about a particular challenge, did the post and the comments from other people help me arrive at the solution? If it did, I am more likely to write more posts and therefore increase the use of social learning tools and technologies to undertake informal learning. It is a matter of learning objective being met – a yes or a no. In that sense, this is perhaps an objective way of looking at the scenario.

2) Degree of satisfaction – Now this one is totally subjective and focuses on the softer aspects of the learning. The degree of satisfaction depends on the quality of learning that happened or the quality of the solution that I came up with basis my informal learning intervention. There is some place for evaluating how satisfied I have been with the learning process and the learning that eventually took place through informal means. The degree of satisfaction depends on many other factors such as how quick, effective, complete, and useful the solution has been.

3) Degree of impact on others – If what I learnt in an informal way, I can utilize and apply to solve my own problems and problems posed by others, I find the learning to be effective and worth the effort. In that sense, this is my parameter to evaluate ‘learning transfer’. Perhaps, I can make it more objective by turning this into numbers – how many people I impacted, how frequently did I utilize my informal learning, how satisfied others were with my recommended solution etc.

There are perhaps other ways of assessing informal learning and I do it unconsciously all the time. Assessing informal learning is challenging but it seems do-able especially if the learning is evaluated by the individual who learnt in the first place!

Assessing informal learning – the organizational focus
Moving to an organizational platform – as employees, we spend time and effort in informal learning. Therefore, it is equally important for an organization to analyze how effective these efforts really are. Instead of focussing on hard numbers such as those typically used for formal learning, organizations should focus their efforts on allowing employees to self-reflect on their informal learning interventions and help employees evaluate and assess the process. Perhaps, the individual parameters suggested above can be incorporated into the performance appraisal or annual learning review process forcing both the employees and the organization to reflect and analyze what works and what doesn’t work for informal learning. Organizations can work towards creating environments that promote informal learning and remove organizational barriers and hierarchies by building informal network of employees across divisions and roles. Organizations can further analyse such networks and evaluate the type and frequency of usage. By enabling social collaboration and promoting the use of social media such as twitter and employee blogs, organizations can increase the reach and impact of informal learning.

I agree that we should assess our efforts towards informal learning. If we don’t reflect and analyse it in ways that seem meaningful to us, we are leaving too much unanswered. Besides if the goal of all learning is to move towards a desirable change in behaviour and performance, it is important to assess and evaluate all forms of learning with that yardstick. However, I don’t think organizations should try to translate informal learning into only measurable numbers and use existing models of training evaluation to evaluate informal learning. I think we should rather focus on increasing both individual and organizational reflection about the learning efforts that find their way in informal learning settings and use that reflection to analyze the impact of informal learning towards increasing performance.


  1. Taruna,

    Provocative post, and I sincerely appreciate your perspective. We agree on some important points, but I'd like to offer an alternative point of view on some elements.

    We both agree that the impact of informal learning ought to be assessed in some manner. Organizations of every size, shape and description ought to be able to identify the return on investment for training - both formal and informal. Likewise, we both agree that informal learning plays a much, much greater role in employee learning than "traditional learning metrics" might suggest. And finally, I think we both agree that informal learning is more effective in the long run for the employee at the individual level.

    I also like your strategic objective for all learning - moving behaviors from current to desired behaviors. And I think this core objective is something that can be measured - regardless of the type of learning that resulted in the improvement.

    The problems with learing or traning ROI are well documented, and it looks to me like some of these problems are actually exacerbated when we try and evaluate just how effective informal learning efforts might be. Further complicating the whole assessment puzzle is the fact that organizational cultures, learning objectives, available data, etc. will vary widely from organization to organization.

    So - for what it's worth, here are a few ideas (very broad, I admit) that might help pin down the effectiveness of informal learning efforts:

    1. Surveys. I think some valid survey questions around the source of information for employees, the speed with whch various sources provide answers, etc. can help isolate the effects of informal vs. formal learning efforts.
    2. Behavior based metrics. If an organization can isolate the current behaviors and the effect on important organizatioal performance indicators, the organization can use the same measures to assess changes in behaviors and chages in outcomes.
    3. Financial resources allocated. We know formal training/learning efforts can be expensive. Informal learning isn't free, but it can be less of a financial commitment. It's important to isolate the costs associated with the infrastructure needed to support informal learning.

    Hope this helps, and it's a conversation we'll be having on our blog in the near future! Would love for you to participate.


  2. Chris,

    Thank you for your comment on my post. I am glad we agree on various aspects of informal learning!

    I also appreciate your inputs regarding establishing the effectiveness of informal learning efforts from an organizational perspective.

    Through this post my intent was to discuss how organizations should figure out alternate means of assessing informal learning rather than focussing on using existing models for evaluating formal learning. In that sense, reflection is possible using various methods including constructive surveys - like you suggested. Similarly, network analysis and promoting learning through shared networks is a alternate view of looking at assessing the informal learning efforts and developing specific behaviours.

    I enjoyed reading your views about the topic and I'd be happy to participate on your blog!

    Taruna Goel

  3. Dear Taruna - it's a month gone bye and your post has travelled around the world. I picked it up in Paris through a woman in the Ozarks of USA while I live in Perth Australia. I'm an instructional designer for remote online students who by-and-large are self-directed in their studies. Ultimately they have to provide sufficient evidence that they are competent in the Unit of Study that make up their qualification.

    Over the last few years however, we have been able to multiply the interactivity among students and from Lecturers through Moodle LMS; and what has emerged is the opportunity for informal learning. By this I mean various dialogues about research and focus; about thinking; about thinking about one's thinking; and thinking about one's learning. These themes have crept into conversation in forums, chats, blogs and messages that fly around while the study is progressing.

    There are very interesting case studies in Canada about how students adjust to declarative and procedural knowledge: knowing "that" as opposed to knowing "how". The networking of a class of students; and the mentoring by experienced teachers; all starts to pull it together and underpins the formal assessment with integrity. You start to see a student focussing on the skills they need for the job they want; and you start to hear the language of the job that articulates those previously invisible routines now becoming visible to a new employee.

    I think you have courage in being a champion of informal learning because its contribution is creative (as we advance beyond rote learning so slowly). Never lose heart! The old industrial society never imagined what a complex adaptive system we live in; they never imagined the emergenge of entirely new properties of order and prosperity; and they never imagined that a single person could create a tipping point under their nose.

  4. Hello Paul,

    I am glad my blog travels around the world and has reached you! Thank you for sharing your comment. I am happy to know about how your students are able to use new technologies to generate conversations about thinking and learning.

    I am determined to both lead and support informal learning practices. I believe that virtual communities and informal learning leaders can help create a continuous learning cycle that goes beyond planned, formal set of activities and help people/students recognize the informal learning events that naturally take place in any social environment. Since learning per se is a lifelong process, informal learning and recognizing its potential gives us the opportunity to continue to learn and to teach...

    Looking forward to your participation on my blog and wishing you good luck with leading meaningful learning!