Activity vs. Performance Measures in Training Evaluation

Image by Gerd Altmann from Pixabay

Organizations are looking to invest in employee training and ongoing development. To make sure these learning and development investments are effective, it is important to evaluate and measure their impact. But which metrics should you be keeping track of? Which measures offer a clearer understanding of training impacts? 

Evaluating the impact of training initiatives can be done using activity measures and performance measures.

Activity measures focus on the "process of learning" and include different aspects of the learning journey. These measures help assess learner participation in various activities such as training completion time, attendance records, and allocated time for specific activities during training, etc. For example, when measuring the impact of software training, activity measures may include tracking how many users participated in the program, what percentage completed it successfully, and how much time was dedicated to working on the software during the training.

Performance measures focus on the "outcome of learning" and help assess how well learners can apply their knowledge to real-world tasks and situations. Job performance, customer satisfaction, and test scores are some examples of the types of measures that can be included in this category. For example, when measuring the impact of software training, performance measures can include the number of errors made by software users, the scores on a test of software proficiency, and customer satisfaction with the work produced by users of the software.

Activity measures are good for assessing initial engagement and training completion while performance measures provide a better picture of how well employees are applying their newly acquired skills.

However, there are challenges in using performance measures. For example, it is difficult to identify performance measures that are directly related to the training and that accurately reflect the desired outcomes. Collecting and analyzing performance data is also costly and time-consuming. Most importantly, it is challenging to isolate the impact of the training and identify if any changes to the performance were directly due to the learning or training initiative, as opposed to other factors such as changes in the work environment. While it is good to plan the evaluation of learning and training projects using performance measures, using performance measures may not be applicable or cost-effective for every kind of training.

The combination of activity measures and performance measures is usually the best option for gathering significant and valuable data. However, what makes up the appropriate combination of these measures may differ according to the particular training or learning approach and intended results.

Navigating Generative AI Citations in Your Writing

There has been much conversation about how generative AI should be cited in written work. I recently wrote an article for BCcampus and leveraged ChatGPT for one of the sections specifically to assist in building on my key ideas and in generating some of the scenarios. 

There were no existing author-writing guidelines or policies at BCcampus that permitted or prohibited the use of AI tools (at the time of writing the article). But as the author, it was important for me to be ethical and transparent about my writing work and take full accountability and responsibility for my ideas. And I was supported and encouraged by the team at BCcampus to do so.

Based on my limited research on how AI is cited in scientific writing, I wanted to acknowledge the use of ChatGPT as a ‘method’ of research used for a specific section. I also referred to Canada’s Guide on the use of Generative AI and the “FASTER” principles.

This is the reference that I crafted and we decided to include it in the section for which ChatGPT was leveraged:

My intent in including this paragraph was to promote the responsible, transparent, and ethical use of AI in my writing. Some may find this useful; some may find it unnecessary or something that can be omitted or perhaps even ignored.

What do you think of this reference? Do you use ChatGPT for your work? How are you including it in the reference? What are some guidelines and principles you are following in using it fairly and transparently?

PS: If you are interested in reading the article I wrote, you can find it here: 
https://bccampus.ca/2023/11/15/digital-pedagogy-toolbox-integrating-digital-literacy-practices/

The article highlights how educators and learners can develop digital competencies and skills by using the B.C. Post-Secondary Digital Literacy Framework as a roadmap. It explore the key facets of digital literacy in the context of the Digital Literacy Framework with a scenario that highlights practical strategies for how digital literacy can be implemented into every aspect of the learning journey by educators and learners. ChatGPT was used to assist in building on my key ideas and generating some of the digital literacy application scenarios.

The BCcampus Open Education Team has since released its guidelines on leveraging generative AI tools for OER content creation. 

#ChatGPT #AIEthics #AIGuidelines #AIForWork #Writing #ArticleWriting #CitingAI #AIReference

Celebrating Collaborative Excellence: A Journey with IEEE 1484.20.2 Recommended Practice for Defining Competencies

gratitude: the feeling or quality of being grateful; thankful

That's the feeling that came over me when I received this beautiful acknowledgment from the IEEE team a few days ago!

Certificate of Appreciation Awarded by IEEE to Taruna Goel 

The IEEE 1484.20.2 Recommended Practice for Defining Competencies standard was published last year and was a culmination of 3 years of work led by Eric Shepherd who was supported by a strong executive team Steven Forth Linda Steedman Dennis Green Jennifer Rogers and a committed working group that included myself and many others.
 
As a competency nerd, I enjoyed being in the company of passionate, motivated, knowledgeable people who were all volunteering their time and whose single-minded goal was to discuss and document best practices for defining competencies: What makes a competency? What must be included in a definition of competency? What makes a good competency statement, etc.
 
In reflecting on this achievement, it's important to recognize that my contribution was just one piece of the puzzle. The collaborative spirit of the team and each person volunteering their time and expertise made this project truly remarkable. I feel humbled that I got to play a small role in this big project!
 
#gratitude #competency #ieee #HR #learning #CompetencyDefinitions
#IEEEStandards #CompetencyStandards #CompetencyFrameworks #Volunteering #VoluntaryWork