Collaborative Performance Measurement
Working together to develop three types of performance measures
Dear collaborative discussion friends,
This week we are highlighting a new activity added to the updated toolkit. This activity helps participants work together to generate input, output and outcome performance measurements. Participants also explore the differences between each type of measurement and what they offer.
This activity is contributed by Jack Byrd Jr., Professor of Industrial Engineering at West Virginia University and President of the Interactivity Foundation and is one of the many activities in the Critical Collaboration Module.
If you missed the previous newsletter, Collaborative Goal Setting, you can access it and our other weekly newsletters by subscribing below.
This week's activity:
Activity 3.10 – Collaborative Performance Measurement
Practicing how to collaboratively develop methods to measure performance and reflecting on the challenges involved
This activity introduces participants to three different levels of measuring performance. Participants reflect on the different characteristics and relative importance of each type of measurement. They then practice working collaboratively to imagine the kind of performance they want to achieve. They also develop ways to measure if this goal is reached using the different performance measures. This activity also encourages them to further explore the different challenges they may encounter and the various factors that might affect these measurements.
Prepare for the Activity
Organize participants into small groups (4-6 ppl). Share this Collaborative Performance Measurement Worksheet as handouts or electronically with participants. Start by introducing the learning goals of this activity:
Distinguish between different types of performance measures.
Develop performance measures using a collaborative process.
Introduce Three Levels of Performance Measurement
Begin by inviting participants to remember a math course they took in high school or college. Ask them, if they were the teacher for this course, what method would they use to evaluate students’ performance. Which of the following three measurements would be the best way to measure performance:
The number of assignments students were asked to do.
The grade students received on a standardized test.
The students’ ability to use core concepts later in life.
Once they respond, introduce the three types of measurement:
Input - These measures represent the effort put into achieving a result.
Output - These measures represent the actual result that was achieved.
Outcome - These measures represent what happened after the output was achieved.
Ask participants to then rank the three types in order of importance. Invite them to discuss which measure is the most useful and why? For example:
Input measures tend to be the easiest to identify but these are not indicative of performance itself but just effort expended. They are however commonly used.
Output measures are often reflective of surrogates of performance. They are often invented to give some sense of credibility. They are often commonly used.
Outcome measures can be challenging because it takes time to assess whether the initial results have a lasting impact. These measures are the most desirable, but hard to develop.
Break into Small Groups and Generate a List of Outcomes
Next, invite participants to break into small groups (4-6 ppl) and review the following scenario, shown on the worksheet, with their group.
Scenario: Artificial Intelligence Ethics Training
An organization is trying to decide if they want to invest in an Artificial Intelligence Ethics Training for their employees. The training is expensive but could be valuable for their workforce. In order to decide whether to make this investment, the organization needs to decide how the effectiveness of the training will be measured.
Ask participants to write a statement describing what they think trainees should be able to do after the training. What is the desired outcome of this training?
Have participants further expand on this statement by listing the specific outcomes related to this overarching goal on the worksheet. For example, what knowledge should they have gained from the training? Remind them to refer back to the definitions of the three levels of measurements to make sure they are listing outcomes and not outputs.
Develop Measures for Outcomes
Invite each group to discuss the following prompts and record their responses on the worksheet:
How do you plan to measure the desired outcomes?
What measurement challenges do you anticipate?
What factors are likely to affect your measurements?
How might your measurements be misleading?
Facilitator Tip: The goal of these steps is to encourage participants to think through the performance they really want to achieve along with the difficulties in measuring this performance.
Develop Measures for Inputs and Outputs
Have each group make a list of potential inputs and outputs of the proposed training program. Then, ask them to discuss the following questions and record their answers on the worksheet.
How would you measure inputs and outputs of the training program?
What are the strengths and weaknesses of each measure?
How do these measures relate to our desired outcome?
Arrive at Short-Term and Long-Term Measures
Invite participants to reflect on the input and output measures they developed. Ask them to think of ways in which these measures can be used in the short-term. Encourage them to imagine how connecting different measures together can help evaluate the effectiveness of the training. For example, looking at the number of participants who participated in the training compared to those who passed a qualifying test after completing the training.
Ask participants to also generate ways of collecting the data needed to evaluate long-term outcomes. For example, to assess if the training led to a difference in skill level or knowledge, they would need to measure participants' skills or knowledge both before and after they completed the training.
Debrief as a Full Group
Come back together as a full group and discuss the following questions:
Why is it important to develop performance measures collaboratively?
When we evaluate outcomes, what matters most? Numbers? Stories? What seems more credible? Why?
If you were to use your understanding of input, output, and outcome measures, how would you answer the question: How are you better because of this workshop, class or program?
In addition to these debriefing questions, the full description of Activity 3.10 Collaborative Performance Measurement includes reflection questions, a practice journal prompt, and additional resources to help participants dive deeper.
Dive Deeper by Pairing Activities Together
Begin by using the method for setting goals shared in Activity 1.9 Collaborative Goal Setting to have participants craft realistic and achievable goals for a project or issue they are working on as a group. They can then use the three levels of performance measurement introduced in Activity 3.10 to develop ways to evaluate if efforts or programs implemented to achieve these goals are effective. This data can then be used by this group or later groups to modify the goals and programs to maximize effectiveness.
If you try out this activity, please share with us what you think:
We hope this toolkit activity helps participants gain a deeper understanding of how to articulate desired outcomes and use input, output and outcomes measures to evaluate the effectiveness of any effort, project or program they are involved in.
Upcoming Events
Saint Ignatius College Prep in Chicago is organizing Guanacaste 2023: The Transformational Listening Conference, from October 5 to October 7, 2023. We are excited to share that Dr. John G. Igwebuike, the founder of Guanacaste: The Lead Listening Institute, and one of our Collaborative Discussion Coaches, will be incorporating part of the Collaborative Discussion Toolkit in the conference this year. All are welcome! Learn more and register here.
The National Conference on Dialogue & Deliberation 2023 will take place from October 13-15 in Atlanta, Georgia. The Interactivity Foundation, including the Collaborative Discussion Project, will be holding sessions during this event and we look forward to seeing you there. The cost to register is $580, with a discounted rate of $300 for students. For a limited time, the NCDD Member Rate is available to everyone! Get $100 off regular registration by clicking “Have a Promo Code?” (located under today’s subtotal in your cart) and entering “100-off”. Students can get $60 off the student registration cost by entering the promo code “Student-deal”. Register now and save! All are welcome! Learn more and register here.
The Interactivity Foundation is organizing a 3-part interactive, small group conversation series, exploring Who are We The People — and who is being pushed out? The events will be held on October 20, November 3, and November 17. Invite your students! Register here.
Looking forward to collaborating,
Ritu Thomas & the Collaborative Discussion Team