As the evaluation space becomes increasingly more technical, digital and complex – think big data, impact dashboards and collective impact – the value of personal stories of impact can sometimes be overlooked. After all, could a first-person narrative really offer more insight on how an initiative is performing than a real-time collation of hundreds of digital survey responses?
What is Most Significant Change (MSC)?
With this in mind, let’s look at a couple of examples where we’ve used MSC to help organisations uncover valuable insights that may have otherwise been missed.
In some cases, yes. Especially if that story was collected and selected using the Most Significant Change technique – an evaluation method specifically designed to uncover deeper and potentially unexpected program impacts. But before we delve further into the powerful potential of storytelling for evaluation insights, let’s first explore the Most Significant Change technique.
What is Most Significant Change (MSC)?
MSC is a participatory, qualitative evaluation tool involving the collection and selection of first-person stories. Each testimony must contain a beginning, middle and ending. This format is important and incredibly effective – every human culture in the world uses storytelling as a vehicle for learning and improvement because our brains are wired to make sense of information through stories. Put simply: you are more likely to remember and act on information shared in a story format.
A typical MSC story might start with a bit of information about the storyteller – usually a beneficiary, program staff member or other key stakeholder – about who are they and what things were like for them before they became involved with the initiative. It then describes how the person came to be involved in the initiative and conclude with a description of what changed for them and why the changes were significant to them.
Story selection
With MSC, the process of story selection is just as important as story collection, and is where data validation and analysis occurs. Key stakeholders are brought together to read the range of stories collected and asked to decide which contains the most significant change that occurred as a result of the program/initiative. The ensuing conversations about why each stakeholder selected their particular story contain rich information about the program and delivers real insight into what stakeholders value.
How is MSC different to other approaches?
MSC is often compared to case studies and appreciative inquiry. Although it can be similar in appearance to a case study, the methodology is quite different with unique steps designed to help organisations learn and improve that go beyond what stand-alone case studies can offer. For instance, case studies are often selected by program staff alone, whereas MSC relies on panels of people and a participatory process to decide what stories are important and should be elevated. The heart of MSC is that it elicits value from the individual in the story and from the group selecting the story.
MSC is a great companion or supplementary approach to many traditional evaluation approaches, and is particularly useful where:
✅ hearing the voices and lived experience of participants and beneficiaries is important
✅ the program is decentralised and there are diverse outcomes and indicators
✅ the changes are intangible
✅ a program is still unfolding or emerging
✅ the environment is changing rapidly
✅ there is a desire to involve stakeholders in making judgements about data in a really engaging manner
✅ the program itself is highly participatory.
When isn’t MSC Suitable?
MSC is potentially unsuitable when:
❌ there is a need to understand the scope of the change
❌ you want to prove the changes happened as a direct result of your program
❌ the program is highly contentious or there is a lot of emotion around it in the community
❌ you just want some good news stories or marketing material (hire a journalist – it will be cheaper!)
With this in mind, let’s look at a couple of examples where we’ve used MSC to help organisations uncover valuable insights that may have otherwise been missed.
Equal Playing Field
Equal Playing Field is a Papua New Guinea-based NGO who run respectful relationship programs for young people. The programs are delivered by volunteers (young university students) to high school students. Volunteers and high school students were asked to provide their stories through an MSC process as a powerful way to capture experiences of the program.
The stories revealed that prior to the program, participants had no idea their behaviour would be considered “teasing”. This had significant impacts for both data collection and overall understanding of how the program was working.
For example, a survey question used in program evaluation asked participants to respond to the statement “I think it is funny to tease people”, with the program managers expecting to see a reduction in those endorsing this statement by the end of the program. While the participants were now aware of teasing and longer thought it was funny, the existing survey wasn’t sensitive enough to capture this change.
The use of MSC informed a redesign of the survey to provide more accurate data about the program’s real impact.
Learn the MSC Technique
Learn more about Most Significant Change by joining our upcoming course. Designed and delivered by Dr Jess Dart, co-creator of the Most Significant Change User Guide. This course will provide you with the practical tools and resources you need to effectively use the MSC technique in your programs.