This post was originally published on InsightShare’s blog. For more on their participatory video work see insightshare.org, or read my reactions to their participatory video training back in 2013 here.
Laptops banned. No notebooks allowed. For those of us who like to write everything down, the instructions for the latest InsightShare course on Participatory Video for Most Significant Change were a bit daunting. How would I remember it all?
Fortunately, visualisation (lots of drawing, arranging of keywords and mind maps) and experiential learning (going through the process ourselves as participants) helps it stick. Here’s what I learned:
1. “Most Significant Change” sounds a bit fluffy, but it’s actually a recognised evaluation technique.
Participatory monitoring and evaluation means that those affected by (and those affecting) a programme are involved in the process of assessing what worked. Together the group negotiates and agrees how to measure progress.
The Most Significant Change process, developed in the mid-1990s, is one form of participatory M&E. Groups collect people’s stories of significant change in their lives, analyse them, and then systematically select the most significant ones. Reflecting on the stories at each stage allows those involved to learn about what causes change.
MSC is now accepted as a valid monitoring and evaluation technique, and has been used by government agencies like the UK’s Department for International Development and international organisations like Oxfam.
2. Adding video makes a lot of sense.
MSC was conceived mainly as an oral exercise, with stories captured in writing. This can have its issues, especially if you’re gathering hundreds of testimonies. One user of MSC reports that it was difficult to get the evaluation teams to write up stories, because they saw it as adding to an already heavy workload. Written stories also risk being left unread: some people may be illiterate; those who can read might be put off by many pages of text. Writing also loses the expression and body language of the storyteller.
Using video, meanwhile, can bring those stories to life, potentially increasing the impact on the viewer. As InsightShare facilitator Isabelle explains: “It takes data off the paper, and it makes it human”. Since anyone can learn basic video skills, storytellers can speak to peers, in a familiar setting; they can watch videos back immediately and as a group. People may also be more inclined to attend a screening than to take part in a focus group. In short, it’s fun and accessible, yet still analytically rigorous and data-rich.
3. It’s best suited for organisations prepared to learn — and maybe even change.
With so much pressure these days to demonstrate the value of a project and show what’s been achieved, it’s easy to think evaluation is about reassuring funders. But as one of my fellow trainees put it, this is evaluation “that aims to improve — not to prove.”
The stories are based on responses to an open question — usually: “What has been the most significant change in your life [in x time period]?”. That prompts unforeseen answers. Perhaps the programme had unintended effects; participants might not mention the aspects you thought were crucial; maybe something else entirely influenced the change they talk about.
So organisations considering PV MSC need to be doing it for the right reasons. (It works especially well as part of a long-term intervention, when the findings of a first phase can feed into the next one.) And they need to know that the process “can bring a cost”, as InsightShare facilitator Neville says, and “the cost is change.”
4. It’s not going to replace quantitative methods any time soon.
Quantitative methods help you see what has changed and by how much. MSC isn’t a replacement for that: it doesn’t use predefined indicators, or anything that needs to be counted and measured.
But MSC can work well alongside quantitative research, by exploring why things have changed, as Soledad Muniz, InsightShare’s head of innovation and development and our trainer for this course, told us. It adds a deeper understanding of what a programme or activity has actually meant to people, she said, and that “lets you understand people’s perspectives on how change happened in their lives, as well as how other enablers contributed to that.”
Many InsightShare clients, such as Nike Foundation, have used PV MSC as part of a much bigger evaluation exercise.
5. It can also build skills and experience among those involved.
PV MSC gets you more than just useful feedback: it can also build capacity.
Training local evaluation teams — beneficiaries, local staff, other stakeholders — and taking them through the whole process means they learn and practice data analysis, presentation and public speaking, consensus-building and negotiation, and video recording/editing skills. Depending on existing levels of education/experience, training and support can be time-consuming and costly; though the process can also be done on a much more limited budget, without ticking all the participation boxes.
6. Sharing personal stories can be powerful stuff.
How often have you simply talked to people who listen, without interruption, for as long as you needed to, knowing you were in a safe space? Oddly, something so simple seems rare these days.
Day two of InsightShare’s PV MSC course offered this experience. After the story circle, one of my fellow trainees said she found it “enriching and liberating to have the opportunity to share my story in a safe and neutral space. It was the first time I’ve shared my story so freely… It was beautiful… to feel truly listened to.” Another said that sharing her story “made it real and opened new doors for reflection and decision-making”.
As for me, talking and listening sparked a palpable sense of connection in a room of near-strangers. We each described different things — but there were common experiences, similar worries. Even when there’s little in common, you’ve shared something of yourself that you maybe don’t even bother your best friends with.
It’s not hard to see how storytelling circles can build solidarity among a community and develop people’s confidence in speaking up. (Of course, the process can also be difficult and even traumatising; in some cases a trauma counsellor might need to be present.)
7. Selecting one story isn’t random.
This was the bit I struggled most with. How can one story ever be representative of a hundred or more? And what about all the detail you miss by focusing only on one story?
The point, though, is that the evaluation team have heard and analysed all the stories. They can choose what data to capture from them, and this can feed into a final report.
And in fact, it’s not really about the selected story being representative, but about being meaningful. Two metaphors for the MSC process help illustrate why it makes sense to focus on the meaningful:
“Do you remember the average things [about a holiday abroad] or the wonderful and terrible things? MSC helps teams of people focus on the memorable events and uses these events to help realign effort towards achieving more of the wonderful things and less of the terrible things. When the focus is on learning, we need to capture more than just the average experiences.”
“A newspaper does not summarise yesterday’s important events via pages and pages of ‘indicators’ (though they can be found in some sections) but by using news stories about interesting events… The most important stories go on the front page and the most important of these is usually at the top of the front page.”
From ‘The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use’ by Rick Davies and Jess Dart
And MSC is robust because of the breadth. Hearing a few stories from the field might be merely anecdotal, but as the InsightShare PV MSC toolkit explains, “when 50 or 300 stories or more are collected and analysed, meaningful patterns emerge.”
What about defining significant? As our group struggled to choose which of our six stories to select, Soledad suggested another metaphor: “think of a chrysalis becoming a butterfly”. Change is something that can’t be reversed.
Aside from this, what any given group decides is significant will be subjective — and that’s ok. Because the criteria that a group uses to select their story also says something about what matters to those people. That’s a valuable thing for any organisation to learn about those it’s trying to help.
References & further reading:
- http://insightshare.org/resources/participatory-video-and-the-most-significant-change/
- https://www.odi.org/publications/5211-msc-most-significant-change-monitoring-evaluation
- http://insightshare.wixsite.com/videogirls/the-pv-process
- http://www.mande.co.uk/docs/MSCGuide.pdf
- http://www.tools4dev.org/resources/the-most-significant-change-msc-technique-tool-review/
modern stories 