Insights

Impact measurement has shifted significantly over the past decade. Previously it was like a cottage industry with only a few specialist providers working with innovative organisations that were at the forefront of wanting to understand the change they were creating. Across each sector – business, government and for-purpose – there is now deeper understanding of the need to not only measure impact but to embed this into improved decision-making to ensure we are all managing for better impact. Here are 12 key lessons from Think Impact around how to improve your impact measurement.

A decade of change

Think Impact is now 10 years old. Over the past decade we’ve worked with around 400 organisations from across each sector – business, government and for-purpose. This has mainly been one-off projects, occasionally in partnership with others, but more recently we’ve seen a shift to multi-year engagements as each sector has become more sophisticated in their approach to measuring impact.

Alongside this shift there is the growing understanding that organisations need to be moving beyond measurement – ‘more than measurement’ has been a mantra of Think Impact over the past few years. We’re seeing this pattern elsewhere, definitely exemplified by the addition of the new principle of ‘be responsive’ by Social Value International (SVI). Be responsive means doing something about what you have learned from the measurement process so you make decisions that optimise social value.

At Think Impact we’ve framed this concept as ‘managing for better impact’. Ideally, we want to see all organisations moving along a continuum from recognising that they are having impact to focusing attention on managing for better impact.

We’ve reflected on our experience over the past decade to draw out a set of 12 key lessons we hope will support organisations to shift the dial from measuring impact to continuous improvement through managing for better impact.

1. Focus on impact not activity

Although impact is rising on the agenda for organisations, there is still a tendency when communicating about impact for people to have a strong focus on the activities being carried out rather than the impact of those activities. This is especially prevalent in impact reporting, where activity and outputs still dominate the narrative. Better impact reporting will have a focus on what has happened for people, communities or the planet as a result of the activities.

2. Stakeholder voice is critical

To genuinely understand impact, you have to engage with the people affected by a program, policy, initiative, service etc. It can be hard to do, particularly where stakeholders are difficult to reach or are affected by trauma.

If we do not have the voice of the people affected by an activity at the centre of our approach to impact measurement then we run the risk of maintaining power imbalances and continuing to do things to people rather than with them. Engaging stakeholders in evaluation or social valuation is critical to better understanding the outcomes achieved.

3. Get the right foundations in place

There are four core foundations or building blocks to good impact measurement. These are:

• a theory of change (integrated into your strategy) that outlines how you will bring about change
• an outcomes framework that outlines what data you need to collect to prove impact
• data collection and analysis processes that capture the data whilst allowing for reflection for continuous improvement
• impact reporting to stakeholders.

Organisations need all four of these foundations, having some of them is a good start but all four are needed if you want to manage your impact more effectively.

4. Don’t overlook the underlying issues

Developing a theory of change can be done as a solo exercise by the person responsible, by teams, by organisations or ideally involving stakeholders to get a fuller understanding of the outcomes.

A key component of any theory of change development process is going beyond looking at how you tackle the events that are leading you to take action and drilling down into more understanding the underlying issues that shape those events. This means looking into the structural, systemic and institutional aspects of any issues and even the belief systems (or mental models) that sustain those responses. This can be hard but vital if the aim of your organisation is to bring about sustained change.

5. Logic is necessary but not sufficient

When building a theory of change or program logic (and the two are not synonymous) a key component is applying logic to the cause and effect of the change. Then you have to consider the ‘people factor’!

People are not always logical or rational, they won’t always behave or think as expected – so when you are dealing with people, this invariably makes management of change complex.

6. Outcomes are a system not a list

It was very early on in Think Impact’s history that we learned this. We, and others, were used to drafting long lists of the outcomes being achieved in order to quantify the change that was happening for people, communities or the planet. But the more we worked with people around these changes we saw the interaction and complexity of cause–effect relationships between them.

We learned to ask the ‘so what?’ question frequently to drive better understanding of how early, intermediate and longer-term changes might be happening for people. Including identifying the occasions where an outcome needed to be sustained throughout as a core early, intermediate and long-term outcome for the intended impact to occur.

We also recognised that understanding the early changes was critical because if they didn’t happen, long-term change was very unlikely to happen. This provided a focus for monitoring systems to be set up in organisation with the focus of our impact evaluation work shifting to understanding whether the intended long-term change had occurred.

7. Beware of miracle leaps

We’ve developed or assessed more that 500 theories of change over the past decade either through working directly with clients or acting in an assurance capacity for impact reports.

One of the most common mistakes we see is the ‘miracle leap’ within a theory of change – if we do this then this will happen! Often it stems from having audacious goals or being overly optimistic. Whilst thinking big is to be encouraged, organisations need to ensure their theory of change outlines how they will realistically bring about their intended impact.

8. Modality

This is ensuring that alongside thinking about ‘what’ will change, you do the thinking about ‘how’ you will bring about change. The ‘how’ goes beyond thinking just about the planned activity but looking at the evidence around how frequently the activity is required, who it is best targeted at and whether there are any other aspects that are critical for success.

9. It is an iterative process

Whilst it can be satisfying to complete your theory of change or outcomes framework, like an unused strategic plan these shouldn’t be left to gather dust.

You need to have a mindset of continuous improvement around the impact aspect of your organisation whether you are a business trying to be more socially responsible, a government department commissioning goods/services or a for-purpose organisation where impact is your core business. As you measure and evaluate you will learn and (re)design what you are doing to make it better and achieve greater social impact.

10. Social value is context specific

Understanding the context you are attempting to achieve impact in is key.

It drives better understanding of the social value you are creating. I used to work in tackling long-term employment exclusion and often noted that supporting a person who was long-term unemployed to get a job in metro Melbourne was different to supporting a someone in regional Queensland – our employment system needs more local flexibility within it.

The same logic applies to thinking about social value, and in particular using social valuation techniques to monetise that social value. The social value for someone of a job in metro Melbourne will be different than the social value of a job for someone in regional Queensland. We need to build this nuance in the social valuation techniques that we are developing.

11. The impact prism

This works better as a visual. And I’m not going to attempt to explain the physics around light refraction through a prism!

How it relates to impact is in the context of impact storytelling. At Think Impact we’ve often seen organisations develop one way of telling their impact story and then it is repeated, regardless of the audience. This doesn’t make sense. What is more effective is to get clear on the impact story that needs to be told, and nuance that story to translate it to the different audiences you are communicating with. This requires good impact storytellers within organisations.

12. No data without stories, no stories without data

We are living in a data-saturated world with increasing amounts of short visual content. For impact stories to hit home then they need to blend data communication with great storytelling. Good impact reporting should be a great blend of data and stories.

Whatever type of organisation you are and wherever you are on your impact measurement journey, by embedding these 12 key lessons into how you are working you will be able to continuously improve the impact you are achieving – you will manage for better impact.