Insights

Getting the building blocks for good impact measurement in place isn’t easy. It has long been understood that given the diversity of organisations within the for-purpose sector in terms of focus, size, range of activities, people affected by those activities, maturity of org, scale of work, geographic spread, etc., it is virtually impossible to get ‘off-the-shelf’ products that support organisations to understand, measure and communicate their impact. In most, if not all, cases they need to be tailored to be fit-for-purpose. Ultimately though getting the building blocks in place provides organisations with the ability to prove and improve their impact. This is becoming increasingly important for organisations of all sizes in their quest for funding, social procurement or investment.

Here I outline how to take an evaluative approach to developing the building blocks for impact measurement and why this is possibly the best option.

Lack of foundations – a common occurrence

I’ve been at Think Impact for over eight years now. One of my very first clients was a community housing provider who came to us with a particular problem. Essentially they had spent money developing an outcomes framework which had left them very perplexed! The outcomes framework had over 220 indicators that they had to measure against but no measurement tools for data collection and they were uncertain how to get started. They had also been asked by a funder for their theory of change, and they didn’t have one linked to their outcomes framework.

We’ve experienced variants of this many times – organisations not getting all the building blocks for impact measurement in place. Or often stopping when they have a theory of change or outcomes framework and never actually moving on to understand, measure and communicate their impact.

The building blocks of impact measurement

We recognise that different terms are used – social impact framework, impact measurement framework, monitoring and evaluation framework, impact framework, etc. At Think Impact we see each of these as ways to describe the building blocks required to understand and communicate your impact.

These building blocks are having a theory of change, an outcomes framework, a process for data collection and analysis, and then reporting on your impact to key stakeholders.

• A theory of change assists you to think about how your activities will bring about change for people or the planet. It enables you to tell your story of change.
• An outcomes framework is a tool used to help you think though what you need to measure to understand if the intended change is occurring. This can include key evaluation questions.
• Data collection is the process of gathering the outcome data to analyse.
• Communication of the outcomes achieved is done through impact reporting.

Each organisation usually needs the building blocks tailored to ensure they are fit-for-purpose for the size, scale and maturity of the entity.

Taking an evaluative approach to develop the building blocks

Whilst these building blocks can be developed conceptually, we’ve been advancing an evaluative approach over recent years that we think is supporting organisations to build more fit-for-purpose solutions. The key advantage is that alongside developing the building blocks of impact measurement, the organisation is also evaluating their impact. This means that you are involving stakeholders in shaping the impact framework, which will ensure it is more fit-for-purpose. It also allows organisations to tell a stronger, evidence based story of the change they are creating when speaking to funders, partners, commissioners or investors.

A recent example of using this approach was with ShoreTrack. We worked with them as their evaluation learning partner.

At the initial stage of the project we immersed ourselves in their work to understand their vision, purpose, motivations, values, context, challenges and business model. Alongside this we engaged with a range of their key stakeholders and also observed them in action. This allowed us to develop a draft theory of change and outcomes framework.

Following this we moved into a more formalised formative evaluation, identifying a wider range of stakeholders to engage with, using a mixed methods approach to further understand their impact.

Choosing a developmental or formative evaluation

When evaluating to also develop a social impact framework, either a developmental or formative evaluation works best. The diagram below provides a very high-level overview of different types of impact evaluations.

A formative evaluation was chosen for ShoreTrack as it was recognised they were a relatively new organisation but had moved beyond start-up phase. The formative evaluation involved one-to-one interviews and focus groups with the young people and key stakeholders in the community. ShoreTrack also administered a survey of their young people and the quantitative data collected by ShoreTrack was analysed.

The evaluation identified the impact that ShoreTrack was achieving for the young people it supported and the flow-on effect into the local community. In addition, a series of recommendations were made for improving the service.

The finalised social impact framework (SIF)

We provided capability building support to ShoreTrack so they could embed their SIF into business as usual. Each of the building blocks are now in place with a theory of change that has been shaped through engagement with the young people, an outcomes framework that is set up to measure change across three key domains, data collection methods that are fit-for-purpose for the young people that are using the service, and a template for impact reporting that allows ShoreTrack to communicate their impact to their wide range of stakeholders.

As ShoreTrack moves forward to use the SIF they will be able to iterate it, incorporating learning from the impact measurement they are doing. This will allow them to not only continuously improve the services they are offering but the processes for impact measurement.

It was great to work with Think Impact as our evaluation learning partner. The process we followed with them really helped us to understand, measure and evaluate the success we are having. Just as importantly, we can convey this message to different stakeholders in the language of the sector. Overall, we are really happy to have a Social Impact Framework that our team can use to continue our impact measurement journey. – Jill Ashley, CEO of ShoreTrack

If you want to know more or to talk through the option of doing an evaluation to support developing your impact framework, don’t hesitate to get in touch with us at hello@thinkimpact.com.au.