Insights

Suicide, family violence, homelessness, inter-generational poverty: they are all huge social battles that place grantmakers on the frontline of life-and-death issues.

But leading social impact thinker Ross Wyatt, managing director of Think Impact, said many policymakers, funders and grantseekers are trapped in an unproductive cycle of using social impact evaluations to prove what they did was right, rather than using them to learn how to do things better.

He said grantmakers and recipients can be held hostage to factors that work to encourage evaluating to justify, rather than evaluating to learn. These include:

• failing to collaborate with beneficiaries, and focusing on doing things for them instead
• relenting to pressures to “evaluate to justify”, so a program can be deemed a success
• an excessive focus on funding and propping up income, instead of a focus on program impact
• an obsession about “attribution” (who should take the credit) rather than “contribution”
• pressure to avoid failure and a related lack of tolerance for risk
• diving into the data flood and gathering information for its own sake without properly understanding appropriate measures
• producing an evaluation that’s inappropriate for the scale of the program, i.e. being too small, too large, or too costly.

Each of these misfires can cause harm to programs that are supposed to help.

“On average, eight people are committing suicide in Australia every day, and every week a woman is killed by an intimate partner … people are dying under the current systems,” Mr Wyatt said.

He said the fact social indicators continue to head in the wrong direction in Australia means we can’t keep assessing programs the same way and expect a better result.

There are many reasons why we’re squeamish about facing up to what’s not working well, and government-funded programs are often the worst culprits.

Think Impact director, Suzi Young leading a consultation session for a women’s housing organisation.

Those programs have a “low tolerance for failure, and a low tolerance for risk”, Mr Wyatt said. The reasons are partly political, with the aversion then reflected in policies, he said. Funding programs may face pressure from elected officials to be seen to be doing something effective, and so must come up with results that demonstrate this.

By the same token, organisations that are being funded, and their beneficiaries, may produce the results they believe are expected of them – again reinforcing existing programs. Mr Wyatt’s observations of several hundred evaluations in recent decades reveal a pattern of seeking to prove that “everyone is doing a great job”. The perverse thing, he said, is that the continued survival, or even success of many organisations relies on the continuation of the very problem they working to eradicate. He gave the example of a homelessness services provider that attracts funding to tackle the issue. If it succeeds in reducing homelessness, then funding will dry up. “It’s a strange realisation but the very survival of these organisations depends on the misery continuing.”

Why it’s time to fight the inertia of the status quo

Mr Wyatt cited the example of a program that aims to help isolated Bhutanese migrants adjust to a new life in Australia. Many are recovering from trauma and struggling to settle into life in regional Australia, and many face significant language barriers and even struggle to understand what’s on television. The program sees volunteers collect migrants on a bus each week to bring them to the local library to draw. This has been their main social activity for nearly a year.

It is of course a welcome respite, yet Mr Wyatt said the grateful migrants are loath to complain or push for a better program. Instead, they’re quick to tell program leaders they are happy with the help they are getting. They think that’s what they want to hear. This is an example of a program that would be easy to “evaluate to justify”: by demonstrating the volunteer effort and the satisfaction of the Bhutanese, and by showing that community connections have improved. But Mr Wyatt asks what would happen if the community themselves were placed at the centre of the program design and evaluation. "I suspect we might see a different type of program emerge", he said.

The revolution is coming: Collaboration for impact versus competition for funding

Mr Wyatt is calling for a “revolution and an evolution in program design and evaluation”. He summarised the keys to effective evaluation that can make a difference:

• Collaborative approaches, especially those with clients “at the centre”
• Unlocking and building the capacity for change
• Supporting work on system change, including advocating for policy changes when needed
• Focusing on funding impact, not activity
• Focusing on contribution, not attribution, when looking at who should be taking credit for programs

New models of program design and evaluation are increasingly looking further to the horizon in assessing the success of projects, particularly in the most difficult challenges.

Citing homelessness again, Mr Wyatt said it is a complex issue that can’t be tackled in isolation. It instead requires new equity and finance models, building and design, local government zoning and policies, street-level social work and a community-wide realisation that housing is a fundamental human right, not just a vehicle for building personal wealth. This systemic approach will get better results in the longer term and will do more for the economy as a whole.

In his office, this type of joint effort is described as “impact-led co-design”, pulling together community organisations, policy makers, government agencies, donors and others in a collective focus on impact. This kind of collaborative mindset helps to ensure that those involved develop a nuanced understanding of difficult issues such as homelessness, suicide, mental health and family violence, rather than a reactive one. “You’re no longer just a bureaucrat writing grant criteria, but someone who understands what paths there are for dealing with complex issues like homelessness.”

Mr Wyatt said the organisations that are less focused on competition for funds or the battle to attribute credit are the organisations more likely to attract support. They become experts and can speak with governments and funders “as equals”. “Every organisation that we’ve worked with to focus on impact, rather than survival, is flourishing financially (because) those who are focused on learning and maximising impact make for a more attractive funding proposition.”

With thanks to Matthew Schulz for writing this article and the Australian Institute of Grants Management, an enterprise of Our Community, for publishing this article. Read the full version here.