Community organisations lack the funding and data to measure their impact

Community organisations are struggling to measure the impact they are having due to a lack of funds and data availability, new research has found. It shows three quarters of community sector charities are trying to measure their outcomes and have increased this effort in the last five years.

However many are held back. Out of our survey participants of 190 community sector organisations, 90% cited a lack of funding and resources as the most significant barrier to measuring outcomes. Second to this was the lack of an established methodology and tools to assist them in measuring.

These community organisations saw the most important driver for measuring impact was not to meet the requirements of funders (important though that is), but to improve their services; to achieve their mission more effectively.

At last count, there were nearly 10,000 registered housing & development, health, and social services charities in Australia. These organisations had a combined income of A$46 billion and employed half a million people. They are in the front line of the delivery of social services.

The pressure is on for these organisations to show their impact as the government takes a priority investment approach to welfare. This is where government spending decisions on social programs are taken on the basis of the long-term economic and social impact of those programs (or more narrowly, the long-term budgetary impact of those programs).

When community organisations are funded to measure their outcomes, significant progress is made. There is a well accepted framework these organisations can use to break down what they do into individual service units with outcomes to be measured. This also involves data collection and assessment. In our study, 32% of community organisations had implemented this framework.

One example of a community sector measuring its outcomes is The Smith Family and its Learning for Life program. This program supports disadvantaged families by covering education-related expenses that aren’t covered by schools, as well as promoting long-term participation in education to Year 12 and subsequent engagement in education, training and employment.

As a result of establishing a data collection system and a research and evaluation framework around that, The Smith Family was able to demonstrate the effectiveness of the program. It improved school attendance, Year 12 completion and subsequent engagement in education, training and employment. This also increases their chance of securing funding and philanthropic support.

However beyond this specific framework, organisations need to use existing population-level administrative data better to understand the impact of social programs. Our survey shows this is not being done and yet there is enormous potential in this data and its analysis.

For example our recently released study of programs delivered under the Council of Australian Government’s National Partnership Agreement on Homelessness (NPAH) showed the program’s impact by linking the health records of program participants with their public housing records. From this we were able to examine the journeys of program participants before and after entering public housing.

The study revealed that those formerly homeless people accessing public housing through the NPAH programs, for the most part, sustained their housing. And as they did so, we found reductions in the use of emergency departments, in nights in hospital and in psychiatric care. This means the program would meet the aims of the government’s priority investment model, as cost savings were made to the public purse.

A challenge facing public policy is to ensure that where administrative data exists, records can be linked and in “real time”. This is so that policy makers are aware of the impact of the program, as it is occurring. A second challenge is to enable community organisations to access such data, or at least provide relevant organisation-level results, so that the impact of their own efforts can be better assessed.

The use of linked administrative data helps to support the task of impact assessment. But it does not remove the need for community organisations to measure their outcomes themselves. There is a large array of program outcomes that are not covered in population-level administrative data and can only be obtained at the organisational level.

In our study, we identified five key areas that need to be addressed in order for organisations to measure their outcomes, these were:

1. Funders requiring more outcomes reporting, with additional financial support
2. Open data from government, particularly in relation to linked administrative data, common infrastructure for data collection and a core set of common outcomes items.
3. The need for standardised language in outcomes reporting so that all stakeholders can understand them
4. Professional development within community service organisations and guidance on how to use standard tools.

If community organisations and government can address these areas, the social impact of organisations will be clearer.