Evaluating community-based programs: challenges and lessons from the front-line

Over recent years, government funding for communitybased support programs has increasingly been directed through non-government organisations (NGOs) – with an obligation on them to evaluate and report on the effectiveness of funded programs, rather than just planning and implementing them. While this allows for more locally-relevant and responsive programs, which most NGOs are comfortable developing and delivering, the emphasis on formal evaluation and reporting of program effectiveness can be more challenging – for both NGOs and external evaluators. The Centre for Children and Young People has supported a number of government-funded NGOs in meeting such evaluation obligations. This paper will outline the range of programs supported, the evaluation methods used and the various findings. It will also highlight some of the philosophical, practical and ethical challenges (faced by the NGOs and by the Centre), how we have approached these, what we have learned in the process and recommendations for future approaches. Examples of issues to be discussed include the types of data NGOs collect, their previous experiences with external evaluators, their need to tailor programs to meet clients’ needs, funding levels and timeframes, concerns about diverting resources from service delivery and ethical issues which all constrain the use of “standard” outcome-focussed evaluation methods. Furthermore, evaluation implies a shift in focus from “making” a difference to being able to “demonstrate” it and the dependence of future funding on demonstrated program effectiveness can create pressure for “positive” results. The Centre’s approach in dealing with these issues in a rural and regional area is outlined.