We are increasingly challenged to demonstrate the worth of what we do in clear and understandable ways. We are asked to prove that our work is effective and that we are making meaningful and measurable changes in the lives of those we serve. “Measurable outcomes” has become the mantra of funders, supporters and participants of all sizes and shapes.
This seems to be a reasonable request. Who can argue with it?
And yet so many organizations, new and old, big and small have difficulty answering this challenge.
Is this because these are poorly run organizations, who would be more successful if they got their act together? Or is the challenge of measurable outcomes more complex than meets the eye?
Complexity is the word too often left out of the conversation.
A typical model put forward for developing an effective measurement system looks like this:
Inputs >>> Activities >>> Outputs >>> Initial Outcomes >>> Intermediate Outcomes >>> Longer term or Ultimate Outcomes
It reflects a flat, straight-line, mechanical model for a reality that is never so flat and never so simple. In an attempt to be easy to understand it blocks out complexity. In an effort to make it simple to demonstrate that it is what we do to get to desired outcomes, it almost assumes that we control what happens around us. It blocks out the real world we live and operate within.
In the real world we are challenged by the sheer complexity of the situations we are working in. And this complexity clashes with the desire for non-complex ways to judge effectiveness. The desire for simple and straight line understandings about how the things we do as service providers result in specific changes in the lives of those we serve, clashes with a human reality which says that few parts of our lives are flat and one dimensional. We are seeing this problem in the growing national debate about the effectiveness of public educational systems where there seems to be an inexorable demand on the part of public and private funders to see direct and one-dimensional connections between a teacher and the success of students. The appeal is that it is easy to collect data in this model of life and it seems to answer the accountability question that is connected to the desire for measurable outcomes. But in doing this simply, we are ignoring the complex set of forces which effect a child, a classroom, a school and a community.
Simple approaches to measurement are often made even more difficult by the expectation that results may be known in relatively short periods of time. Speed has become an unquestioned virtue but it may not be possible to know if what we do today with a young child will make a meaningful difference in their lives when they are a parent twenty years from now.
Measurement, real measurement of effectiveness is not simple and quick.
In a time of scarce resources and stiff competition this is not a popular message.
In an age of instant gratification, this says that it is not obtainable.
In a moment when faith and trust in organizations is weak, this reality demands more faith and trust.
Are we willing to recognize we may not be able to demonstrate effectiveness easily or quickly and begin a conversation about what we can do?