posted on: Thursday, August 20, 2009

by Lisa Ranghelli

Lisbeth Schorr’s recent commentary, “To Judge What Will Best Help Society's Neediest, Let's Use a Broad Array of Evaluation Techniques,” in the Chronicle of Philanthropy made an important argument about the need to recognize and validate the knowledge and expertise of individuals and communities as they seek to solve complex social problems. Schorr cautioned against a reliance solely on “evidence-based” programs, since this gold standard of evaluation is almost impossible to attain – precisely because multi-faceted problems and solutions are hard to reduce to simple, linear cause-and-effect relationships.

Advocacy and community organizing tap the myriad skills and knowledge of a variety of stakeholders, who come together and pool what they know to craft solutions to community problems and create the political will to implement them. Sometimes these can be messy, nonlinear processes, yet their tangible and intangible impacts can be measured in a variety of ways. And the learning from those efforts—both successes and failures—informs future change strategies.

As foundation leaders, academics and practitioners seek ever more ways to measure and assess impact, it will be important to not separate the people on the ground and their knowledge from the seemingly neutral tools of evaluation. FSG Social Impact Advisors’ comprehensive study, “Breakthroughs and Shared Measurement of Social Impact,” noted that new shared systems “offer an important complement to more rigorous evaluation studies by promoting ongoing learning in timely and cost-effective ways.”

Key elements of these measurement systems include: voluntary participation by the organizations providing data; independence from funders in developing and managing the system; and, in more advanced systems, an opportunity for participants to get together and talk about the results, share learning, and improve coordination. These participatory tracking processes allow everyone involved in a complex set of activities to benefit from the information collected. These approaches can help shift focus from unrealistic attempts to learn whether a grant to one organization to achieve a specific outcome was effective, toward less fragmented, more holistic examinations of collective impact. Perhaps convening participants is as important as data collection, so that their “on the ground” knowledge can inform data analysis and future planning based on what the data reveals.

A quick review of the publicly-available approaches highlighted in the report indicates that only a few are attempting to capture data related to advocacy, organizing and civic engagement. The Center for What Works/Urban Institute Indicators Project stands out for having a set of outcomes and indicators for both advocacy and organizing.

Would more attention to advocacy and organizing in shared measurement systems–and the associated collective learning that participants undertake—be helpful to advocates and organizers? Would these tools give more funders confidence to provide grants for these strategies?

Lisa Ranghelli is senior research associate at the National Committee for Responsive Philanthropy and co-author of Strengthening Democracy, Increasing Opportunities: Impacts of Advocacy, Organizing and Civic Engagement in North Carolina.