Fall 2010

Wanted: Better Evaluation Practices for a Better Philanthropy


Written by: Steven E. Mayer

Date: December 02, 2010

Philanthropy, by definition, is a noble endeavor. Generous gifts of time, talent and treasure have no doubt improved the world in countless ways. Whether the motive is to provide immediate relief of suffering, to support people in their development or to change systems’ ground rules to improve outcomes for even more people, philanthropy enjoys visibility and growth.

And yet, people want to know what good comes from their gifts. They want to know their gifts are in fact helping to make the world a better place. Congress, which covets the sector’s considerable assets, wants to know how to justify tax-exempt status. The pressure’s on to “show results.” Unfortunately, there’s a big disconnect between the knowledge of results we crave and the “knowledge” we get back from formal evaluation inquiries.

It seems simple enough. People want the data, knowledge and wisdom that assure we’re making progress on the big issues – issues like hunger, homelessness, injustice, ignorance. We want to know that the institutions we give to – our educational, arts and culture, social service, advocacy and justice institutions – are delivering. Does evaluation of results show our gifts make a difference? Are the measures of quality of life going up or down?  Are we “moving the needle” and “bending the trend lines?” Unfortunately, the field of evaluation, itself an evolving profession with noble intentions, has not geared its practices to answer these questions in a way that satisfies our hunger to know.

The problems with evaluation are rooted in problems with philanthropy

A concern for funding the well-being of individuals can prevent us from seeing the larger systemic problems that keep people suffering. When we see a hungry person, we want to feed him. When we see a homeless person, we want to house her. When we see an injustice done, we want to right it. However, this practice of rescuing casualties one a time keeps our vision too close to the shore where casualties are found, without enough focus upstream where casualties are created (see “Saving the Babies: A Clash of Philanthropic Approaches”.[1] Funding the rescue of one person at a time evokes evaluation practices that score success one person at a time, however temporary. Evaluation, asked to give an accounting of these rescue efforts, gives adequate information on how many people received soup, or affordable housing, or their day in court. These data may answer the question well enough, but they say nothing about the bigger question, “Are we stemming the flow of hungry, homeless or wronged people?” Both charitable and structural reform efforts are needed to stem these flows, with evaluation practices that honor the difference.

The typical practice of grantmaking – making one grant at a time, to one organization at a time, for one project at a time, for one year at a time – is a good way to keep order, but a poor way to support progress on the bigger issues. It keeps grantees’ efforts fragmented and focused on the small and the short-term, just so that they can report immediate results.  But the short-term outcomes of a single grant can’t possibly resolve the big issues. One soup kitchen’s outcomes will not move the hunger needle beyond the meal just served, and neither will the efforts of all soup kitchens combined.

Making nonprofit organizations compete for relatively small amounts of cash puts them in competitive rather than cooperative modes. To survive in the marketplace of institutional philanthropy, nonprofit competitors play their cards close to the chest, choking innovation and stalling momentum and consolidation of effort that could solve real problems.

The Lady Bountiful attitude that prevailed at the birth of American philanthropy still exists and still gums the works. An attitude of righteousness can creep into the philanthropic transaction, in which those seeking money to make the world a better place are made to feel like common beggars. This attitude is well-established in institutional philanthropy, as evidenced in the kinds of criticisms leveled at funders by their nonprofit supplicants: arrogant, tardy, distant, uncommunicative, condescending, etc.  As a colleague says, “Philanthropy is what allows a nonprofit  to stand on  its own two knees.”

Who owns philanthropy? For the answer, look at the board of directors of any philanthropic organization. Today’s form of philanthropic organizational governance comes directly from the private sector, drawing on forms from the early 20th century. “Scientific management,” all the rage then, took over American industry and from there, the institutionalization of human welfare and philanthropy itself. Scientific management advocated a standardized approach to the production of goods, managed from the top down, leading to consistent outputs that easily can be stamped out, measured and accounted for.  And if it works for the production of goods, why not services? This put evaluation firmly into the hands of the bean counters. Business managers on philanthropic boards today continue to focus on organizational goals, organizational outcomes and the organization’s bottom line. The organization’s contribution to solutions to larger problems is off the books and not considered. The field of evaluation, born under the same signs that birthed scientific management, serves up what it can, resulting in the big disconnect between the answers we get and the knowledge we want.

To improve the quality of philanthropy and evaluation, we have to reframe both

  1. Let’s start with recognizing that a better world doesn’t get built in a day. If we accept this, then we can escape the limitations of short-term management thinking, and go directly to problem-solving how the bigger issues can be addressed productively. Evaluation should redirect its focus in turn.
  2. For any given problem, like hunger, regional leaders of all kinds should adopt a set of metrics that reflect current progress. For example, they could adopt number of kids coming to school unfed, spoilage of healthful and nutritious food in stores, parts of town not getting enough and other related measures. Both charity and justice are reflected in those metrics. The goal of regional philanthropic efforts against hunger then becomes to move the metrics – “moving the needles and bending the trend lines.”  Finding and adopting a good set of metrics is no small task, but it’s important for anchoring a serious effort, for engaging community support and for keeping eyes on a jointly held prize. It also relieves individual nonprofits of unrealistic data demands. The point is to reflect productivity at the community or regional level, rather than the organizational level.
  3. Identify what moves these needles, and what keeps them from moving, to make clear what kinds of efforts to support. The social and institutional mechanisms that affect a needle’s movement and a problem’s solution are complex. Fortunately, most communities have people with a good view of the various working pieces of those puzzling mechanisms, and they can be engaged.
  4. Look beyond the capabilities and performance of one organization, and look instead to supporting a more organized and intentional effort. Getting metrics to change isn’t the work of just one organization. Moving the needle requires organizations that identify, advocate and move solutions; leadership that bridges divides; financial and technical resources; organizing a growing base of support – fit together over time in ways that can move the needle. Currently dismissed by both philanthropy and evaluation as “process” rather than “impact,” the successful implementation of each piece of needle-moving infrastructure must become legitimized; doing so recognizes the realities of improving the world and encourages specialized capacities to be developed and deployed in the service of community betterment. Productivity in philanthropy could become defined as the work required to move a needle, and be measured across contributing organizations.
  5. Think of evaluation as a human activity more than a scientific one. Noticing and naming simple signs of progress is the first and most important step in measuring it. We have to measure what’s important, not just what’s easily countable. Can we measure what’s important? The answer is “Yes,” though much depends on what is meant by “measure.” There’s merit in the old phrasing “to take the measure” of an effort – to size it up, sift through the evidence, get peoples’ judgments, compare notes and reckon where we’ve made progress and where we still need to push.
  6. The field has to recognize the truth about funder-grantee relationships – an awkward dance tending to belittle the supplicant and glorify the donor. Things would improve if donors constructed their job as creating cooperative partnerships, helping others succeed and building on lessons learned. Evaluation in this context means getting feedback on the better qualities of grantmaking practice, and paying attention to the results. Nonprofits should get up off their knees.
  7. Connect the dots. Just as funders should be assembling pieces of trend-changing infrastructure, evaluators should be building methods that connect organizational-level outputs with community-level impact on the adopted metrics. Can a case be made that the efforts of organization X, when joined with the efforts of organizations Y and Z and perhaps A through W, connect to the needles we’re trying to move? If not, funders have to redouble their efforts to put more pressure on the mechanisms that maintain the problem at problem levels, and to improve their ability to notice results.

Some parts of this vision are forthcoming. At the Community Foundation of Greater South Wood County in Wisconsin, staff position descriptions are geared to foundation goals and indicators of community vitality. At several large private foundations, such as the Bill and Melinda Gates Foundations, Packard Foundation, W.K. Kellogg Foundation and Hewlett Foundation, evaluation staff work to construct evaluation as a “core learning practice,” helping program staff make sense of measured experience with foundation grants and initiatives. The Boston Foundation and Community Foundations of Canada make good of public data to guide philanthropic strategy. Foundations using the Center for Effective Philanthropy’s Grantee Perception Report can compare their performance with others’. Examining the root causes of social problems, inequities and injustices is becoming acceptable. But assembling actionable pieces to apply pressure on a set of community-held problem indicators, yielding a wider view of productivity, is an approach in need of promotion.


Some of the above themes are sounded admirably in Grantmakers for Effective Organizations’ 2009 publication, Evaluation in Philanthropy.

The complexity of the world and its implications for philanthropy and evaluation in support of change is well-noted in Blueprint Research & Design’s (2005) publication, “The Challenge of Assessing Policy and Advocacy Activities: Strategies for a Prospective Evaluation Approach.”

FSG Social Impact Advisors’ “From Insight to Action: New Directions in Foundation Evaluation” (April 2007), spotlights emerging approaches to evaluation in philanthropy.

The website shows how the arena of racial inequities is approachable by philanthropic practice that deliberately tries to close the disparity gaps.

Steven E. Mayer, Ph.D. is the director of Effective Communities Project and can be reached at