It’s usually a moment of dread for the leader of a nonprofit. One of your major funders commissions a study, and they inform you they want you to spend who-knows-how-many hours working with the consultants they hired. “Damn,” you think to yourself. “There goes a whole bunch of hours down the drain.” You have to say yes and agree to participate, of course, or risk alienating that funder. But most of the time, if we’re being honest, the study that was commissioned has little or no relevance to how you do your work.
I was pleasantly surprised, therefore, when the field scan commissioned by the William and Flora Hewlett Foundation turned out to be tremendously helpful to NCRP, our strategic planning process and our plans for future work.
Peer to peer: At the heart of influencing more effective philanthropy is a first-of-its-kind study of how U.S. foundations access and use knowledge. It involved more than a dozen grantees of Hewlett’s Effective Philanthropy program, including NCRP, GEO, CEP, Bridgespan and more.
One of the scan’s most significant findings is that foundation leaders are more likely to be influenced by their peers and colleagues than by any other single factor. Learning this information last year, while NCRP was in the process of strategic planning, helped us commit to double down on using an organizing approach to our work under our new strategic framework.
Another key way foundation staff acquire knowledge is at conferences, the scan revealed. Knowing this makes it easier to justify the huge budget allocation at NCRP for us to attend and present at more than 50 philanthropy conferences each year. Thanks to the field scan, we don’t just have anecdotal evidence now that this investment is worth it, we have hard data.
There are many other interesting findings in the study, and I encourage anyone interested in improving philanthropic practice to read it.
But I want to focus for a minute on a slightly different question: Why was the end product useful to us? Too many foundations commission studies that are not useful to grantees. So what did the Hewlett Foundation do differently that made their effort a success?
The most important thing Hewlett did was to engage the grantees up front in thinking through what questions the study should seek to answer. Fay Twersky and Lindsay Louie listened to our input. They didn’t have the whole plan fully baked and then pretend to listen to us – they listened first, and only then designed the study.
Additionally, Hewlett spent enough time and money on the study to actually answer the questions we wanted to answer. Too often, foundations have limited budgets for their studies, or they have arbitrarily short timelines. When that is the case, a foundation will naturally design the study to give them the information they need when they need it, but they don’t make the adjustments necessary to make the study useful to grantees. I don’t know how much Hewlett spent on the field scan, but I’m sure it was a lot. The investment was probably worth it because the foundation has invested more than $35 million in grants in this program over the past 17 years and they are committed to millions more in grants in the coming years. In other words: If you’re going to pinch pennies on a study, it’s probably not worth doing it at all.
Finally, Hewlett gave us data privately that was specific only to our organization. More than 700 people responded to the survey, and hundreds answered questions specifically about what they like or don’t like about NCRP’s knowledge products. Having that information has been really helpful to our team as we seek to make our products as useful as possible. In past years, we have paid thousands of dollars to have an independent firm get that kind of data to inform our work. The Hewlett study, and the fact that they shared all the data with us, means we won’t have to pay for this kind of feedback for several more years.
Funders, if you’re going to commission a study of your grantees, you should do the three things Hewlett did in this particular case: 1. Engage grantees up front in designing the study and determining what questions are worth answering. 2. Spend the time and money necessary to actually answer the questions. 3. Share as much data with the grantees being studied as possible. If you’re not committed to doing it right, please don’t waste everyone’s time with another foundation-commissioned study. We all have more important things to do.
Aaron Dorfman is president and CEO of NCRP. Follow @NCRP on Twitter.