The Notah Begay III Foundation (NB3F), which seeks to reduce obesity and Type 2 diabetes among Native American children, partnered with Indigenous Methods, LLC to develop an evaluation plan for Native Strong, its grantmaking program that provides tribal communities with the tools and information they need to improve children’s health.In Native communities across the country, childhood obesity and Type 2 diabetes have grown in the past decades. Studies show that Native American youth, ages 10-19 are nine times more likely to have Type 2 diabetes than white youth of the same age. The incidence of Type 2 diabetes in American Indian youth between the ages of 15 and 19 more than doubled between 1990 and 2009.1
NB3F’s work is based on the understanding that, in order to create meaningful, sustainable change, it must authentically engage community partners2 and the communities they serve in a collaborative process that meets partners where they are, acknowledges the context in which they work and works with and in communities, rather than on them. The foundation’s approach to working with its partners involves acknowledging the profound strengths and resources that Native American communities bring to this work and the barriers they may experience in efforts to mobilize these resources to affect positive change. It assumes that individual communities know who they are and what they need and that they hold valuable resources in their values, culture, history and stories. At the same time, the history of colonization, generations of trauma and pervasive devaluing of Native ways can mask these assets and make it difficult for a community to clearly see its assets and understand how to use them in effective ways to improve health.
Indigenous Methods’ task was to find a process for defining and developing metrics that are flexible and diverse enough to fit the many different projects and communities involved with Native Strong and yet structured and rigorous enough to provide data that can be aggregated for a meaningful assessment of Native Strong’s overall grantmaking. Additionally, we needed to outline methods for the gathering and analysis of data that were effective and easy to use and metrics that are meaningful to both Native American communities and mainstream funders and researchers. Finally, the metrics needed to be created in such a way that would support the work and its impact rather than hinder success through lack of alignment with the work itself or the values of the communities where the work takes place.
Using Indigenous Methodology
Before we could define what to measure or how to measure it, we needed a model of health that reflects the experience and values of NB3F and the communities in which it works. The Indigenous Health Model (see Diagram 1) we built is represented as a convergence of multiple components bound together in a spider web. The image demonstrates how the various components are connected as parts of the whole and how changes in any area will affect other areas and components of that web.Indigenous Methods and NB3F made the decision that, rather than providing technical assistance to community partners to help them engage in mainstream evaluation techniques, we would instead develop an evaluation plan based on approaches more congruent with those of the communities served and demonstrate the rigor of this approach to funders and others unfamiliar with it.
This model of health includes an ecological systems model, drawing upon the work of Urie Bronfenbrenner3, but uses indigenous terminology and perspectives at each of the ecological levels. The Individual Level speaks to a subjective perspective and an awareness of individual biases, understandings, relationships and assumptions. This is the level at which the individual asserts personal agency and choice. The Family Level references familial connection, including clan, kiva, moiety and extended family connections. It involves people who are related to the individual and have a strong influence on the individual’s actions and reflections. The Community Level represents the community in which the individual and family reside and/or are most closely associated. The community dictates norms and values and can have a powerful influence on the individual through the framing of meaning and value conveyed in cultural and traditional understandings. The final level is Creation. This level includes all that exists beyond the community, what binds everything together: the ecology, environment, mainstream culture and messages. It adds an important cultural element missing from Western models and allows for a broader understanding of the contextual influences and impact of the work.
Understanding this complex and intertwined relationship is key to understanding multiple impacts from projects and programs. Seeing health through this lens acknowledges that the health of each individual is influenced by the people around him or her and the context in which that person lives as well as by personal choice. This not only removes blame from those who are unwell, avoiding shame and stigma, but also demonstrates that actions taken at the levels of family, community or creation can have a profound impact on the health of individuals, and that it is appropriate to use metrics that look at these levels to evaluate a program. Each community partner is urged to select metrics that fit this model and are also appropriate to the organization’s program and vision of success.
Among indigenous peoples, the four aspects of health – physical, mental, emotional and spiritual – are significant determinants of healthy behaviors and community sustainability. Thus any health-related project or program must take into account all of these aspects and have awareness of how each aspect is affected by its work.
In indigenous communities, identity and meaning are defined and understood through relationships and are stored and reflected in language, culture, stories and ceremony. In the modern Western model, the individual can determine his or her identity in isolation, choosing to “be whoever you want to be.” In indigenous cultures, a person is defined through relationships with family, community and creation.
Using this indigenous understanding of health, we went on to create a simple model that shows how the relationship among planning, project implementation and project evaluation is ongoing and that evaluation is something we all do naturally, though we may not document the process carefully. The latter point was essential in order to demonstrate to community partners that evaluation is not something alien and that, if properly organized and recorded, the ways in which they already assess their work can be built into a rigorous evaluation structure that will provide the necessary data for reporting on their programs and also help them to structure and understand the lessons learned from their work.
The Indigenous Model of Planning and Evaluation (see Diagram 2) shows this work as an iterative cycle in which the lessons learned in one cycle inform the choices of the next. This reflects an understanding that, while we don’t always know what actions will successfully address a particular concern, what we can do is develop an idea, try it out, see what happens, learn from the experience and refine our choices. As long as we are learning, our work is successful and we are moving closer to a solution.
Next, we looked for an already tested evaluation framework based on Native American culture and chose the one developed by the American Indian Higher Education Consortium4. This framework is guided by five main principles: context is critical, place-based, recognition of gifts, centrality of family/community and Nation building.
To bolster this evaluation framework, we incorporated the Seven Directions approach developed by Red Star Innovation’s Tribal Public Health Institute (TPHI) Feasibility Project, which examines the potential roles for a tribal public health institute in improving health among American Indian and Alaska Native communities5.
- Knowledge: access and use data and information in a meaningful way.
- Service: develop internal capacity, “grow your own.”
- Governance: strengthen public health authority as a function of sovereignty.
- Sovereignty: expand advocacy and influence on federal policy.
- Culture and identity: reclaim, revitalize and reaffirm indigenous knowledge and traditional practices.
- Integration: make important connections and collaborations to integrate public health and health care systems.
- Families and communities: create healthy environments that support well-being.
Our model also incorporates the indigenous realms of knowledge described by Marlene Castellano6: traditional knowledge, empirical knowledge and revealed knowledge. Traditional knowledge encompasses the critical learnings and teachings that emerge from stories and cultural engagements as passed on through multiple generations of families, clans and community members. Empirical knowledge encompasses the learnings that can be explained through observation and experimentation. Revealed knowledge encompasses the learnings gained from prophesy or spiritual revelations. In a modern context, we can adjust this to mean intuited knowledge that can be elusive and difficult to explain without connections to the other two domains.
Applying the Evaluation Process
This basic structure is only the beginning. In working through an indigenous lens, it is important to include the knowledge and experience of all individuals and communities involved in the process: the NB3F staff, the staff of community partners and members of the communities where the projects take place. Additionally, the process of planning, development and assessment is ongoing and cyclical. Thus the evaluation plan must be a living, evolving structure that is able to develop and change as new information becomes available.
Recognizing that the community partners that are most successful at engaging and working with their communities may never be fully conversant with the theory and vocabulary of program evaluation and may not be comfortable or adept at reporting on their work in written form, we suggested that NB3F adapt its application and evaluation process to help partners succeed with this aspect of the work. Suggestions included the use of less-specialized vocabulary in documents that ask community partners to talk about their work, trainings in how to integrate evaluation into their programs from the outset to make the process less onerous, and the implementation of procedures to formally capture information gathered from partners through conversations and interviews, so that those who are better at expressing themselves verbally than in written form can still report fully on the work.
Ultimately, the development of a system, process and metrics for an indigenous health evaluation model is centered around aligning the work or, in this instance, the grantmaking to the way in which that work is measured. Too often, Native communities are forced to analyze and assess their work in a way that does not support the work they have accomplished in the community. In fact, modern Western evaluation methodology can undermine the work by using metrics that are not reflective of the Native context, such as culture, ecology, environment and history.
The model that Indigenous Methods has cultivated is an example of how community and cultural engagement has the potential to yield far richer information, which can be utilized in more effective and efficient ways to the benefit of indigenous communities.
Robert Sturm and Lee Francis IV are consultants at Indigenous Methods. Write to robert@teoxihuitl.com.
1. “Diabetes in American Indians and Alaskan Natives: Facts at a Glance,” Department of Health and Human Services, Indian Health Service Division of Diabetes Treatment and Prevention, 2012, https://www.ihs.gov/MedicalPrograms/Diabetes/HomeDocs/Resources/FactSheets/Fact_sheet_AIAN_508c.pdf
2. One of the decisions made during the process of updating the foundation’s grantmaking model was to replace the word “grantee” with the term “community partner” to acknowledge shared learning, responsibility and goals.
3. Urie Bronfenbrenner, “Developmental research, public policy and the ecology of childhood,” Child Development, March 1974, Vol. 45, No. 1, pp. 1-5.
4. Joan LaFrance and Richard Nichols, Reframing Evaluation: Defining an Indigenous Evaluation Framework (Alexandria, VA: American Indian Higher Education Consortium, 2009).
5. Judy Beaudette and Allison Matsumoto (editors), Seven Directions: A Blueprint for Advancing the Health and Wellness of Our Native Communities (Tuczon, AZ: Red Star Innovations, 2015),
6. Marlene B. Castellano, “Updating aboriginal traditions of knowledge,” in G. Dei, B. Hall & D. Rosenberg (eds.), Indigenous Knowledges in Global Contexts: Multiple readings of our world (Toronto: University of Toronto Press, 2000), pp. 21-36.