Data Collection Methods

This section includes a number of resources, tools and tipsheets on many forms of data collection used in evaluation: surveys, focus groups, group data, community mapping. There are also resources on some other methods, such as ethnography and story-gathering. Some of the resources also show how particular methods have been adapted for particular issues – for example, evaluation of advocacy. For more, see also the sections on Inclusive Evaluation and Evaluation Basics, and the tipsheets on How Can We Design Focus Groups to Give Us the Best Information Possible? and [...]

Resources

Title
Author
Organization
Sampling
Ellen Taylor-Powell
University of Wisconsin Cooperative Extension
Collecting Evaluation Data: An Overview of Sources and Methods
Ellen Taylor-Powell and Sara Steele
University of Wisconsin Cooperative Extension
We Did it Ourselves: An Evaluation Guidebook (Chapter 10)
SRI International; Sierra Health Foundation
Best Practices for Survey and Public Opinion Research
The American Association for Public Opinion Research
Survey Design
Creative Research Systems
Basics of Conducting Focus Groups
Carter McNamara, MBA, PhD
Authenticity Consulting, LLC
Getting Inside the Story: Ethnographic Approaches to Evaluation
Craig McGarvey and Toby Volkman
GrantCraft
Measuring Impact Isn’t for Everyone
Mary Kay Gugerty and Dean Karlan
Stanford Social Innovation Review
Lean Data Approaches to Measure Social Impact: Use Low-Cost Technology to Gain High-Quality Customer Data and Insights
Acumen
2 of 5 - By the Numbers: Using Disaggregated Data to Inform Policies, Practices and Decision-Making
Catherine Mong et al.
The Annie E. Casey Foundation
Why Am I Always Being Researched?: A Guidebook for Community Organizations, Researchers, and Funders to Help Us Get from Insufficient Understanding to More Authentic Truth
Chicago Beyond, Equity Series

Tipsheets

Title
Author
Organization
How Can We Design Survey Interviews and Questionnaires to Give Us the Best Information Possible?
CAPD
How Can We Design Focus Groups To Give Us The Best Information Possible?
CAPD
What You Should Do If You Haven’t Gotten a Respectable Response Rate
University of Wisconsin Cooperative Extension
Collecting Group Data: Nominal Group Technique
University of Wisconsin Cooperative Extension
Collecting Group Data: Delphi Technique
University of Wisconsin Cooperative Extension
Focus Group Interviews
University of Wisconsin Cooperative Extension
Collecting Group Data: Affinity Diagram
University of Wisconsin Cooperative Extension
Survey Procedures
University of Wisconsin Cooperative Extension
Methods for Collecting Information
University of Wisconsin Cooperative Extension
Record the Decisions You Make with Your Data
University of Wisconsin Cooperative Extension

Tools

Title
Author
Organization
Community Mapping (Equitable Development Toolkit)
PolicyLink
A Handbook of Data Collection Tools: Companion to “A Guide to Measuring Advocacy and Policy”
Jane Reisman, Anne Gienapp, and Sarah Stachowiak
Organizational Research Services; The Annie E. Casey Foundation
Moving Toward Equity: Data Review Tool – Getting Started with Equitable Access Data
Center on Great Teachers & Leaders (GTL) at American Institutes for Research

“We act as if simple cause and effect is at work. We push to find the one simple reason things have gone wrong. We look for the one action, or the one person, that created this mess. As soon as we find someone to blame, we act as if we’ve solved the problem.”

~ Margaret J. Wheatley

SPOTLIGHT

Using Data Collection to Inform Equitable Grantmaking – Keecha Harris and Associates

Also in this section:
  • Collecting Data

  • Information Sources

GLOSSARY

Cultural Appropriation

Theft of cultural elements—including symbols, art, language, customs, etc.—for one’s own use, commodification, or profit, often without understanding, acknowledgement,or respect for its value in the original culture. Results from the assumption of a dominant (i.e. white) culture’s right to take other cultural elements.


SOURCE:  Colours of Resistance Archive, “Cultural Appropriation (accessed 28 June 2013).

Cultural Racism

Cultural racism refers to representations, messages and stories conveying the idea that behaviors and values associated with white people or “whiteness” are automatically “better” or more “normal” than those associated with other racially defined groups. Cultural racism shows up in advertising, movies, history books, definitions of patriotism, and in policies and laws. Cultural racism is also a powerful force in maintaining systems of internalized supremacy and internalized racism. It does that by influencing collective beliefs about what constitutes appropriate behavior, what is seen as beautiful, and the value placed on various forms of expression. All of these cultural norms and values in the U.S. have explicitly or implicitly racialized ideals and assumptions (for example, what “nude” means as a color, which facial features and body types are considered beautiful, which child-rearing practices are considered appropriate.)


SOURCE:  RacialEquityTools.org


Related Resources:  Racism (scroll down alphabetically to the box for “Cultural Racism”)

Location: FUNDAMENTALS / Core Concepts

Ethnicity

A social construct that divides people into smaller social groups based on characteristics such as shared sense of group membership, values, behavioral patterns, language, political and economic interests, history, and ancestral geographical base.


Examples of different ethnic groups are: Cape Verdean, Haitian, African American (Black); Chinese, Korean, Vietnamese (Asian); Cherokee, Mohawk, Navaho (Native American); Cuban, Mexican, Puerto Rican (Latino); Polish, Irish, and Swedish (White).


SOURCE:  Teaching for Diversity and Social Justice: A Sourcebook, edited by Maurianne Adams, Lee Anne Bell, and Pat Griffin, Routledge, 1997.


Related Resources:  Race, Ethnicity, and Indigeneity (scroll down alphabetically to the box for “Ethnicity”)

Location: FUNDAMENTALS / Core Concepts

Inclusion

Authentically bringing traditionally excluded individuals and/or groups into processes, activities, and decision/policy making in a way that shares power.


SOURCE:  OpenSource Leadership Strategies

People of Color

Often the preferred collective term for referring to non-White racial groups. Racial justice advocates have been using the term “people of color” (not to be confused with the pejorative “colored people”) since the late 1970s as an inclusive and unifying frame across different racial groups that are not White, to address racial inequities. While “people of color” can be a politically useful term, and describes people with their own attributes (as opposed to what they are not, e.g., “non-White”), it is also important whenever possible to identify people through their own racial/ethnic group, as each has its own distinct experience and meaning and may be more appropriate.


SOURCE:  Race Forward, “Race Reporting Guide” (2015).

Racial and Ethnic Identity

An individual’s awareness and experience of being a member of a racial and ethnic group; the racial and ethnic categories that an individual chooses to describe him or herself based on such factors as biological heritage, physical appearance, cultural affiliation, early socialization, and personal experience.


SOURCE:  Teaching for Diversity and Social Justice: A Sourcebook, edited by Maurianne Adams, Lee Anne Bell, and Pat Griffin, Routledge, 1997.


Related Resources:  Race, Ethnicity, and Indigeneity (includes “Resources by Specific Racial/Ethnic Groups”)

Location: FUNDAMENTALS / Core Concepts

Structural Racialization

Structural racialization connotes the dynamic process that creates cumulative and durable inequalities based on race. Interactions between individuals are shaped by and reflect underlying and often hidden structures that shape biases and create disparate outcomes even in the absence of racist actors or racist intentions. The presence of structural racialization is evidenced by consistent differences in outcomes in education attainment, family wealth, and even life span.


SOURCE:  Systems Thinking and Race: Workshop Summary by john a. powell, Connie Cagampang Heller, and Fayza Bundalli (The California Endowment, 2011).


Related Resources:  Structural Racism

Location: FUNDAMENTALS / Core Concepts

Whiteness

  1. The term white, referring to people, was created by Virginia slave owners and colonial rules in the 17th century. It replaced terms like Christian and Englishman to distinguish European colonists from Africans and indigenous peoples. European colonial powers established whiteness as a legal concept after Bacon’s Rebellion in 1676, during which indentured servants of European and African descent had united against the colonial elite. The legal distinction of white separated the servant class on the basis of skin color and continental origin. The creation of ‘whiteness’ meant giving privileges to some, while denying them to others with the justification of biological and social inferiority.

  2. Whiteness itself refers to the specific dimensions of racism that serve to elevate white people over people of color. This definition counters the dominant representation of racism in mainstream education as isolated in discrete behaviors that some individuals may or may not demonstrate, and goes beyond naming specific privileges (McIntosh, 1988). Whites are theorized as actively shaped, affected, defined, and elevated through their racialization and the individual and collective consciousness formed within it ... Whiteness is thus conceptualized as a constellation of processes and practices rather than as a discrete entity (i.e. skin color alone). Whiteness is dynamic, relational, and operating at all times and on myriad levels. These processes and practices include basic rights, values, beliefs, perspectives, and experiences purported to be commonly shared by all but which are actually only consistently afforded to white people.


SOURCE:

1. PBS, “Race: The Power of an Illusion” (2018–2019 relaunch of 2003 series).

2. Robin DiAngelo, “White Fragility” (International Journal of Critical Pedagogy, 2011).


Related Resources:  System of White Supremacy and White Privilege

Location: FUNDAMENTALS / Core Concepts