In answer to the challenges that face corporate philanthropy in identifying a shared definition of impact measurement, the author sets out to assess current measurement practices, clarify what is needed in terms of impact evidence, and identify next steps. The article is organized into three conversations between key stakeholders engaged in corporate philanthropy.
Resources
The Annotated Guide to Tools & Resources provides tools, frameworks, and resources to help you develop and implement your evaluation. It’s a repository of useful, practical materials that can help you create an evaluation plan; design your evaluation approach; develop or adapt tools and instruments; and otherwise move your evaluation forward.
This Guide was originally assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts. We continue to add resources. Your suggestions are welcome!
Do you have a useful tool or resource to add? Contact animatingdemocracy@artsusa.org.
Resource Format: practical tool
What is the difference between outputs and outcomes? Between outcomes and impact? “Measuring What Matters: The Challenge of Quantifying Social Change” steers clear of jargon and offers an accessible evaluation framework for practitioners.
Resource Format: practical tool
This tool serves as a model to align values, actions, and measures of progress for State Art Agencies. In table form, it lays out a generic base for locating concepts of participation within a framework of concepts of public value and motivating values of different groups. The table can serve as a basis for developing the types of outcomes and measures related to State Art Agencies' actions to broaden, deepen and diversify creators, stewards and spectators/participants.
Resource Format: book / article
This paper is one of a number of working papers produced for the project “Understanding the drivers of, and value and benefits afforded by, engagement in culture and sport”. The objectives of the project are to define and model the following broad relationships: 1. The impact of policy options and other factors on the level of engagement in sport and culture. 2. The outcome of engagement in sport and culture. 3. The value of these outcomes. This paper focuses on the last of these points.
Resource Format: book / article, case study
Artist Rha Goddess’s Hip Hop Mental Health Project (HHMHP) seeks to contribute to shifting the cultural paradigm of shame and alienation surrounding mental illness, and satisfy a need for a SAFE place to confront the issue and obtain vital information.
Resource Format: book / article, case study
When Flint Youth Theatre began planning for a new play addressing the local and national problem of school violence, it had no idea that, in the process of developing the project, its own community would experience a devastating elementary school shooting. A year after the tragedy, the play ...My Soul to Take, written by artistic director and playwright William Ward, became a focal point for fresh attention on this persistent and painful issue.
Publication Date: May 1, 2012
Resource Format: book / article
Outcome Harvesting is a method that enables evaluators, grant makers, and managers to identify, formulate, verify, and make sense of outcomes while acknowledging the complexities of social change projects. Unlike some evaluation methods, Outcome Harvesting does not measure progress towards predetermined outcomes, but rather collects evidence of what has been achieved, and works backward to determine whether and how the project or intervention contributed to the change.
"In so many evaluations,” said a program officer, “no one thinks to ask the users.” Participatory action research offers grantmakers a way to do so. It engages all parties in all aspects of an evaluation, from defining the problem to gathering and analyzing data to preparing recommendations. In this guide, learn about a unique evaluation method and how grantmakers used it to evaluate programs in agriculture, early childhood development, and immigration.
Publication Date: May 23, 2017
Resource Format: practical tool
The Performing Artist Companion to Animating Democracy’s framework, Aesthetic Perspectives: Attributes of Excellence in Arts for Change, offers ideas and insights to help performing artists and performance companies apply the framework to address their needs and interests. Aesthetic Perspectives aims to enhance understanding and evaluation of creative work at the intersection of arts and community/civic engagement, community development, and justice.
Publication Date: December 31, 2009
Resource Format: book / article
This piece suggests that the accountability movement is “setting a floor for minimum standards” (p. 84) and has consequences for effective social change work. Foundations, in particular, measure impact in terms of attentiveness to accountability standards, but this is a false measure of success. Instead, the organization’s focus should be on its transformative value to society.
Resource Format: practical tool, website
Point K Learning Center ("Point K"), a website and online workstation, was created by Innovation Network (Innonet), a leader in the participatory evaluation field and one of the first to make online tools available that could be used by smaller as well as larger organizations. Point K features practical tools and resources for nonprofit planning, evaluation and action. It is a major part of Innonet’s mission of sharing know-how to create lasting social change. Point K aims to help users assess strengths, articulate goals, use data and better tell the organization’s story.
Resource Format: case study
Edutainment - the mixture of education and entertainment ranging from video games to soap operas, harnesses the power of entertainment to promote social change.
Resource Format: case study, practical tool
The Preliminary Menu encompasses process outcomes (short-term), intermediate outcomes (during the life of the project), and impact (long-term, post-project results) for the Art & Soul Project in Starksboro, VT. Community members and researcher/evaluator Chris Dwyer used the worksheet to clarify what to measure and how.
Publication Date: January 1, 2014
Resource Format: book / article
Presenting Data Effectively is a step-by-step guide to making research results more useful and interesting. This book provides guiding principles for designing effective data presentations and reports.
These complex concepts are outlined in straightforward manner and can easily be applied by students, researchers, evaluators, and non-profit workers. The book guides the reader through design choices related to four primary areas: graphics, type, color, and arrangement.
Publication Date: May 1, 2014
Resource Format: book / article
In response to the NEA’s call to identify and create data sources in the arts, the Prison Arts Resource Project (PARP) compiles resources on how the arts work in correctional settings and how they impact the lives of inmates, their families, and their communities.
Publication Date: October 16, 2013
A new take on bringing activism to life, Public, Imagining America's new online journal, empasizes dialogue between communities and humanitarians.
Publication Date: May 14, 2012
Resource Format: practical tool
Continuum: Awareness, knowledge, understanding
MWW: Audio in Philadelphia serves as an example of innovation in data collection and evaluation. This resource captures Penny Balkin Bach’s contribution from Animating Democracy’s 2012 Social Impact & Evaluation Blog Salon.
Resource Format: book / article, practical tool
A bible on the topic of qualitative inquiry, this book introduces the principles and methods of qualitative research and evaluation. Chapters cover conceptual issues in qualitative inquiry (including strategic themes and variety), the design of qualitative studies, fieldwork strategies and observation methods, interviewing, analysis, interpretation, and reporting. Important controversies are outlined, and key points are illustrated with examples. This text is a resource and training tool for applied researchers, evaluators, and graduate students.
Resource Format: practical tool
This paper reflects Animating Democracy co-directors’ thinking following Working Group discussion of Chris Dwyer’s “Arts and Civic Engagement: Briefing Paper for the Working Group of the Arts & Civic Engagement Impact Initiative.” The discussion got to the heart of the challenges of measuring the social impact of arts-based engagement and somewhat shifted the Initiative’s theory of change.
Resource Format: practical tool
ixia, the public art think tank, is funded by the Arts Council England and aims to provide guidance on the role of art in the public realm. Through its activities, ixia identifies and challenges restrictive practices which result in limited and missed opportunities for artists working in the public realm. ixia works with artists, policymakers and implementers within the public and private sectors and carries out research, supports events, delivers training, and commissions publications.
Publication Date: June 12, 2014
In a response to the article Strategic Philanthropy for a Complex World, Darren Walker calls on philanthropists to change the current paradigm of evaluative practice employed in social change work to develop sharper tools and more inclusive frameworks that account for the considerable complexity of real-world problem solving.
Publication Date: April 30, 2012
Resource Format: website
Principal investigator of the Social Impact of the Arts Project at the University of Pennsylvania, Mark J. Stern suggests that cultural research must move beyond purely economic yardsticks in judging well-being. This resource captures Stern’s contribution from Animating Democracy’s 2012 Social Impact & Evaluation Blog Salon.
Publication Date: June 25, 2014
Resource Format: practical tool, website
Ripple Effect Mapping (REM) engages program and community stakeholders to retrospectively and visually map the "performance story" of their work. The REM method, presented here by University of Minnesota, is a participatory group method for evaluating the impact of complex programs or interventions.
Since 2000, the Saguaro Seminar's mission has been both to improve social capital measurement and the availability of social capital data and to undertake analysis of building social capital in increasingly diverse communities. The web site includes extensive information on measurement of social capital including Putnam’s 2000 Social Capital Community Benchmark Survey, as well as evaluation tool kit, guide, and links to useful organizations and resources. The Saguaro Seminar is an ongoing initiative of Professor Robert D. Putnam at the John F.
Publication Date: June 25, 2014
Resource Format: practical tool
See, Say, Feel, Do from Fenton Communications helps practitioners track the performance of their communications and marketing tactics against the broader goals of their project or organization by using four discrete metrics. The first metric, “do” reflects a project’s intended outcome or participant outcome; the second “see” metric indicates audience exposure; and finally, the “say” and “feel” metrics refer to two distinct stages of audience engagement. This four metrics model seeks to answer the following questions: