Evaluation in THEORY
This section explores theory and concepts that help deepen understanding of assessing the social impact of the arts.
The plethora of material that exists on the concepts and theory of evaluation is mindboggling! Here we offer selected resources that are particularly relevant because of their focus on evaulating social and civic impacts. Many reflect current knowledge from fields others than the arts: human services, dialogue/deliberation, community development, urban planning, philanthropy, as well as the evaulation field.
You'll find:
- Resources from the Arts & Civic Engagement Impact Initiative
- Feautured items from IMPACT Arts' Guide to Tools & Resources
- Other essays, articles, papers, and publications
- Featured websites
Title |
Author | Description | Tags | |
|---|---|---|---|---|
| A Developmental Evaluation Primer | Jamie A. A. Gamble | This paper defines developmental evaluation and addresses some myths. Offering an alternative approach to linear logic of traditional methods, developmental evaluation supports the process of innovation within an organization and its activities. The paper outlines developmental evaluation's uses in innovation and developing project ideas and defends the approach's credibility. It is an appropriate resource for organizations that work in an innovative manner and thus seems tailored to arts organi... | analysis approach, assessment, capacity building, context, cultural sensitivity, developmental evaluation, documentation, human capital, program monitoring, social entrepreneurship, social innovation, assessment tool, evaluators, foundations, grantmakers, policymakers, evaluation tool, practitioners, framework, researchers, report, working paper | |
| Action Evaluation | Jay Rothman | This 5-page article on Action Evaluation from the Beyond Intractability website provides a knowledge base and tools regarding social conflict resolution and community reconciliation. This article assists participants, funders, and facilitators in reaching consensus about what they seek to accomplish (people's goals), why (their values and beliefs), and how (suggested account strategies). The article addresses the process in three stages: establishing the baseline, formative monitoring, and summa... | action evaluation, conflict resolution, evaluation, goals, implementation evaluation, measurement, reconciliation, framework | |
| Art and Culture in Communities: A Framework for Measurement |
Maria Rosario Jackson Joaquin Harranz |
Based on several years of field research in communities around the U.S., this six-page brief presents an insightful framework for better capturing and measuring arts, culture, and creative expression at the neighborhood level. Based on Urban Institute's in-depth work in this area over many years, the brief outlines four principles that guide this work on arts and culture in community building: (1) definitions of art, culture, and creativity depend on the cultural values, preferences, and realiti... | arts-related, community building, cultural sensitivity, evaluation, indicators, report | |
 |
Arts and Civic Engagement: Briefing Paper for Working Group of the Arts and Civic Engagement Impact Initiative | Chris Dwyer | Written for Animating Democracy's Arts and Civic Engagement Impact Initiative Working Group, this 14- page paper presents a conceptual framework (or logic model) for arts-based engagement. It offers a discussion of the components of the framework, and a list of questions to guide research explorations. It defines and gives examples of each element: programmatic initiative in terms of the core arts element and related civic/social purpose; context; implementation choices and actions; in... | arts-related, civic engagement, context, evaluation, impacts, implementation evaluation, intermediate effects, logic model, outcomes, program design, social capacity, framework, working paper |
| Arts and Culture Indicators Project |
Maria Rosario Jackson Florence Kabwasa-Green |
The site offers an overview of Urban Institute's Arts and Culture Indicators Project, defines cultural vitality, briefly addresses cultural vitality indicators, offers case study examples, and gives abstracts and links to high-quality relevant research studies. The site is a beautiful resource for arts and civic engagement practitioners and policymakers. Graphically strong with top notch research, it is a model for the field. Civic leaders and heads of large organizations could review this site ... | arts-related, cultural vitality, indicators, quantatative measures, evaluators, foundations, policymakers, practitioners, researchers, website | |
| Arts and Social Exclusion: A Review Prepared for the Arts Council of England | Helen Jermyn | This 43-page report is a literature review commissioned by the Arts Council of England to support its two-year social inclusion research program. It examines the impact of arts in addressing social exclusion with the purpose of informing the design of the research and placing it in a policy context. The first section attempts to define social exclusion, offers ideas and obstacles to measuring it, and relates it to the arts. It describes the government’s Social Exclusion Unit (SEU) that was cre... | arts-related, evaluation, impacts, indicators, measurement, policy, social inclusion, report | |
| Bowling Alone: The Collapse and Revival of American Community | Robert D. Putnam | This 544-page book by sociology professor Robert D. Putnam makes the case that the social capital of American citizens is declining because of increasingly diminished connections with their communities. He supports this claim with extensive interviews and research, much of which is arranged in charts and graphics. Putnam’s argument contends that disparate factors including television, two-career families, and suburban sprawl have combined to make citizens more isolated and less empathetic, fee... | active citizenship, case-making, civic engagement, community building, cultural vitality, data collection, evaluation, social capital, graphic | |
| California Endowment | The California Endowment believes that evaluation represents an important opportunity to assess achievements, generate knowledge and cultivate learning. It works to support grantees, evaluators and others interested in learning more about culturally competent health evaluation. Its website has a wealth of resources, reports, and articles useful to both funders and practitioners. | advocacy, arts-related, evaluation, process evaluation, resources, website | ||
|
|
Civic Engagement and the Arts: Issues of Conceptualization and Measurement |
Mark J. Stern Susan Seifert |
Based on a literature review drawing from the social sciences, humanities, and public policy, Stern and Seifert suggests documentation and evaluation strategies that artists, cultural and community organizations, philanthropists, and public agencies could take to improve the quality of knowledge about the social impact of arts-based civic engagement work. Their paper offers practical considerations for evaluation in areas of methodological issues and data collection strategies. Stern and... | arts-related, civic engagement, community mapping, context, data collection, developmental evaluation, evaluation, experimental methods, impacts, learning organization, methodological strategies, outcomes, policy, program design, qualitative methods, social capital, social inclusion, surveys, theories of action, indicators, report |
| Civic Participation and the Promise of Democracy | Craig McGarvey | |||
|
|
Critical Perspectives: Writings on Art and Civic Dialogue |
Caron Atlas Pam Korza |
Critical Perspectives is a collection of reflective, critical, and creative essays about the role of arts and humanities in civic dialogue. Twelve essays focus on three very different projects that employed the unique capacities of theater, visual art, and historic preservation to reach people and stimulate them to talk together in new ways about issues that matter in their communities. This book deepens understanding of the practice and impact of arts-based civic dialogue work through the enga... | arts-related, criticism, critics, participatory evaluation, evaluators, book, practitioners, researchers |
| Cultural Vitality in Communities: Interpretation and Indicators |
Maria Rosario Jackson Florence Kabwasa-Green Joaquin Harranz |
This 104 page report (with eight chapters and seven appendices), based on extensive community research, discusses cultural vitality and gives extensive information on indicators. This piece -- primarily for evaluators, researchers, and policymakers -- is written in clear, user-friendly language. The fundamental goal of the Urban Institute's Arts and Culture Indicators Project (ACIP) is to help policymakers make better decisions for neighborhoods and cities. To this end, ACIP provides researcher... | analysis approach, arts-related, assessment, community indicators, community mapping, cultural vitality, data collection, indicators, measurement, methodological strategies, quantatative measures, resources, social impact, indicators | |
| Deliberative Democratic Evaluation Checklist |
Ernest R. House Kenneth R. Howe |
This checklist is a guide to incorporating democratic processes in an evaluation to assure valid conclusions where there are conflicting views. A concise checklist aims to ensure that the evaluation process, and thus conclusions, are inclusive and unbiased. The checklist is built around three principles: 1) Inclusion, 2) Dialogue, and 3) Deliberation. The Inclusion principle implies that all relevant interests, values, and views of major stakeholders are taken into account when designing and con... | accountability, methodological strategies, participation, checklist, evaluation tool, worksheet | |
| Drivers and Value of Engagement | The initial scoping phase, so important for providing a sound and lasting framework for the data and evidence reviews, has now ended and the hard work has begun. Technical work on the modelling is under way, including reviewing over 50 different datasets in the UK that could be useful for the model. The systematic review of the research – a wide-ranging hunt for any study in our sector that addresses engagement – has unearthed over 10,000 studies. This allows us to begin the process of mappi... | |||
| Evaluation Tools for Racial Equity | This website is a good resource for individuals and community groups working for change in their communities, specifically with issues relating to race and racial equity. The site is written for users who are new to evaluation -or perhaps those who give funds related to racial equity, but who are not yet clear on how to evaluate it. The site discusses how to apply a “racialized” perspective to evaluation, meaning using the ideas of racism, oppression, privilege and access to power as a lens ... | action evaluation, assessment, community building, conflict resolution, core values, cultural sensitivity, cultural vitality, data collection, diversity, evaluation, goals, impacts, indicators, outcomes, process evaluation, program evaluation, resources, training, bibliography, database, evaluation tool, framework, indicators, online tools, sample evaluation tools, website | ||
| Getting Inside the Story: Ethnographic Approaches to Evaluation | Ethnography is a powerful way to step inside the culture of an organization or community, hear ongoing feedback from multiple points of view, and understand a program's real impact. In this guide, learn about ethnography's benefits and pitfalls, and see how grantmakers use the method to document, evaluate and improve approaches to youth engagement, HIV education and neighborhood policing. | enthographic evaluation, foundations, grantmakers, practitioners, guide | ||
| Importance of Culture in Evaluation: A Practical Guide for Evaluators | Kien Lee | Written for evaluators, this 17-page report is a very nice guide to communicating about culture in evaluation practices and may be used as a reference in data collection. The report emphasizes that cross-cultural competency is a necessary skill for evaluators to have. The author encourages evaluators to think of culture as a factor to be considered as much as sampling and measurement. In addition to the introduction and conclusion, the paper has three main sections: culture, social identity and ... | cultural sensitivity, data collection, evaluation, report | |
| Indicators for Programming in Human Rights and Democratic Development: A Preliminary Study |
Ilan Kapoor Consultant, Loka International |
This 33-page document calls itself a study but can serve as a guide to indicator development. Though done in 1996, it an extremely thorough, thoughtful overview of indicator consideration and development. It defines indicators, gives examples of types, and talks of the challenges of measuring difficult areas such as political development. It discusses issues related to indicators in governmental agencies versus NGOs. Most importantly, it makes distinctions between qualitative and quantitative in... | indicators, outputs, participatory evaluation, qualitative evaluation, quantatative measures, evaluators, grantmakers, practitioners, guide, indicators, report | |
| Introduction to Process Evaluation in Tobacco Use Prevention and Control | Centers for Disease Control and Prevention | From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers. The piece is written in fiv... | accountability, data collection, indicators, inputs, methodological strategies, outcomes, outputs, process evaluation, program evaluation, program monitoring, tobacco, evaluators, report | |
| Lessons Learned from the Implementation of the Neighborhood and Family Initiative: A Summary of Findings | Robert J. Chaskin | This 35-page report provides a summary of findings and lessons learned by the Neighborhood and Family Initiative (NFI), a comprehensive community initiative launched by the Ford Foundation that began in 1990. NFI was a 10-year effort that sought to strengthen a single neighborhood in each of four cities and to improve the quality of life for the families who live in them. The report is divided into two sections; the first provides a brief overview of NFI while the second section distills the les... | active citizenship, community building, implementation evaluation, outcomes, program design, program evaluation, social capital, report | |
| Making Measures Work for You: Outcomes and Evaluation | Craig McGarvey | This piece is an excellent primer on how to think about outcomes and the hurdles that may arise in measuring them. It addresses and deals with challenges such as measuring intangibles. Published by GrantCraft (a division of the Ford Foundation), the eleven-page guide is written for grantmakers to describe outcomes-based evaluation. It defines key terms and makes a case for why outcome measurement is important. The guide has four sections: 1) A concise definition of outcomes, and a contrast of ou... | indicators, outcomes, grantmakers, guide, indicators | |
| Mapping Change: Using a Theory of Change to Guide Planning and Evaluation | Craig McGarvey | "What are we doing, and why do we think it’s going to make a difference? Are we being effective?" Grantmakers ask evaluation questions like these of their grantees and themselves. This brief guide explains why grantmakers use theories of change to guide their questioning, unearth assumptions that underlie their work, establish common language, and develop strong action plans. Contributors to the guide also describe how a theory of change sets the stage for evaluation by clarifying goals, strat... | assessment, case-making, evaluation, goals, impacts, logic model, outcomes, planning, realistic evaluation, theory of change, framework, guide, overview | |
| Modeling Engagement, Impacts and Value | This paper is one of a number of working papers produced for the project “Understanding the drivers of, and value and benefits afforded by, engagement in culture and sport”. The objectives of the project are to define and model the following broad relationships:1. The impact of policy options and other factors on the level of engagement in sport and culture. 2. The outcome of engagement in sport and culture. 3. The value of these outcomes. This paper focuses on the last of these points. Spe... | case-making, community indicators, core values, impacts, indicators, framework, report, working paper | ||
| Participatory Action Research: Involving “All the Players” in Evaluation and Change | Craig McGarvey | In this guide from the respected GrantCraft series, learn about participatory action research, a unique evaluation method and how grantmakers used it to evaluate programs in agriculture, early childhood development, and immigration. | participatory evaluation, qualitative methods, interactive guide | |
| Point K Learning Center | Point K Learning Center ("Point K"), a website and online workstation, was created by Innovation Network (Innonet), a leader in the participatory evaluation field and one of the first to make online tools available that could be used by smaller as well as larger organizations. Point K features practical tools and resources for nonprofit planning, evaluation and action. It is a major part of Innonet’s mission of sharing know-how to create lasting social change. Point K aims to help users asse... | evaluation, logic model, networking, planning, program design, social change, assessment tool, database, online tools | ||
| Qualitative Research and Evaluation Methods | Michael Quinn Patton | A bible on the topic of qualitative inquiry, this book introduces the principles and methods of qualitative research and evaluation. Chapters cover conceptual issues in qualitative inquiry (including strategic themes and variety), the design of qualitative studies, fieldwork strategies and observation methods, interviewing, analysis, interpretation, and reporting. Important controversies are outlined, and key points are illustrated with examples. This text is a resource and training tool for app... | analysis approach, data collection, measurement, program design, qualitative design, qualitative evaluation, qualitative methods, practitioners, textbook | |
|
|
Reflections on Arts and Civic Engagement Briefing Paper |
Barbara Schaffer Bacon Pam Korza |
This paper reflects Animating Democracy co-directors’ thinking following Working Group discussion of Chris Dwyer’s “Arts and Civic Engagement: Briefing Paper for the Working Group of the Arts & Civic Engagement Impact Initiative.” | arts-related, case-making, civic engagement, context, impacts, intermediate effects, measurement, qualitative evaluation, social impact, grantmakers, policymakers, practitioners, working paper |
| Saguaro Seminar: Civic Engagement in America | Since 2000, the Saguaro Seminar's mission has been both to improve social capital measurement and the availability of social capital data and to undertake analysis of building social capital in increasingly diverse communities. The web site includes extensive information on measurement of social capital including Putnam’s 2000 Social Capital Community Benchmark Survey, as well as evaluation tool kit, guide, and links to useful organizations and resources. The Saguaro Seminar is an ongoing init... | data collection, measurement, methodological strategies, resources, social capital, social impact, surveys, interactive guide, online tools, sample evaluation tools, toolkit, website | ||
|
|
Shifting Expectations: An Urban Planner’s Reflections on Evaluation of Community-Based Arts | Maria Rosario Jackson | In this paper, Jackson argues for a shift toward more realistic expectations of social impact and evaluation of arts-based civic engagement both on the part of practitioners and funders. The paper also provides recommendations for practical ways of moving towards and operationalizing that paradigmatic shift. Based on 13 years of national research on integrating arts and culture into concepts of healthy communities, Jackson observes how sound and worthy community arts programs with social and... | article, arts-related, case-making, civic engagement, context, evaluation, impacts, foundations, grantmakers, policymakers, practitioners, researchers |
| Social Impact of the Arts Project | The Social Impact of the Arts Project (SIAP) is a research center at the University of Pennsylvania School of Social Policy & Practice. SIAP conducts research on the role of arts and culture in American cities with a particular interest in strategies for arts-based revitalization. Since 1994, SIAP has focused on developing empirical methods to study the links between cultural engagement and community well-being. This excellent web site includes a number of papers generated by the center refl... | arts-related, case-making, community building, community mapping, cultural vitality, data collection, impacts, policy, quantatative measures, social impact, website | ||
| Speaking for Themselves: Advocates’ Perspectives on Evaluation | Innovation Network, Inc. (Innonet) | This 20-page report was produced by Innovation Network – a nonprofit organization that shares planning and evaluation tools and know-how by providing consulting, training, and online tools to help organizations create lasting change in their communities – with support by Annie E. Casey Foundation and The Atlantic Philanthropies. The report examines the current state of advocacy strategy and evaluation practice. It includes a section on the importance of interim measures of success and a li... | advocacy, capacity building, evaluation, methodological strategies, outcomes, indicators, report, sample evaluation tools | |
| Theory of Change | This website is devoted to the application, methodology, use and practice of Theory of Change methodology. For 2009, it has been revamped to include more information on how to best use TOC, and lessons from the many areas in which TOC has been applied, and the role of TOC in organizational functioning and capacity. The Theory of Change website is a project of ActKnowledge, an action research organization dedicated to working with community organizations, not-for-profits,... | program design, theory of change, website | ||
| Three Approaches to Evaluation: Evaluation Evolution? | Otto Hospes | This seven-page article from The Broker magazine aims to help policymakers, practitioners, and scientists make better use of research and evaluation (specifically in the fields of poverty reduction and international development) and uses relatively advanced language. It consists of three sections: evidence-oriented evaluation, realistic evaluation, and complexity evaluation. The introduction explains the important point of how evidence-oriented evaluation has been the dominant approach to date... | article, complexity theory, realistic evaluation | |
| z3 place holder | no current resource |
