Featured Resources
These tools and frameworks found in The Annotated Guide to Tools & Resources offer diverse information about indicators. This list has been assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts.
Title |
Author | Indicator Notes | Tags | |
|---|---|---|---|---|
| A Developmental Evaluation Primer | Jamie A. A. Gamble | The author defines and distinguishes leading and lagging indicators in a one-page section on page 35. An example specific to the developmental approach to evaluation follows. | analysis approach, assessment, capacity building, context, cultural sensitivity, developmental evaluation, documentation, human capital, program monitoring, social entrepreneurship, social innovation, assessment tool, evaluators, foundations, grantmakers, policymakers, evaluation tool, practitioners, framework, researchers, report, working paper | |
| A Handbook of Data Collection Tools: Companion to A Guide to Measuring Advocacy and Policy |
Jane Reisman Anne Gienapp Sarah Stachowiak |
Indicators relate to advocacy and the desired outcome of improved policies. Specific examples of indicators attempt to measure policy change and progress toward policy change. On page 8, the authors describe how the Alliance for Justice Tool Advocacy Capacity Assessment Tool is designed around indicators. On page 32, there is a list of six resources related to indicators in advocacy work, and on page 35 there is a description and brief definition of indicators in the context of measuring short-term incremental objectives. | advocacy, case-making, civic engagement, data collection, evaluation, evidence-based evaluation, impacts, indicators, outcomes, policy, qualitative data, sample, social change, social impact, surveys, grantmakers, evaluation tool, practitioners, framework, guide, interactive questions, manual, overview, sample evaluation tools | |
| Advocacy Evaluation Project | Innovation Network, Inc. (Innonet) | This database includes at least 11 items, estimated to be over 300 pages, of indicator resources. This extensive website is likely to generate very useful information about indicators. | accountability, advocacy, capacity building, case-making, community indicators, data collection, evaluation, impacts, indicators, measurement, methodological strategies, outcomes, resources, social impact, assessment tool, bibliography, database, evaluation tool, framework, guide, indicators, interactive guide, online tools, sample evaluation tools, website | |
| Art and Culture in Communities: A Framework for Measurement |
Maria Rosario Jackson Joaquin Harranz |
ACIP does a great deal of work on indicators, but this item focuses on other topics. | arts-related, community building, cultural sensitivity, evaluation, indicators, report | |
| Arts and Culture Indicators Project |
Maria Rosario Jackson Florence Kabwasa-Green |
Many of the papers one can access through this site speak to indicators. | arts-related, cultural vitality, indicators, quantatative measures, evaluators, foundations, policymakers, practitioners, researchers, website | |
| Arts and Social Exclusion: A Review Prepared for the Arts Council of England | Helen Jermyn | The general discussion of indicators references organizations that have developed them around arts and social inclusion with few specific examples. On pages 7-8, the section “the value of indicators” speaks to the importance of their use in focusing program objectives and creating a framework for data collection. Appendix 1 gives two indicators lists, grouped by outcome area, along with sample tools. | arts-related, evaluation, impacts, indicators, measurement, policy, social inclusion, report | |
| Build Your Advocacy Grantmaking: Advocacy Evaluation Tool & Advocacy Capacity Assessment Tool | Alliance for Justice | Indicators relate to measuring an organization's capacity and preparedness for advocacy work. The Advocacy Capacity Assessment Tool is organized around nine broad indicators of capacity: decision making structures; advocacy agenda; organizational commitment to and resources for advocacy; advocacy base; advocacy partners; advocacy targets; media skills and infrastructure; advocacy strategies; and knowledge, skills, and systems to effectively implement strategies. A series of specific measures (statements) follows each indicator category. Users can assess their organizations'sadvocacy readiness and space is provided for their comments. All indicators are directly connected to advocacy work. | advocacy, assessment, evaluation, assessment tool, grantmakers, evaluation tool, indicators, interactive guide | |
 |
Civic Engagement and the Arts: Issues of Conceptualization and Measurement |
Mark J. Stern Susan Seifert |
Indicators relate directly to arts and civic engagement work, specifically participatory evaluation of community cultural development efforts. They come from two sections: one on organizational/program-scale strategies and the other on regional-scale strategies. Six indicators recommended for organizational/program-scale strategies are meant to guide evaluation of community cultural development projects and relate to project level impact. Four indicators recommended for regional-scale strategies pertain specifically to the example of a community-based program with an outcome goal of enhancing youth civic engagement. The authors further discuss and give examples of indicators on pages 47-49, where there is a section with definitions of indicators and benchmarks. They encourage readers to select indicators that reflect the activities and outcomes; are easily measurable; are adaptable to changes in program activities; are logically connected to what is measured; and are understandable to all stakeholders. The following section focuses on community and cultural data mapping as potential measures. | arts-related, civic engagement, community mapping, context, data collection, developmental evaluation, evaluation, experimental methods, impacts, learning organization, methodological strategies, outcomes, policy, program design, qualitative methods, social capital, social inclusion, surveys, theories of action, indicators, report |
| Creativity Matters: The Arts and Aging Toolkit | Johanna Misey Boyer | Indicators relate to the experiences had by older adults during specific arts projects. Because this toolkit is designed to help users develop arts programs for aging adults, most of the indicators are intended to help evaluate the quality and depth of participants' experiences as they relate to emotion and self-esteem. Two indicators also relate to the stability of an organization. A sample instrument in Appendix 7: Memories in the Making lists four indicator categories (engagement, expression of pleasure, self-esteem, and expression of emotions and feelings) and has three or four questions corresponding to each. | arts-related, evaluation, indicators, older adults, outcomes, program design, guide, toolkit | |
| Cultural Planning Toolkit : Integrating Community and Cultural Planning | Creative City Network of Canada | Indicators focus on integrating community and cultural planning, two independent yet related endeavors. Indicators measure the successfulness of the intergration efforts -- how well the organizers are doing at integrating cultural development into local government thinking, planning, and decision-making. They focus on outcome goals of raising awareness/extending engagement and fostering vision and relationships. On pages 7-8 of the web resource, there is a good list of specific indicators of progress arranged around elements of awareness, vision and relationships, commitment and development, and actual integration. | arts-related, assessment, community building, cultural planning, data collection, implementation evaluation, leadership, program design, checklist, guide, indicators, toolkit | |
| Cultural Vitality in Communities: Interpretation and Indicators |
Maria Rosario Jackson Florence Kabwasa-Green Joaquin Harranz |
Indicators are each linked specifically to the Arts and Culture Indicators Project (ACIP) domain of cultural vitality (presence, participation, or support); a data source; and one of four data "tiers" (which are described in depth in the paper). The indicators come from one of three reports on arts and civic engagement projects and are concrete examples of measures used in context. The authors discuss the community indicator field and the progress it has made in widening its treatment of arts and culture management and how prospects for continuing in this direction look good. The report develops and recommends an initial set of arts and culture indicators derived from nationally available data, and compares selected metropolitan statistical areas based on the measures developed. | analysis approach, arts-related, assessment, community indicators, community mapping, cultural vitality, data collection, indicators, measurement, methodological strategies, quantatative measures, resources, social impact, indicators | |
| Evaluating Outcomes and Impacts: A Scan of 55 Leadership Development Programs. Appendices D, E, F, G, H: Outcome Indicators | W.K. Kellogg Foundation | Indicators relate to how leadership development programs evaluate impact. This initiative aimed to learn how programs were assessing their impact, the outcomes they hoped to achieve, indicators of success they had identified, approaches they used for evaluation and learning, and methods and sources of information they relied on. Indicators are presented in five separate appendices, based on scale, from individuals to organizations to communities to the field to the whole system. Indicators are in the form of specific questions that are tied to general outcome categories, such as collaboration, communication, courage/confidence, cultural competence, knowledge development, leadership in action/demonstrating leadership, leadership development, personal development, perspective development, professional development, skill development, visibility, networks, partnerships, community change; community decision-making; community leadership; engagement/participation, public awareness; resource development; and social capital. | community building, community indicators, indicators, indicators, interactive questions | |
| Evaluation Tools for Racial Equity | Indicators are addressed in Section 2a and in the Tip Sheet “What Is An Outcome and What Is An Outcome Indicator?” | action evaluation, assessment, community building, conflict resolution, core values, cultural sensitivity, cultural vitality, data collection, diversity, evaluation, goals, impacts, indicators, outcomes, process evaluation, program evaluation, resources, training, bibliography, database, evaluation tool, framework, indicators, online tools, sample evaluation tools, website | ||
| Indicators for Programming in Human Rights and Democratic Development: A Preliminary Study |
Ilan Kapoor Consultant, Loka International |
Indicators relate to human rights and democratic development. This thorough study, useful for understanding indicator development, discusses and differentiates quantitative and qualitiative measurements as part of a participatory evaluation process. Indicators are grouped in six categories: support to electoral process, legislative assistance, enhancing democratic culture and public dialogue, legal/human rights support, women's rights and economic and social and cultural rights, and support to civil society organizations. The authors designate four different types of indicators: input, output, outcome, and impact. They also offer a cautionary note that, because human rights and democratic development activities are closely related to institutional strengthening/capacity development, the indicators presented in the report may need to be complemented with indicators for capacity development. The study also compares indicators in governmental agencies versus NGOs and highlights the role of participatory indicators (which are developed by stakeholders and are unique to each project). It states that participatory indicators, while sometimes criticized as being "subjective" and "qualitative," are often the most useful ones because they go beyond mere numbers to incorporate context and reflect local knowledge and creativity. It then offers one approach to developing indicators in areas of human rights and democratic development and related policy. In summary, it emphasizes the uniqueness of indicators to specific projects but gives guidance on how to develop them. The Appendix gives sample indicator lists in the areas of human rights and development. | indicators, outputs, participatory evaluation, qualitative evaluation, quantatative measures, evaluators, grantmakers, practitioners, guide, indicators, report | |
| Introduction to Process Evaluation in Tobacco Use Prevention and Control | Centers for Disease Control and Prevention | Indicators relate to process evaluation as it applies to efforts to control and prevent tobacco use. Chapter 3 discusses indicators at length and gives examples. Presented as part of a comprehensive chart depicting stages in process evaluation, indicators are organized into four main categories: inputs, activities, outputs, and other elements. Each indicator is connected to a specific item to be measured (such as financial, personnel, facillities, accessibility, and persons reached) that is briefly described. Chapter 3 especially discusses indicators at length and gives examples that pertain to tobacco use prevention. For example, in the outputs section, an item is "characteristics of persons reached;" the description is "demographic, family, personal, health, and other attributes of the people reached by [tobacco] prevention/control activities;" and the indicators are age, gender, and comparison of characteristics of persons reached with characteristics of the intended target population. This section also mentions other elements relating to an initiative and/or its context, including the developmental stage, organizational structure and components, program theory and fidelity, collaborations or partnerships, and social/political/economic environment in area of focus. Many measurable examples are provided. | accountability, data collection, indicators, inputs, methodological strategies, outcomes, outputs, process evaluation, program evaluation, program monitoring, tobacco, evaluators, report | |
| Making Measures Work for You: Outcomes and Evaluation | Craig McGarvey | Pages 8-9 of this guide have a nice discussion about selecting indicators (also called measurement proxies). It cautions against indicators that are too large in scale relative to the initiative and against using the most readily available measures, which not be the most appropriate. The piece discusses the difference between outcomes and indicators and why they ought not be confused, since specific indicators may show only part of the big picture. The guide gives three nice examples of how over-emphasis on indicators may mean losing sight of the more important outcome. | indicators, outcomes, grantmakers, guide, indicators | |
| Measuring Innovation: Evaluation in the Field of Social Entrepreneurship | Mark R. Kramer | In section 3.3, Measuring Progress Against Self-Determined Goals, social entrepreneurs offer examples of indicators they have found effective for both field work and reporting to funders. Section 3.5, Estimating Economic Benefits and Financial Leverage, discusses financial performance indicators as they relate to social entrepreneurship as well as the importance of having such concrete measures. | article, evaluation, social entrepreneurship, grantmakers, indicators, sample evaluation tools | |
| Measuring Social Impact |
Deb Levy Kathy Brennan |
Written by social entrepreneurs, this item strives to give users guidance in identifying and presenting evidence of social impact by systematic data collection and evaluation. Indicators address questions of what, who, how much, how many, and when, and they relate specifically to the following outcomes: program participants develop empathy and understanding for alternative perspectives; participants educate family and friends based on what they have learned; and the [campus] community has improved empathy for alternative perspectives. Three examples are given and tied to the desired outcomes. | indicators, outcomes, qualitative evaluation, social entrepreneurship, social impact, guide, indicators | |
| Model of State Arts Agency Values and Actions to Broaden, Deepen, and Diversify Participation | Chris Dwyer | The indicators relate specifically to state arts agencies and their efforts to "broaden, deepen, and diversify creators, stewards, and spectators/participants." The table sets up five steps for planning evaluations: identifying a range of audiences for the evaluation results of the agency; identifying motivating assumptions/interests for each of those target audiences, determining desired outputs and outcomes, identifying adjustments in implementation to better meet desired outputs/outcomes, and selecting preferred forms of evidence, including indicators of progress and preferred methodologies. Therefore, indicators are connected to specific desired outcomes or outputs for state arts agencies in their efforts to increase and deepen constituent participation. It presents sample indicators and examples as they pertain to specific goals and outcomes. Indicators are also briefly addressed in a section of text following the table which describes their relationship to beliefs, actions, and values. | arts-related, diversity, evaluation, indicators, participation, framework | |
| Modeling Engagement, Impacts and Value | case-making, community indicators, core values, impacts, indicators, framework, report, working paper | |||
| Possible Outcomes/Indicators/Measures from the AEI: Police Calendar Project | Chris Dwyer | Indicators relate to a specific arts and civic engagement project in which police and citizens of the City of Portland, Maine, created a calendar that included poetry, writings, and photography. The desired outcomes/values included enhanced self-esteem in police role, built relationships to art, development of new skills/improve skills, improved department morale, official recognition of value of police department and its work, and improved community relationships to communities of color/immigrant communities. Each indicator relates directly to one of these desired outcomes. Data collection strategies for each indicator are also suggested. Other aspects of the proposed evaluation design included formative evaluation of the implementation of the project as well as the project’s contribution to larger goals of the overall initiative. | indicators, worksheet | |
| Preliminary Menu of Possible Outcomes/Indicators/Measures from the Starksboro Art and Soul Project, Orton Family Foundation | Chris Dwyer | This three-page chart reflects a worksheet format for working through outcomes, indicators, and other measures related to the Art & Soul Community Project which is taking place in Starksboro, VT (2009). Thirty-two specifically articulated indicators are listed and tied to eight specific outcomes. Data collection strategies for each indicator are also suggested. The example, in its specificity, is instructive in how to frame indicators and link them clearly to outcomes and data collection. A blank worksheet is also included. | arts-related, capacity building, community building, data collection, evaluation, indicators, outcomes, process evaluation, qualitative evaluation, sample evaluation tools | |
| Research on Public Art: Assessing Impact and Quality |
Catherine Ward Thompson Andrew Patrizio Alicia Montarzino |
This piece mentions indicators throughout and gives examples without delving into the topic of selection. Many of the indicators mentioned in the Review of Literature section are governmental measures specific to the UK. | arts-related, indicators, program design, public art, qualitative design, assessment tool, policymakers, evaluation tool, toolkit | |
|
|
Shifting Expectations: An Urban Planner’s Reflections on Evaluation of Community-Based Arts | Maria Rosario Jackson | Two types of indicators are mentioned in this piece: quality of life and informal/anecdotal. On page 2, quality of life indicators are described as groupings of measures that describe community conditions over time. Examples include recurrent measures of crime, education, and housing. On pages 8-9, Jackson discusses informal indicators of impact, such as letters from program participants and inquiries about future plans for programs, which can become the basis for more systematized data collection. | article, arts-related, case-making, civic engagement, context, evaluation, impacts, foundations, grantmakers, policymakers, practitioners, researchers |
| Singing Our Praises: Case Studies in the Art of Evaluation | Suzanne Callahan | Indicators are tailored to each of two case studies: 1) Chicago Dancemakers Forum, which provided commissioning fees and professional development services for choreographers and 2) Improbable Theater, around the premier of a new work, including its impact on the artists who created it and audiences who viewed it. The indicators for the Chicago Dancemakers Forum relate directly to artists (degree of support felt; evidence of risk-taking; enhancement of creative process), as well as to the response to the work on the part of viewers (both audiences and professional peers). Professional development indicators relate to ease of logistics, participation levels at events (including active conversations generated among artists), and high rate of return to events. Long term indicators are increased booking, recognition and other opportunities for artists. Improbable Theater’s indicators related to the audiences (their emotional reaction; awareness of the brand of the two venues who presented it; awareness that the created was British; interest in following these artists; and increase in “cultural curiosity,” or cultural experiences that are new and different). A main indicator for artists was their view of the presenters as a “home away from home,” or as providers of support and freedom in the creative process. | arts-related, context, data collection, evaluation, logic model, outcomes, participatory evaluation, program design, qualitative data, qualitative evaluation, sampling, surveys, framework, guide, interactive guide, interactive questions, manual, sample evaluation tools, worksheet | |
| Speaking for Themselves: Advocates’ Perspectives on Evaluation | Innovation Network, Inc. (Innonet) | Indicators relate to advocacy strategies and their evaluation practice. The measures derived from this item are specifically interim measures of success, as the authors feel that over-emphasis on long-term goals does not serve the advocacy campaign as well as this incremental approach does. A list of 27 specific indicators on page 13 is tied directly to eight outcome categories: building the base, decision maker support, strengthened infrastructure or position within issue movement, communications, opening window of opportunity, issue campaign sustainability or strengthening, interim progress tied to legislative victory, and general mention of evaluation. | advocacy, capacity building, evaluation, methodological strategies, outcomes, indicators, report, sample evaluation tools | |
| The Community Tool Box | Indicators relate to community health and development projects. The indicators provided are about a specific program's ability to accomplish its objectives and the impact it has on its constituents and community. Presented in Chapter 36, indicators are intended to strengthen the evaluation by guiding users to gather credible and measurable evidence. The piece has a few examples of indicators as well as a discussion about intermediate effects and modifying indicators throughout the process. | arts-related, capacity building, community building, data collection, evaluation, logic model, networking, program design, theory of change, checklist, database, evaluation tool, framework, guide, indicators, online tools, website | ||
| W.K. Kellogg Foundation Evaluation Handbook | W.K. Kellogg Foundation | This item is about project level evaluation for a variety of disciplines. The indicators are specific examples that relate to the desired outcome of increased self-esteem, but are not connected to any particular field, program, or art form. On pages 33-34, two questions are posed that relate to indicators: How will you measure outcomes? And, what data will you collect and how will you collect it? In answering these questions, Kellogg differentiates outcomes from indicators (with examples) and addresses data collection strategies as they relate to the selected indicators. It also cautions against planning around the most readily available indicators, as Kellogg feels the focus of planning ought to be on overall goals and outcomes. | analysis approach, context, data collection, indicators, outcomes, process evaluation, program design, guide |
