Introduction to Process Evaluation in Tobacco Use Prevention and Control

From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers. The piece is written in five chapters: 1) Introduction; 2) Purposes and Benefits of Process Evaluation; 3) Information Elements Central to Process Evaluation; 4) Managing Process Evaluation; and 5) Conclusion. The introduction outlines the planning/program evaluation/program improvement cycle (with a handy graphic) and distinguishes process evaluation from outcome evaluation. Chapter 2 begins by defining process evaluation: the systematic collection of information on a program’s inputs, activities, and outputs, as well as the program’s context and other key characteristics. It then relates process evaluation to tobacco use prevention and defines its scope. It then presents the four primary purposes of process evaluation: Program Monitoring (track, document, and summarize the inputs, activities, and outputs of a program; describe other relevant characteristics of the program and/or its context); Program Improvement (compare the inputs, activities, and outputs of your program to standards or criteria, your expectations/plans, or recommended practice; relate information on program inputs, activities, and outputs to information on program outcomes); Building Effective Program Models (assess how process is linked to outcomes to identify the most effective program models and components); and Program Accountability (demonstrate to funders and other decision makers that you are making the best possible use of program resources). Chapter 2 concludes with a discussion of users and uses of process evaluation information. Chapter 3, on central elements of process evaluation, begins with a section on indicators of inputs, activities, and outputs in which it shows information commonly obtained through process evaluation as it relates to these categories. Information about inputs includes financial, personnel (quantity, professional and personal characteristics), facilities, and locale. Information about activities includes population-oriented prevention activity, individual-oriented activity, accessibility, and recruitment. Information about outputs (as they relate to tobacco use prevention) include persons reached, characteristics of persons reached, products resulting from activities, and documents that influence the issue in a target area. This section also mentions other elements relating to an initiative and/or its context, including the developmental stage, organizational structure and components, program theory and fidelity, collaborations or partnerships, and social/political/economic environment in area of focus. All of these inputs, activities, outputs, and other elements are accompanied by descriptions and examples. Chapter 3 concludes with a section on comparing process information to performance criteria. A comprehensive summary graphic in this section separates activities from outputs and gives examples of specific criteria and indicators for each. This may help the reader to better understand indicators as many concrete examples are provided. Chapter 4, on managing process evaluation, opens with a ten step process, goes on to discuss the program evaluation standards and protecting participants in evaluation research, proceeds to describe how to choose a process evaluation design (methodology), and wraps up with by noting that process evaluation ideally goes beyond a single study. After the conclusion (Chapter 5) is a glossary of terms and four appendices. The appendices include information on the CDC’s framework for program evaluation; a detailed list of the Joint Committee on Standards for Evaluation (Program Evaluation Standards); additional information on the purpose, selection, and roles of the evaluation advisory group; and the actual process evaluation questions and logic model from the Center for Tobacco Policy Research. Interspersed throughout the report as a number of exhibits or graphics (some of which were mentioned above) along with several case examples.