How to Develop Measurable Program Outcomes

Outcomes are the changes you expect to result from your program. These can be changes in individuals, families, systems, policies, or institutions that you are trying to alter. Outcomes are often confused with outputs – outputs are the direct products of the activities undertaken to reach the desired outcomes. In the domestic violence field, outputs are often programs, number of trainings that occurred, materials distributed, and number of families served; whereas outcomes are things like knowledge transferred, behaviors changed, symptoms reduced, etc.

When considering how best to measure the success of a program or project, outcomes should be directly related to the goals and objectives. Program goals and objectives establish criteria against which you can determine program impact and effectiveness. It’s important to first identify the goals and objectives of the program or intervention you plan to evaluate. Logic models are a useful tool that can help you do this. Project-focused goals and objectives should be written as SMART goals to help you meet the long-term goals of your program. According to SAMSHA, “setting SMART objectives keeps the project moving forward, helps with accountability and timing, and lets you know that you are accomplishing what you set out to accomplish.” Goals should be specific, measurable, achievable, relevant, and time-bound.

SMART goals provide a framework for measurement of outcomes. The Promising Futures Guiding Principle on Storytelling encourages us to “capture stories and spread their impact using a wide range of interpersonal, cultural, and research and evaluation approaches…and to ensure different forms (qualitative, quantitative) and methods of collection (experimental, narrative, participatory action) are utilized in research and evaluation.” Ensuring that evaluation strategies are trauma informed and culturally relevant to the families you are working with is essential to gathering meaningful and accurate outcome data.

Surveys are the often the primary means of data collection in most programs, as data is often easy to collect from many people in a short amount of time. Descriptive statistics are generally enough analysis of surveys to show that outcomes have been achieved. Surveys help the team quantify opinions of participants, consumer satisfaction, utility of services, and other related topics. Other quantitative analyses are completed when using standardized and validated measures such as those used with clinical interventions. Then, data will be aggregated to show trends in symptom improvement as a result of treatment completion. Consider – written surveys must attend to literacy levels and be offered in languages that reflect the populations you are working with. Also conducting written surveys with communities that value oral communication over written may not be the most effective strategy to get accurate information about your program.

Qualitative data collection methods include individual semi-structured key informant interviews, open-ended surveys (also known as structured interviews), group interviews, focus groups, and observations. Qualitative data collection allows for more in-depth answers to questions and goes beyond what happens to why and how something happens. Most interviews will be semi-structured; that is, interviews will follow a protocol, but can be flexible. These interviews maximize data collection efficacy by making sure that pertinent questions are definitely asked but also allowing the interviewer to follow an unexpected train of thought. True focus groups require a specific format and group composition in which group members will ask each other questions to further the goals of the group facilitator. Most evaluators use simple group interviewing techniques to get the opinions of many similar individuals all at once. Most group interviews will be semi-structured, and a good facilitator will encourage all participants to speak.