For a startup nonprofit, the idea of "program evaluation" can be overwhelming, and yet funders are increasingly asking grantees to provide not only numbers served, but demographic information and outcomes-based information. As more nonprofits compete for a reduced funding pool, it's important that they know and understand the difference their investment made in your ability to carry out the organizational mission. Let's say your mission is to provide a safe place for youth. Sure, you may have given Jim a place to sleep for the night, but how did that positively impact his life and his future? What did you do for Jim to prevent him from having to use your services in the future? How many "Jim's" did you help in 2011? How many did you turn away? What are their ages? Where did they come from? Why did they come? How many are high school drop outs? How many have their GED? How many are immigrants? And so on....
The secret to getting started is to keep program evaluation simple and include board members, staff, and anyone else involved in data collection and use. It's easy to end up with "analysis paralysis" if you try to do too much too soon. Realize that program evaluation is a process, not an event. It can take many months to implement, and years to refine. What's important is that you get started now. If you receive grant funding, you really have no choice. We are here to help!
STEP ONE: Decide What You Will Measure. First you need to know what outcomes your funders want at year's end. Second, you must track the information your organization's leadership needs in order to make knowledgeable decisions that either impact the course of your organization's future or ensure your organization is tracking with your strategic plan (if you have one). This process can be as simple as reviewing different reports required by existing funders and the grant guidelines of your prospects. Sit down with your board president or create an ad-hoc Program Evaluation committee to take on the task. There are several levels of knowledge you might need when it comes to collecting program data:
· Data including age, residence, income, race, ethnicity (demographics)
· Numbers served, goods distributed, or other units of service you define (outputs)
· The impact of your services on the lives of your clients (outcomes)
Demographic data is easy to collect. It can be as simple as handing each client a form to fill out. If confidentiality is important, your form doesn't require a name. In order to count numbers served or units of service, staff must be trained to record the people they serve each day and the specific activities they engage in with the client.
Measuring the impact your services make on clients is a bit difficult, but it can be the most valuable and rewarding data you collect. If you decide to measure impact, it will benefit you to explore creating a logic model. A logic model illustrates the relationships between inputs, activities, outputs, outcomes and goals that create the structure and flow of your organization. Briefly, inputs are the resources that go into making your program activities possible, such as technology, training, people and funding. The activities you undertake as part of your program are what you will do to accomplish your goals. Outputs are the product of your program, as in the number of people served or the number of items distributed or made. Finally, outcomes are the impact of your work on those you serve, or the change affected in their lives. Logic models are a complex topic and there are many different approaches that can be taken to developing one for your program. Resources are available for assistance in creating a logic model, including the United Way of America's book "Measuring Program Outcomes," which is available at United Way Store's website for a mere $5. The W.K. Kellogg Foundation has also created a more complex logic model guide which can be downloaded free from their Knowledge Center.
Finally, set quantitative goals around the items you've decided to measure. If you exclusively measure demographics, create a goal that 80% of clients served are classified homeless by federal definition, which helps you measure the impact of organizational outreach, or marketing and advocacy work. If you measure numbers served or other units of service, your goal could be to provide 1,000 children with shoes in a 12-month period, or provide 800 hours of community outreach to areas with high youth crime rates. An example of an outcome to measure impact would be that 75% of clients are able to provide for their food and shelter needs upon exit from your program, or that 60% of individuals receiving parenting education report a greater sense of harmony and well being in their households after they complete a course.
STEP TWO: Determine the Tools You Will Need. Assess your current evaluation tools, such as client intake forms or reporting already required by current funders, refining what you're already doing and preventing duplication. The tools you need and the cost of collecting data depends on what you've decided to measure and the complexity of your plan. If you've decided to collect demographic data, you can use current intake forms by including questions which add depth. If you collect numbers served or units of service involve your staff in deciding how to do it efficiently. Use simple ticker sheets compiled by hand, or use electronic spreadsheets with embedded formulas to automatically tabulate numbers. Google Docs offers a free way to create electronic forms in an online database which allows you to distribute to a number of users via email. The software automatically compiles the numbers submitted on the forms into a single spreadsheet which you can use for reporting. A more costly option is to purchase database management software. There are several companies offering tailor-made solutions.
When collecting outcome-based data to measure impact, it can be difficult to determine the best method. Let's go back to Jim. After providing Jim with a place to sleep for the night, your outcome is to link him with community resources and increase his knowledge and utilization of such services, and to ensure he follows through and uses your referrals. In order to accurately measure the outcome, you could provide him with an exit survey allowing him to rate your services and the likelihood that he will follow through with the referrals on a numerical scale. Let's say Jim does return; during the intake rate his level of improvement on a scale from 1 to 5 since his last visit. At year-end calculate how many clients returned once, twice, or never. If you come up with a logical way of quantifying this data, you can easily record it in a database and tabulate it. Remember that you are the expert when it comes to your work and the data collection methods most applicable and reasonable for your particular service(s).
STEP THREE: Assign Responsibility. When possible, put this responsibility in the appropriate staff person(s) job description. When you designate a person to manage the data collection process, including compilation and reporting, you are 90% on your way to success. Make sure you choose someone who has the skills and understands the value in evaluation, and then provide them with the tools and training. In our hypothetical "safe place for youth" scenario, it makes sense to designate staff in charge of client intake and discharge to collect and manage data collection. If data collection is new to your organization and the responsibility was not in a job description upon hire, you may encounter resistance from program staff to take on additional tasks. We suggest you don't act flip about adding to their already long list of responsibilities Instead, introduce data collection in the form of a PowerPoint presentation to the entire staff. Begin with your logic model, as it will best convey that program evaluation requires everyone's full support and assistance. It will also clearly explain who is best positioned to collect the data: program staff. Place a quantitative value on the importance of data collection and how it will directly and positively impact their job and their work with clients by, for instance, sharing the amount of support received from funders who require the data. Well done evaluation opens the door to new and increased funding; it gathers emotional and financial support from your community as you tell your story through qualified and quantified data which proves you're making a difference.
STEP FOUR: Implement. You're ready to implement the project evaluation plan once you've decided what to measure, obtained and prepared the necessary tools, and determined who is responsible. Implementation should begin by creating a clear and concise procedure manual. This manual will define those responsible, the timing and frequency of data collection, and when and how often reports are produced and reviewed. It will also serve as an invaluable resource for training new staff. Your designated data manager(s) must be vigilant in the beginning, checking in on staff to ensure they follow the procedure. In an effort to make your oversight seem less invasive, solicit suggestions on how the system is working and whether there are ways to improve the system and make it more efficient. Ask all staff responsible to track the amount of time data collection is requiring and watch closely to see if the time allotted begins to wane as they grow accustomed to the process. When you secure new funding, double check your current data collection to ensure your existing system will provide the evaluation data they require; if not, implement their unique specifications as seamlessly as possible and update the staff, forms and procedure manual.
STEP FIVE: Utilize your Data. This is the step where many nonprofits tend to fall short; conversely it can be where the process of collecting data and evaluating comes full circle via new, renewed or upgraded gifts.
Most likely your organization has a strategic plan. Every decision your board makes should relate directly to that plan. A strategic plan primarily takes into account all things related to successfully and sustainably carrying out your mission over the long haul. This information boils down to finance and program impact to make certain your organization or program is carrying out its mission. You feed your board this information as a result of successful data collection. If your data reveals problems or issues, leadership can make informed decisions and act accordingly. If your data reflects positive results, send out a press release and publicize your successful outcomes in your newsletter(s), annual report, and website.
All donors (whether corporate, foundation or individual) want and need to know their return on investment. Their return on investment can be given qualitatively through client testimonials, and quantitatively by showing impact in numbers. For example, maybe you've helped 1,000 people increase their knowledge of how to gain employment by 80%. That's something to celebrate!
A program evaluation system is beneficial to nonprofits on many levels. Developing the system requires organizational investment, but it doesn't have to be cumbersome or costly. Some orgaÃ�Ânizations may feel they can't afford to spare any amount of time for such a project. We say organizations can't afford to ignore such a project. If nonprofits want to survive and thrive, they must acknowledge their duty to supporters and clients by measuring the impact (or lack of impact) of their mission.
Marcie L. Wagner, fundraising consultant, is the founder and principal of ML Wagner Fundraising Group. We are a boutique, fundraising consulting firm. With two decades of experience in professional fundraising, grant research, grant writing, and communications, Marcie is an expert in raising money with little expense attached; as it should be.
Her favorite success stories are those attached to working with smaller organizations. Groups with passionate leadership, a unique mission, poised for greatness - all they need is a gentle push in the right direction. Ms. Wagner is considered by many to be an expert in the area of annual giving, online fundraising, grant research and writing, and branded communications; particularly with organizations who are just beginning to form strategic development and communication efforts. Her no-nonsense approach in branding an organization to enhance its development efforts gave many organizations the tools to launch successful grant and major giving campaigns.
The secret to getting started is to keep program evaluation simple and include board members, staff, and anyone else involved in data collection and use. It's easy to end up with "analysis paralysis" if you try to do too much too soon. Realize that program evaluation is a process, not an event. It can take many months to implement, and years to refine. What's important is that you get started now. If you receive grant funding, you really have no choice. We are here to help!
STEP ONE: Decide What You Will Measure. First you need to know what outcomes your funders want at year's end. Second, you must track the information your organization's leadership needs in order to make knowledgeable decisions that either impact the course of your organization's future or ensure your organization is tracking with your strategic plan (if you have one). This process can be as simple as reviewing different reports required by existing funders and the grant guidelines of your prospects. Sit down with your board president or create an ad-hoc Program Evaluation committee to take on the task. There are several levels of knowledge you might need when it comes to collecting program data:
· Data including age, residence, income, race, ethnicity (demographics)
· Numbers served, goods distributed, or other units of service you define (outputs)
· The impact of your services on the lives of your clients (outcomes)
Demographic data is easy to collect. It can be as simple as handing each client a form to fill out. If confidentiality is important, your form doesn't require a name. In order to count numbers served or units of service, staff must be trained to record the people they serve each day and the specific activities they engage in with the client.
Measuring the impact your services make on clients is a bit difficult, but it can be the most valuable and rewarding data you collect. If you decide to measure impact, it will benefit you to explore creating a logic model. A logic model illustrates the relationships between inputs, activities, outputs, outcomes and goals that create the structure and flow of your organization. Briefly, inputs are the resources that go into making your program activities possible, such as technology, training, people and funding. The activities you undertake as part of your program are what you will do to accomplish your goals. Outputs are the product of your program, as in the number of people served or the number of items distributed or made. Finally, outcomes are the impact of your work on those you serve, or the change affected in their lives. Logic models are a complex topic and there are many different approaches that can be taken to developing one for your program. Resources are available for assistance in creating a logic model, including the United Way of America's book "Measuring Program Outcomes," which is available at United Way Store's website for a mere $5. The W.K. Kellogg Foundation has also created a more complex logic model guide which can be downloaded free from their Knowledge Center.
Finally, set quantitative goals around the items you've decided to measure. If you exclusively measure demographics, create a goal that 80% of clients served are classified homeless by federal definition, which helps you measure the impact of organizational outreach, or marketing and advocacy work. If you measure numbers served or other units of service, your goal could be to provide 1,000 children with shoes in a 12-month period, or provide 800 hours of community outreach to areas with high youth crime rates. An example of an outcome to measure impact would be that 75% of clients are able to provide for their food and shelter needs upon exit from your program, or that 60% of individuals receiving parenting education report a greater sense of harmony and well being in their households after they complete a course.
STEP TWO: Determine the Tools You Will Need. Assess your current evaluation tools, such as client intake forms or reporting already required by current funders, refining what you're already doing and preventing duplication. The tools you need and the cost of collecting data depends on what you've decided to measure and the complexity of your plan. If you've decided to collect demographic data, you can use current intake forms by including questions which add depth. If you collect numbers served or units of service involve your staff in deciding how to do it efficiently. Use simple ticker sheets compiled by hand, or use electronic spreadsheets with embedded formulas to automatically tabulate numbers. Google Docs offers a free way to create electronic forms in an online database which allows you to distribute to a number of users via email. The software automatically compiles the numbers submitted on the forms into a single spreadsheet which you can use for reporting. A more costly option is to purchase database management software. There are several companies offering tailor-made solutions.
When collecting outcome-based data to measure impact, it can be difficult to determine the best method. Let's go back to Jim. After providing Jim with a place to sleep for the night, your outcome is to link him with community resources and increase his knowledge and utilization of such services, and to ensure he follows through and uses your referrals. In order to accurately measure the outcome, you could provide him with an exit survey allowing him to rate your services and the likelihood that he will follow through with the referrals on a numerical scale. Let's say Jim does return; during the intake rate his level of improvement on a scale from 1 to 5 since his last visit. At year-end calculate how many clients returned once, twice, or never. If you come up with a logical way of quantifying this data, you can easily record it in a database and tabulate it. Remember that you are the expert when it comes to your work and the data collection methods most applicable and reasonable for your particular service(s).
STEP THREE: Assign Responsibility. When possible, put this responsibility in the appropriate staff person(s) job description. When you designate a person to manage the data collection process, including compilation and reporting, you are 90% on your way to success. Make sure you choose someone who has the skills and understands the value in evaluation, and then provide them with the tools and training. In our hypothetical "safe place for youth" scenario, it makes sense to designate staff in charge of client intake and discharge to collect and manage data collection. If data collection is new to your organization and the responsibility was not in a job description upon hire, you may encounter resistance from program staff to take on additional tasks. We suggest you don't act flip about adding to their already long list of responsibilities Instead, introduce data collection in the form of a PowerPoint presentation to the entire staff. Begin with your logic model, as it will best convey that program evaluation requires everyone's full support and assistance. It will also clearly explain who is best positioned to collect the data: program staff. Place a quantitative value on the importance of data collection and how it will directly and positively impact their job and their work with clients by, for instance, sharing the amount of support received from funders who require the data. Well done evaluation opens the door to new and increased funding; it gathers emotional and financial support from your community as you tell your story through qualified and quantified data which proves you're making a difference.
STEP FOUR: Implement. You're ready to implement the project evaluation plan once you've decided what to measure, obtained and prepared the necessary tools, and determined who is responsible. Implementation should begin by creating a clear and concise procedure manual. This manual will define those responsible, the timing and frequency of data collection, and when and how often reports are produced and reviewed. It will also serve as an invaluable resource for training new staff. Your designated data manager(s) must be vigilant in the beginning, checking in on staff to ensure they follow the procedure. In an effort to make your oversight seem less invasive, solicit suggestions on how the system is working and whether there are ways to improve the system and make it more efficient. Ask all staff responsible to track the amount of time data collection is requiring and watch closely to see if the time allotted begins to wane as they grow accustomed to the process. When you secure new funding, double check your current data collection to ensure your existing system will provide the evaluation data they require; if not, implement their unique specifications as seamlessly as possible and update the staff, forms and procedure manual.
STEP FIVE: Utilize your Data. This is the step where many nonprofits tend to fall short; conversely it can be where the process of collecting data and evaluating comes full circle via new, renewed or upgraded gifts.
Most likely your organization has a strategic plan. Every decision your board makes should relate directly to that plan. A strategic plan primarily takes into account all things related to successfully and sustainably carrying out your mission over the long haul. This information boils down to finance and program impact to make certain your organization or program is carrying out its mission. You feed your board this information as a result of successful data collection. If your data reveals problems or issues, leadership can make informed decisions and act accordingly. If your data reflects positive results, send out a press release and publicize your successful outcomes in your newsletter(s), annual report, and website.
All donors (whether corporate, foundation or individual) want and need to know their return on investment. Their return on investment can be given qualitatively through client testimonials, and quantitatively by showing impact in numbers. For example, maybe you've helped 1,000 people increase their knowledge of how to gain employment by 80%. That's something to celebrate!
A program evaluation system is beneficial to nonprofits on many levels. Developing the system requires organizational investment, but it doesn't have to be cumbersome or costly. Some orgaÃ�Ânizations may feel they can't afford to spare any amount of time for such a project. We say organizations can't afford to ignore such a project. If nonprofits want to survive and thrive, they must acknowledge their duty to supporters and clients by measuring the impact (or lack of impact) of their mission.
Marcie L. Wagner, fundraising consultant, is the founder and principal of ML Wagner Fundraising Group. We are a boutique, fundraising consulting firm. With two decades of experience in professional fundraising, grant research, grant writing, and communications, Marcie is an expert in raising money with little expense attached; as it should be.
Her favorite success stories are those attached to working with smaller organizations. Groups with passionate leadership, a unique mission, poised for greatness - all they need is a gentle push in the right direction. Ms. Wagner is considered by many to be an expert in the area of annual giving, online fundraising, grant research and writing, and branded communications; particularly with organizations who are just beginning to form strategic development and communication efforts. Her no-nonsense approach in branding an organization to enhance its development efforts gave many organizations the tools to launch successful grant and major giving campaigns.