16 Nov How to evaluate your adult literacy program in 4 easy steps

As a non-profit literacy organization, you’re expected to regularly report to your board of directors, funders, service users, and other key stakeholders about the effectiveness of your literacy program’s initiatives.

In an industry that’s strapped for resources – both financial and human capital – it can be challenging to make program evaluation a priority. Nonetheless, it’s extremely important in ensuring your organization is held accountable to its funders. Additionally, it helps to provide an understanding of the program’s progress, success and effectiveness.

While program evaluation can often feel overwhelming and intimidating, Kim Chung, Co-Executive Director of Programming and Training at the Centre for Family Literacy in Edmonton, Alberta, has welcomed it. ABC Life Literacy Canada interviewed Chung to find out more about her best practices for program evaluation.

“Most people do evaluations because they have to, and there usually limited resources to devote to it. We’re all guilty of doing the bare minimum just to get by, but it doesn’t help that much when we look at it as just another box to tick off our to do list.”

Chung explains that when organizations don’t use the evaluation metrics to actually improve the program and make change, there’s really no point in collecting it.

If the evaluation process seems cumbersome to you, follow our four simple steps in evaluating your organization’s adult literacy program.

Step #1: Consider Overall Program Objectives

Before you begin your program evaluation, take time to pinpoint what exactly you will be evaluating. Will you be reviewing only one of your programs or multiple programs? When did the programs occur that you will be using for your evaluation? What specific skills are learners expecting to obtain from attending your programs?

Evaluation is not a one-size-fits-all process. Depending on the requirements of the evaluation, different tools or techniques may be required.

Chung says that the Centre for Family Literacy relies on logic models to help with program evaluation, and that each program has a different model. Logic models identify what you’re trying to achieve, and whether or not you’ve met those goals. Different programs will likely have different logic models, and it can be helpful to set these up at the start of each program.

Step #2: Use Forms and Tools to Assess Participants’ Progress

Assessing the impact of the program on learners is one of the most important part of the evaluation process. Determining whether learners have improved the skills you set out to improve and met their objectives can be the difference between getting continued funding or having your funding pulled.

Conducting and reviewing assessment interviews is one of the best ways to determine if your programs are meeting your participants’ expectations and requirements. The process involves gathering information from the participant at three different stages during the learning process – typically at the program’s onset, partway through the program, and at program completion.

Chung recommends a combination of tactics, such as surveys, polls, observations and conversations or focus groups. She says that evaluating people in different ways eliminates barriers and ultimately provides more helpful feedback.

Using the results from the assessments, data can be gained about program information. Are the evaluations displaying advancements towards learner goals? Have learners achieved optimal outcomes before leaving the program?

According to Chung, the most important measurables are the story pieces. “When we talk to learners and hear their stories of progress, the ones that hit you in the heart, it is so much more impactful. You still need the hard numbers, the metrics, but a combination of numbers and stories makes the results of the program more tangible. Stories bring the data to life.”

To get participants to open up more, Chung advises explaining to them how their stories will be told and how their feedback may help someone else. Allowing their stories to remain anonymous can help reduce any hesitancy from learners to participate.

Step #3: Seek Insight from Program Staff

Besides the literacy program participants, the staff at your organization should also be involved in the program evaluation process. As the individuals who are promoting, coordinating and teaching the programs, they can offer valuable insight.

Chung advises on creating a culture of evaluation within the organization to ensure that practitioners are constantly thinking about it throughout their teaching. To do so, Chung says to ask staff for their input and participation in the development of the measurement tools.

“Measurement is an ongoing process. If practitioners can monitor their learners throughout the program, ask questions, and see whether they’re reaching their objectives, then they can easily report back on the success of it. The stories and conversations are just as important as the metrics.”

Choose whether you would prefer gathering information either anonymously (using online survey programs) or in-person information sessions. Chung suggests using a variety of evaluation methods.

“If you do a survey, you only have the information as a moment in time. But if you train your staff to watch and observe throughout the program, it can provide much more helpful information.”

Ask staff to provide honest feedback about program offerings. What is currently being done that is working well? Are they finding that the programs are in alignment with your organization’s goals? If not, what can be improved upon? Are there any major barriers that are preventing students from reaching their full potential before program completion?

Step #4: Communicate Findings with Key Stakeholders

After compiling the information from the previous three steps, it’s ready to present to key stakeholders.

Sharing your evaluation results presents a great opportunity for you to validate your interpretation of the information collected. Based on those findings, program partners can provide input about your programs and help make decisions about your organization’s future.

With regards to the way you present your information, it may depend on what is required by your funders.

Chung says for most government funders, there’s a template that is provided where you simply fill in the information. There’s usually a limit to the number of words, so it can be challenging to present the story of your program success in an innovative way.

“People remember stories, not numbers,” says Chung. “Even though funders need the numbers, it’s important to find a way to combine the numbers with one of your best success stories to make the results memorable. Remember that funders don’t usually have a lot of time to read the information, so getting creative in the way you present it, perhaps through a well-written case study or a short video, can go a long way.”

Conclusion

To be a provider of a quality adult literacy program, you need to continually evaluate your program offerings. Doing so not only holds your organization accountable to funders but also provides useful information to direct future program planning.

For more information about adult literacy programs and resources, contact us today.