If you run a small after-school program, you’ve probably had some challenges finding an evaluation tool that isn’t super-complicated and doesn’t cost a small fortune. This post is designed to point you in the right direction so you can find what you need.
First, let’s face it: it can be a challenge to assess after-school outcomes. Why? Because students are usually learning far more than just the skills you’re teaching.
You know this, right? It’s obvious that music students are learning how to play their instruments and express themselves musically, but that’s not all they’re learning. They’re also improving their ability to tolerate their frustration, listen to other players, collaborate, persevere, take risks, and a lot more. Those are all very valuable skills.
Students don’t just need these skills and abilities to play music – they need them to become good at just about anything. You probably already know this, but if you’re going to get the support you need for your after-school program, you need to assess and report on their learning so you can connect the dots for parents and funders.
You’ll connect the dots when you evaluate student improvement in those underlying skills and abilities they’re learning. By evaluating those, you’ll show your students are becoming more capable learners. A teacher observation scale can accomplish this nicely, and if you want to really ramp it up you can add a student self-esteem scale and possibly assessments by people who also know the student (like a parent, coach, or pastor). Then you’ll convincingly demonstrate your program’s competency in youth development.
Assessment will communicate the real value of your program to parents or funders. It will raise your program above the usual “skill-building program” to the greater sphere of youth development. Even better, it won’t be nebulous … your detail will show you know exactly what you’re doing.
Joanne Fritz’s post “How Nonprofits Benefit From Measuring Outcomes” at thebalancesmallbusiness clearly states why assessing student outcomes is so important:
Demand keeps growing for nonprofits to provide proof that what they are doing accomplishes something. All stakeholders of charities should have ample access to information about the organizations to which they donate, volunteer for, and from which they seek services.
Data is everywhere. That’s why charitable nonprofits must come to grips with explaining their results and their social impact.
This demand for data has grown steadily over the years, so it’s no longer a question of “if” you should evaluate, it’s a question of “how” you should evaluate.
Let’s start with an overview of what’s available for small after-school programs and how they work. Our research over the years has identified some overarching classifications that can be helpful in your search. These include:
One of the first questions you should ask yourself is whether you want an assessment that takes a one-time snapshot or one that provides periodic feedback. Your program’s age could be the deciding factor here.
If you’re a startup after-school program or just developing a new one, you will probably want to conduct frequent oversight assessments as you grow and tweak your program for success. In that case, the periodic assessment would probably be best for you, at least for now.
If you’re an established program serving large groups of young people, a yearly or bi-yearly assessment might be appropriate for you. Programs serving large student groups typically don’t perceive the need to drill deeply into individual student needs. They also often have the funding necessary to purchase outside assessments, which can be very pricy (our research revealed that the least expensive starts at just under $3,500 for the first year). However, a larger program that incorporates mentoring might consider ongoing assessment for just the mentoring component.
Actually, no matter what your size, if your program provides one-on-one mentoring services, the periodic assessment will probably be best for you. We say this because mentoring services typically address young people with higher risk factors, and regularly assessing at-risk students will allow teachers/mentors to drill down on students’ needs so they can provide the necessary support.
Just keep in mind that if you choose a periodic assessment, you’d better find one that your teachers and staff will like! If they don’t resonate with the metrics you’re using for assessment (the framework, or questions), they might not use the tool most effectively.
Most cloud-based assessment solutions we’ve found are for large, well-established social service organizations. These tools go beyond surveys and reports to include management tools such as case management. They are typically expensive, with onboarding and monthly charges. Since we’re focusing on affordable tools, we have not included those in this post.
Consider what you want to spend on your tool. While you probably have a budget in mind, we encourage you to read the rest of this post before you cast your budget in stone. While a free assessment tool might be perfect, for example, in the long run, it could end up being more expensive …
Since available affordable tools vary so widely, you need to clarify what you want to measure — what’s most important to you. We’ve created some categories to help you frame your search. While it’s true that most tools cross the categories, each one seems to fit more snugly into one.
While most tools attempt to measure overall student development, including their relationship with themselves and others, you may also want to measure two other important areas:
We hope you now have a frame in which you can position some of the available options and decide which is right for you. How would you like to proceed? You can devise an assessment tool on your own, or you can use any currently available resources. We’ve outlined a range of choices below to give you an overview of the available options.
In this post we offer five options for your consideration. Each is unique, so your choice will likely be determined by your decisions on what and how you want to measure, as well as whether or not you want to create a tool from scratch or use one out of the box (just be sure it’s customizable).
There are hundreds of choices available today! We hope this list offers something interesting or, at least, helps you narrow your field in terms of what kind of tool you want.
1. The Search Institute — Surveys
The Search Institute offers a well-established cadre of surveys based on their Framework of Developmental Assets – the 40 assets they delineate as the building blocks of positive youth development. They group these assets as external or internal; they cover almost every aspect of a student’s life.
The Institute’s surveys address the following topics:
The Institute will organize and report on the data you gather. Prices range from $250 – $400 for up to 100 surveys.
2. Youth.gov — The Youth Involvement and Engagement Tool (surveys)
The Youth Involvement and Engagement Tool is a 2-page .pdf survey. It’s available at youth.gov. This tool addresses:
While it, like many surveys, focuses on overall program outcomes rather than those of individual students, it does provide a good overview of program competence. This tool has no reporting facility. It is free of charge.
3. Merge Education — SETS: Student Evaluation & Tracking System (software)
The Student Evaluation & Tracking System is integrated software for smaller mentoring programs wanting to delineate, track, and report on specific student progress, with a drill-down on student learning as it relates to youth development. Originally developed to legitimize the arts as a powerful tool for learning and mentoring, this software assesses and reports on students and staff and manages students and classes. It implements educator Bill Rossi’s Principles of Empowerment detailed in Venturing Together: Empowering Students to Succeed.
It offers three assessment surveys, all of which can be modified if you want to use your own metrics:
As noted below, Merge Education offers free printouts of these assessments on its site.
This tool is PC-based and has a one-time price of $495. It’s sold by Merge Education at https://merge-education.com.
4. The After-School Initiative — Toolkit for Evaluating Positive Youth Development (guidance)
Published by The Colorado Trust, the initiative’s Toolkit is a free resource for assessing youth outcomes. This toolkit falls somewhere between “available tools” and “do it yourself” since it simply makes suggestions of focus areas and provides questions that you can use when surveying youth. It does not, however, include a way of organizing or reporting on data gathered.
5. The Afterschool Alliance — The Alliance offers information on organizations that offer STEM program assessment resources and tools here.
As noted on their site, these are:
If none of these existing tools suit your needs, you may want to create an assessment process on your own. If that’s your choice, be sure to enlist all of your staff in the effort. Everyone can be helpful, including your teachers, front-office staff, and even volunteers… you’ll also need everyone’s buy-in.
Plan on two or three meetings in which you and your team can drill down on what you feel constitutes a skillset that’s essential for empowered youth. Your list can be as simple or lengthy as you want – just don’t overdo it, since you’ll need to conduct assessments on a regular basis.
Once you’ve developed your list, think about how to reasonably organize the skills or abilities. Organizing your data is a prerequisite to generating reports to share with parents and funders. As we suggest below, combining surveys with Excel is a good way to do this.
Following are two very different methods you can use to design your tool. The first provides guidelines and an Excel template to create your tool; the second provides specific skills and abilities you can use as the basis of measurement.
1. The Emerging Markets Foundation (EMpower) provides a wide range of youth development evaluation guidelines on its website including:
2. Merge Education director Bill Rossi has identified an empowerment skillset he calls The Principles of Empowerment (POE). These principles highlight the most important skills and abilities students need in order to learn.
As detailed in his book Venturing Together: Empowering Students to Succeed, these are divided into Relationship to Self, Relationship to Teacher, and Skills Development. Following is a brief sketch of these principles and how they apply to assessing students in your afterschool program.
Merge Education offers its Teacher Observation Scale free of charge on its site https://merge-education.com.
Of course, your after-school assessment system needs to do more than collect data … you also need to organize that data to provide valuable reports you can use to manage your program and share with parents and funders.
If you’re doing it yourself with any of the free survey apps for data collection, you can use an Excel-like app to aggregate the results.
If you’re using existing surveys you can purchase reporting from the organization from which you purchased the surveys, such as the Search Institute, to aggregate your data.
If you’re a well-established after-school program such as Big Brothers, Big Sisters, or The Boys & Girls Club, you have a wide range of choices, although those tools typically focus heavily on program management.
One last thing to consider is how your program assessment data could help your teachers improve. If your assessment finds that students typically do not “feel valued for who they are”, for example, teachers will begin to find ways to let students know they’re appreciated. If you’re using an app that measures specific learning skills and it finds that some students are respectful of the environment but many are not, teachers will begin to consider how they can empower students to take more self-responsibility.
This might be a new experience for after-school teachers since they typically haven’t learned how to identify these skills. This means their teaching isn’t grounded in the principles of learning.
They also haven’t been trained how to assess those skills. This results in a lack of data, and the data can seriously enhance teaching. By taking periodic snapshots of your students, you not only clearly understand where they are today, but you also see what they need in order to take the next step. It’s like a roadmap through unfamiliar terrain – once you know exactly where the student is you can devise a path to help her get to where she needs to go… step by step.
A smart skill-building student assessment tool helps you stand back and look at students from a more comprehensive educational perspective. This helps you delineate the skills and abilities they’re actually developing. As you do this, you’ll also start to see more clearly how they’ll use those skills to learn everything else.
So, as I said, parents and funders typically don’t get how learning one skill relates to learning others. But they will, once you begin implementing an effective evaluation tool.
We hope we’ve offered a way to understand student assessment! If you know of an awesome tool we haven’t covered … or want more information on how to develop your own tool, contact us! We’d love to hear from you.
Mary Helen Rossi is Co-Director of Merge Education, which grew out of 20 years of after-school fine arts mentoring to over 2,500 challenged young people. Merge’s SETS software reflects this experience: as one client said, “It’s clear that a lot of thought and love went into SETS!”