funding-for-youth-development-program

5 Ways to Assess Youth Development Outcomes (and why you should)

If you run a small youth development program, you’ve probably had some challenges finding an evaluation tool that isn’t super-complicated and doesn’t cost a small fortune. You may also be frustrated because programs that track youth development outcomes are all so different. This post is designed to point you in the right direction so you can find what you need.

First, let’s face it: it can be a challenge to assess youth development outcomes. Why? Because students are usually learning far more than just the skills you’re teaching.

You know this, right? If you teach music, for example, it’s obvious that students are learning how to play their instruments and express themselves musically, but that’s not all they’re learning. They’re also improving their ability to tolerate their frustration, listen to other players, collaborate, persevere, take risks, and a lot more. Those are all very valuable skills.

Students don’t just need these skills and abilities to play music – they need them to become good at just about anything. You already know this, but if you’re going to get the support you need for your youth development program, you need to assess and report on their learning so you can connect the dots for parents and funders.

Student assessment after-school connects the dots for more funding

You’ll connect the dots when you evaluate student improvement in the skills and abilities I mentioned above, those skills that are important to all learning. By evaluating those, you’ll show that your students are becoming more capable learners.

A teacher observation scale can accomplish this nicely, and if you want to really ramp it up you can add a student self-esteem scale and possibly assessments by other people who know the student (like a parent, coach, or pastor). Then you’ll convincingly demonstrate your program’s competency in youth development.

Assessment will communicate the real value of your program to parents or funders. It will raise your program above the usual “skill-building program” to the greater sphere of youth development. Even better, it won’t be nebulous … your detail will show you know exactly what you’re doing.

Outcomes assessment ensures more sustainable funding

Joanne Fritz’s post “How Nonprofits Benefit From Measuring Outcomes” at thebalancesmallbusiness clearly states why assessing student outcomes is so important:

Demand keeps growing for nonprofits to provide proof that what they are doing accomplishes something. All stakeholders of charities should have ample access to information about the organizations to which they donate, volunteer for, and from which they seek services.

Data is everywhere. That’s why charitable nonprofits must come to grips with explaining their results and their social impact.

This demand for data has grown steadily over the years, so it’s no longer a question of “if” you should evaluate, it’s a question of “how” you should evaluate.

A step-by-step guide to measuring outcomes for youth development programs

Let’s start with an overview of what’s available for small youth development programs and how they work. Our research over the years has identified some overarching classifications that can be helpful in your search. These include:

  • One-time or periodic?
  • Cloud- or computer-based?
  • Free, affordable, or higher cost?

One-time or periodic student assessment?

student-assessmentOne of the first questions you should ask yourself is whether you want an assessment that takes a one-time snapshot or one that provides periodic feedback. Your program’s age could be the deciding factor here.

If you’re a startup youth development program or just developing a new one, you will probably want to conduct frequent oversight assessments as you grow and tweak your program for success. In that case, the periodic assessment would probably be best for you, at least for now.

If you’re an established program serving large groups of young people, a yearly or bi-yearly assessment might be appropriate for you. Programs serving large student groups typically don’t perceive the need to drill deeply into individual student needs. They also often have the funding necessary to purchase outside assessments, which can be very pricy (our research revealed that the least expensive starts at just under $3,500 for the first year). However, a larger program that incorporates mentoring might consider ongoing assessment for just the mentoring component.

Actually, no matter what your size, if your program provides one-on-one mentoring services, the periodic assessment will probably be best for you. We say this because mentoring services typically address young people with higher risk factors, and regularly assessing at-risk students will allow teachers/mentors to drill down on students’ needs so they can provide the necessary support.

Just keep in mind that if you choose a periodic assessment, you’d better find one that your teachers and staff will like! If they don’t resonate with the metrics you’re using for assessment (the framework, or questions), they might not use the tool most effectively.

Cloud-based or computer-based?

Most cloud-based assessment solutions we’ve found are for large, well-established social service organizations. These tools go beyond surveys and reports to include management tools such as case management. They are typically expensive, with onboarding fees and extra charges. Since we’re focusing on affordable tools, we have not included those in this post.

Free, affordable, or higher cost?

Consider what you want to spend on your tool. While you probably have a budget in mind, we encourage you to read the rest of this post before you cast your budget in stone. While a free assessment tool might look perfect, for example, in the long run, it could end up being more expensive …

You need to find the right tool so you have effective oversight
and
attract the funding you need.
Since available affordable tools vary so widely, you need to clarify what you want to measure — what’s most important to you. We’ve created some categories to help you frame your search. While it’s true that most tools cross the categories, each one seems to fit more snugly into one.

What do you want to measure?

While most tools attempt to measure overall student development, including their relationship with themselves and others, you may also want to measure two other important areas:

  • student learning, by focusing on specific skills and abilities
  • teacher effectiveness

So, what tool is right for you?

We hope you now have a frame in which you can position some of the available options and decide which is right for you. First, though, you’ll need to decide if you want to create an assessment tool on your own or use any currently available resources.

We’ve outlined a range of choices below to give you an overview of the available options.

Using an existing assessment tool or a mix of surveys

In this post, we offer five options for your consideration. Each is unique, so your choice will likely be determined by your decisions on what and how you want to measure, as well as whether or not you want to create a tool from scratch or use one out of the box (just be sure it’s customizable).

There are hundreds of choices available today! We hope this list offers something interesting or, at least, helps you narrow your field in terms of what kind of tool you want.

 

1. The Search Institute — Surveys

The Search Institute offers a well-established cadre of surveys based on their Framework of Developmental Assets – the 40 assets they delineate as the building blocks of positive youth development. They group these assets as external or internal; they cover almost every aspect of a student’s life.

The Institute’s surveys address the following topics:

  • Attitudes and Behaviors
  • Developmental Assets Profile
  • Developmental Relationships
  • Youth and Program Strengths

Once you’ve implemented their surveys and collected your data, the Institute will organize and report on the data you gather. Prices range from $250 – $400 for up to 100 surveys.

2. Youth.gov — The Youth Involvement and Engagement Tool (surveys)

The Youth Involvement and Engagement Tool is a 2-page .pdf survey. It’s available at youth.gov. This tool addresses:

  • Youth Involvement
  • Youth Engagement (Community)
  • Youth Retention

While it, like many surveys, focuses on overall program outcomes rather than those of individual students, it does provide a good overview of program competence. This tool has no reporting facility. It is free of charge.

 

3. Merge Education — SETS: Student Evaluation & Tracking System (software)

The Student Evaluation & Tracking System is integrated software for small and growing mentoring programs wanting to delineate, track, and report on specific student progress, with a drill-down on student learning as it relates to youth development. Originally developed to legitimize the arts as a powerful tool for learning and mentoring, this software assesses and reports on students and staff and manages students and classes. It implements educator Bill Rossi’s Principles of Empowerment detailed in Venturing Together: Empowering Students to Succeed.

It offers three assessment surveys, all of which can be modified if you’d like to use your own metrics:

  • Self Esteem
  • Teacher Observation
  • Assessment (by a ‘significant other’)

As noted below, Merge Education offers a free printout of the assessment scale on its site.

This tool is PC-based (not in the cloud) and can be purchased for a one-time fee or on a subscription basis.  It’s the most affordable tool we have found on the market, and sold by Merge Education at https://merge-education.com.

,

4. The After-School Initiative — Toolkit for Evaluating Positive Youth Development (guidance)

Published by The Colorado Trust, this Toolkit is a free resource for assessing youth outcomes. This toolkit falls somewhere between “available tools” and “do it yourself” since it simply suggests focus areas and provides questions that you can use when surveying youth. It does not, however, provide a means of organizing or reporting on data gathered.

 

5. The Afterschool Alliance — The Alliance offers information on organizations that offer STEM program assessment resources and tools here.

As noted on their site, these are:

  • PEAR’s Assessment Tools in Informal Science, a searchable database of assessment tools for evaluating program quality and outcomes
  • Assessing Women and Men in Engineering Project, a set of surveys and assessment tools you can use to evaluate afterschool STEM programs
  • CAISE’s evaluation toolkit, for those who have been awarded an NSF grant for informal science projects and useful for learning how to successfully evaluate a program.
  • Better Evaluation, an international collaboration focused on improving evaluation theory and practice through the development and sharing of resources

Creating your own assessment tool (DIY)

If none of the existing tools suit your needs, you may want to create an assessment process on your own. If that’s your choice, be sure to enlist all of your staff in the effort. Everyone can be helpful, including your teachers, front-office staff, and even volunteers… and you’ll need everyone’s buy-in.

Plan on two or three meetings in which you and your team can drill down on what you feel constitutes a skillset that’s essential for empowered youth. Your list can be as simple or lengthy as you want – just don’t overdo it, since you’ll need to conduct assessments on a regular basis.

Once you’ve developed your list, think about how to reasonably organize the skills or abilities. Organizing your data is a prerequisite to generating reports to share with parents and funders. As we suggest below, combining surveys with Excel is a good way to do this.

DIY: Tool design

Following are two very different methods you can use to design your tool. The first provides guidelines and an Excel template to create your tool; the second provides specific skills and abilities you can use as the basis of measurement.

1. The Emerging Markets Foundation (EMpower) provides a wide range of youth development evaluation guidelines on its website including:

  • data collection methods
  • pre- and post-test design
  • measuring changes in self-esteem and confidence
  • measuring changes in gender belief and norms
  • an Excel template for pre- and post-analysis

assessing-students

2. Merge Education director Bill Rossi has identified an empowerment skillset he calls The Principles of Empowerment (POE). These principles highlight the most important skills and abilities students need in order to learn.

As detailed in his book Venturing Together: Empowering Students to Succeed, these are divided into Relationship to Self, Relationship to Teacher, and Skills Development. Following is a brief sketch of these principles and how they apply to assessing students in your afterschool program.

  • Relationship to Self: These are the personal qualities and skills a student needs to develop so she’ll be able to learn well. Self-confidence is high on the list, as is her level of motivation. Another is how well she does under pressure — including the pressure of her own frustration. These are delineated as: Ability to concentrate, Level of motivation, Self-confidence to succeed, Frustration tolerance, and Consistent effort.
  • Relationship to Teacher: Students’ relationships with others is an important indicator of empowerment, and your teachers are well-positioned to evaluate this. How well does the student listen? Better yet, does she understand what she’s hearing? Can she communicate her ideas and her needs, or is she frustrated but unable to explain why? These are delineated as: Listens to the teacher, Follows directions, Communicates needs, Communicates ideas.
  • C. Skills Development: This concerns whether or not the student is becoming a more capable learner. Does the student stay safe with what she already knows or is she willing to try new steps? Is she able to build on prior learning? How does she relate to the equipment and materials in the classroom — does she treat them carefully yet with a sense of curiosity and freedom? These are delineated as: Respects equipment and materials, Willing to try new steps, Freedom of expression, Identifies correlations (relationships), Able to build on prior learning, Incorporates elements of the skill.

 

DIY: Organizing and reporting on your data

Of course, your youth development assessment system needs to do more than collect data … you also need to organize that data to provide valuable reports you can use to manage your program and share with parents and funders.

If you’re doing it yourself with any of the free survey apps for data collection, you can use an Excel-like app to aggregate the results.

If you’re using existing surveys you can purchase reporting from the organization from which you purchased the surveys, such as the Search Institute, to aggregate your data.

If you’re a well-established after-school program such as Big Brothers, Big Sisters, or The Boys & Girls Club, you have a wide range of choices, although those tools typically focus heavily on program management.

Youth development outcomes data can improve teaching

assess-teachers

One last thing to consider is how your program assessment data could help your teachers improve. If your assessment finds that students typically do not “feel valued for who they are”, for example, teachers will begin to find ways to let students know they’re appreciated. If you’re using an app that measures specific learning skills and it finds that some students are respectful of the environment but many are not, teachers will begin to consider how they can empower students to take more self-responsibility.

This might be a new experience for teachers since they typically haven’t learned how to identify these skills. This means their teaching isn’t grounded in the principles of learning.

They also haven’t been trained how to assess those skills. This results in a lack of data, and the data can seriously enhance teaching. By taking periodic snapshots of your students, you not only clearly understand where they are today, but you also see what they need in order to take the next step. It’s like a roadmap through unfamiliar terrain – once you know exactly where the student is you can devise a path to help her get to where she needs to go… step by step.

Take it one step at a time!

A smart skill-building student assessment tool helps you stand back and look at students from a more comprehensive educational perspective. This helps you delineate the skills and abilities they’re actually developing. As you do this, you’ll also start to see more clearly how they’ll use those skills to learn everything else.

So, as I said, parents and funders typically don’t get how learning one skill relates to learning others. But they will, once you begin implementing an effective evaluation tool.

We hope we’ve offered a way to understand student assessment! If you know of an awesome tool we haven’t covered … or want more information on how to develop your own tool, contact us! We’d love to hear from you.

Merge Education

Mary Helen Rossi is Co-Director of Merge Education, which grew out of 20 years of after-school fine arts mentoring to over 2,500 challenged young people. Merge’s SETS software reflects this experience. As one client said, “It’s clear that a lot of thought and love went into SETS!”