SETS: Student Assessment and Program Management for Youth Programs
Developmental student assessment and program management are rarely baked together for afterschool and youth development programs. SETS changes that. Observe each student from three perspectives, manage rosters, attendance, and demographics, and produce funder-ready reports – all in one system.
Staff rate students at regular intervals on items that describe what growth looks like, so a program can see how each student is developing during the year, not only after it ends. Because student assessment and program records live in the same place, staff enter information once, and reports for funders, parents, and boards draw on both.
SETS grew out of decades of direct work with young people, and its items are written in the language staff already use.
The problem SETS addresses
Afterschool and youth development programs are asked to show outcomes, but most small and mid-size programs have no practical way to measure them. Attendance and enrollment are easy to count. Growth in confidence, self-responsibility, and engagement is not.
Programs that do student assessment often piece together a survey tool, spreadsheets, and a separate data system. That adds administrative work for staff who are already stretched, and it rarely produces reports a funder can use. Many programs learn how their students are doing only from an annual outside evaluation, after the program year is over.
SETS gives programs one system for observing student development, tracking participation and demographics, and reporting results to funders, parents, and boards.
Three perspectives on every student
SETS is a multi-informant assessment. Each student can be rated from three perspectives, so a program sees growth from more than one vantage point. Programs may use all three scales or any one of them.
Each perspective shows something the others can’t. Staff see how a student works and responds in the program. The student’s own ratings show how they see themselves as their skills grow. A parent, guardian, or counselor (significant other) shows whether that growth carries over to home, school, and elsewhere. Repeated over time, the three together give a fuller picture of a student’s progress than any single rating can.
| Scale | Completed by | What it shows |
|---|---|---|
| Teacher Observation Scale | Teachers and program staff | Growth staff observe in the program, rated on behaviorally anchored items |
| Student Self-esteem Scale | The student | How students see themselves and their own progress |
| Significant Other Scale | Parents, guardians, counselors, or other adults close to the student | Growth observed outside the program |
Observation that staff can apply consistently
Every item on the Teacher Observation Scale is behaviorally anchored: each rating point is defined by a description of what that level looks like in a student. This design supports consistency across observers and reduces the subjectivity common in student assessment rating scales.
For example, the anchors for “Ability to concentrate”:
1 – Needs frequent reminders to refocus.
2 – Sometimes needs reminders to refocus.
3 – Rarely needs reminders to refocus.
4 – Refocuses self during occasional lapses in concentration.
5 – Focuses on work without reminders or cues.
This scale and its anchors do something most assessments don’t: the scale tracks benchmarks that are integral to learning and growth, and the anchors describe what growth looks like. That means that simply by using SETS, new and part-time staff learn what to look for in their students. Assessment becomes a form of professional development.
“SETS has actually made my job easier. It’s changed my perception of evaluation tools.” Thomasina Winslow, Program Director, St. Anne Institute, Albany, NY
Current data throughout the year
SETS gives programs current data throughout the year, so staff can see how students are progressing while there is still time to respond, rather than waiting for an annual evaluation.
Because students are rated at regular intervals, SETS also shows when a student’s progress stalls or slips, early enough for staff to adjust their approach and build that area into a strength. When an outside evaluation does take place, the program has a year of consistent observations to offer it.
Grounded in practice
Bill Rossi began his youth programming by founding YATMA (Youth Advancement through Music & Art). Together, YATMA and Merge Education went on to serve more than 3,000 youth facing significant challenges.
Rossi’s strengths-based approach came naturally to him, but most teachers have to learn it. Over years of that work, he developed the Principles of Empowerment, a way of putting his approach into words so other teachers could learn and apply it.
SETS was built by a practitioner, for practitioners. Its items describe what staff actually see in a session, in the language they already use, so it fits programs whose staff come to the work through experience rather than formal training. Rossi developed SETS with a team of educators, psychologists, therapists, and software developers, with input from a child and adolescent psychiatrist, Wander de C. Braga, M.D.
The Principles are the framework SETS is built on. They define fifteen observable skills students need in order to learn, from concentration and frustration tolerance to communicating ideas and building on prior learning, grouped into three areas: Relationship to Self, Relationship to Staff, and Skills Development.
The approach is set out in Rossi’s book Venturing Together: Empowering Students to Succeed. Rather than staying in theory, the book grounds the approach in concrete teaching moments: the situations teachers meet in lessons and the words they can use with students. It closes with teachers’ own profiles of their students, showing the approach in practice.
SETS student assessment is aligned with CASEL’s framework for social and emotional learning.
Student assessment proven in the field
SETS took shape in institutional work in Albany, NY. Beginning in 1999, early versions were used and refined at Parsons Child and Family Center, a residential treatment and education center serving children with severe trauma, and at St. Anne Institute. Working with those students and the staff who served them shaped the instrument into what it is today.
For four years, SETS then served as the outcome-reporting instrument for Merge Education, Inc. under a contract with the Chester County Department of Human Services (West Chester, PA), serving both its health and mental health divisions. SETS reports satisfied the Department’s reporting requirements throughout the contract period, showing that a single instrument could meet the reporting needs of two funding streams.
What programs get
Assessment & Reporting
- Individual student profiles and progress reports
- Group and program-level comparisons, including comparison groups
- Clear, professional reports for funders, parents, and boards
- Support for writing concise, achievable student goals
- Surveys that programs can adapt to their own goals
Program Management
- Students, families, staff, and partner organizations in a single database
- Class rosters, activities, attendance, and demographics
- Staff evaluations that identify strengths and training needs, reported alongside students’ goal achievement
- Data import from the systems a program already uses
- Multi-user use, with staff working from one shared data file
- One source of truth for both participation data and student outcomes, so staff enter information once
Where SETS fits
Most programs already use one of two kinds of tools. Assessment tools provide an instrument but not program management. Administrative platforms handle enrollment, classes, and attendance but don’t measure how students are developing.
SETS provides both a developmental instrument and the program data around it, at a cost small and mid-size programs can manage. It can also work alongside the administrative systems a program already uses, as the outcome layer those systems lack.
“SETS is a comprehensive and unique assessment tool… exactly what Mighty Writers needed! It’s simple, concise, and effective.” Laura Corbett, Mighty Writers, Philadelphia, PA
Current status
SETS is a locally installed Windows application. The current version runs well and is available for demonstration, with four manuals built into the software. Its scales, report designs, and documentation provide a complete specification for a web-based version.
About Merge Education
Merge Education, Inc. supports programs that make a difference in the lives of vulnerable students.
Merge Education is open to acquisition conversations with organizations that want to bring SETS to more programs.
Contact: Mary Helen Rossi, Co-Founder & CEO, Merge Education, Inc. · mhrossi@merge-education.com
