Client(s):
College Possible
Catalyze Process Evaluation for College Possible
Strengthening supports for first-generation college students
In collaboration with College Possible and several of its college and university partners, we examined the challenges and successes of the Catalyze program implementation process and made recommendations to strengthen the program and improve student outcomes.
Impact
College Possible is building on its past work to increase undergraduate student success by implementing the innovative, capacity-building program, Catalyze. Our study assisted in evaluating this effort and provided actionable recommendations aimed at enhancing the quality of the implementation and the efficacy of the program.
Services
- Process Evaluation
- Data Collection and Analysis
- Participatory Research
Expanding Services Through Catalyze
College Possible has expanded its reach by partnering with institutions of higher education through the Catalyze program, which focuses on undergraduate student persistence, retention, and graduation. In the program, near-peer coaches mentor students, while campus liaisons manage the partnership and the program. The coaches, who guide and support students, are recent college graduates themselves, and many come from similar backgrounds as the students they serve.
The diversity of partner campuses’ geographical locations and institutional differences presents challenges in program replication and consistency. Through HSRI’s process evaluation, researchers produced a battery of concrete recommendations as well as a logic model of the Catalyze program to assist leaders in conceptualizing the interaction of system components and improve program delivery.
Evaluating the Program
The evaluation examined College Possible’s implementation process using a mixed-methods design and incorporating a participatory approach by engaging stakeholders in the evaluation design process. Using the lens of implementation science, the study explored the key drivers of successful implementation at all levels of the program—programmatic, organizational, and institutional—and made recommendations for resolving challenges and enhancing successes.
Program levels and feedback loops
Collecting the Data
Data collection activities included:
- An intensive review of guiding program documents
- Semi-structured qualitative interviews with key stakeholders at the organizational and institutional levels
- Online surveys to capture key Catalyze activities from the perspectives of leadership, staff, and students
- Site visits that included observation, focus groups, and staff feedback on survey findings
This multi-step, comprehensive data collection strategy ensured that our team pursued the most useful lines of inquiry and captured a cross-section of organizational and institutional perspectives.
Results
Recommendations included:
- Avoid mission drift and manage system complexity by focusing on core program elements.
- Enhance support for coaches by providing more opportunities for peer engagement, connecting coaches to other young professionals on campus, and encouraging partner institutions to offer additional campus amenities like meal plans or bus passes.
- Streamline in-house data collection and utilization.
- Integrate strengths of the program, identified through the evaluation, into Catalyze’s value proposition.