by Kimberly Braxton, Director, Equal Measure
How can pathways intermediaries and their partners measure their progress as they work together to create equitable pathways?
As part of Equal Measure’s learning and evaluation engagement focused on the Building Equitable Pathways initiative (2019–2022), we developed surveys for the staff and partners of pathways intermediaries. Based on the Equitable Pathways Intermediary Framework, the surveys were designed to capture insights about intermediary activities and indicators of systems change (organizational change led by sector partners and change in the broader pathways ecosystem).
This is the second release in the series of insights and tools, Mending the Path: The Central Role of Intermediaries in Building Equitable Pathways from Education to Work, grounded in the work of intermediaries participating in the Building Equitable Pathways Community of Practice. Pathways intermediaries are forging a new system in which K–12, postsecondary, and workforce partners collaborate to create conditions that lead to more learners of color and learners experiencing poverty completing education and securing good jobs. For more about the Community of Practice, facilitated by JFF, visit the JFF website.
What’s in the Surveys?
Each survey contains a glossary of key terms (e.g., equity, learners of color, and pathways ecosystem) followed by three sections:
- Intermediary Activities
- Organizational Changes
- Ecosystem Changes
We operationalized the Equitable Pathways Intermediary Framework into the survey instruments, with framework items mapping closely to survey items.
Equitable Pathways Intermediary Framework
The survey captures key activities associated with pathways intermediaries across four domains.
Intermediary Activities:
- Data Collection and Use
- Partnership Building
- Pathways Design and Delivery Support
- Policy and Funding Advocacy
Activities reflecting equitable practice are embedded within each domain. Survey respondents are prompted to share their perspectives on the importance for the intermediary in leading each activity and the effectiveness of the intermediary in carrying out each activity using four-point scales.
The survey also captures organizational changes within partner organizations, categorized across three domains:
Organizational Changes:
- Organizational Culture, Practice, and Policy Changes
- Increases in Cross-organizational Collaboration
- Changes in Resource Allocation and Funding
Examples are provided for each question under Organizational Changes. Survey respondents are prompted to indicate the extent to which the statement describes partner organizations well.
Lastly, the survey captures changes in the broader ecosystem, categorized across two domains.
Ecosystem Changes:
- Public Policy Change
- Public Narrative Change
Examples are provided for each question under Ecosystem Changes. Survey respondents are prompted to indicate the extent to which they agree with the statement.
How Can the Surveys Be Used?
When completed by staff, intermediaries can use the Intermediary Activities section of the survey as a self-assessment tool. Executives can complete this section as a team or include a broader range of staff members to reflect on areas of strength and areas for growth and improvement. Intermediaries can draw from the sets of questions and language within the survey (e.g., Organizational Culture, Practice and Policy; Cross-organizational Collaboration) to integrate with their own surveys that they administer to partners. Intermediaries with personnel experienced with survey administration can load the survey into an online tool like SurveyMonkey or Qualtrics to administer the survey to staff and/or partners. It can also be printed and completed by hand.
Intermediaries and their partners can use both the framework and the survey data to reflect on their progress in creating more equitable pathways ecosystems, particularly areas of strength and areas for growth across the activity domains, systems changes, and learner outcomes. They can clarify, determine, and prioritize which activities the intermediary is best positioned to lead versus other partners in the collaborative. The section on systems change can prompt deeper understanding about the types of changes that sector partners across the pathways ecosystem need to achieve to advance equitable pathways. Partnerships can assess the extent to which they are capturing and monitoring learner outcomes data across the education-to-work continuum—and where they can bolster their data infrastructure to better track their impact.
What Have We Learned Through Using these Surveys?
Equal Measure administered the survey in 2021 with a set of six intermediary organizations and their partners when the survey included only items based on the activity domains. The second survey administration was in 2022 and included items based on five systems change domains in addition to the activity domains. Five sites participated in the survey in both 2021 and 2022. In total, nine unique intermediaries participated in the survey.
A few takeaways:
- The survey findings reinforced equity-oriented Data Collection and Use, Partnership Building, Pathways Design and Delivery Support, and Policy and Funding Advocacy as the core activity domains of pathways intermediaries.
- Partners provided positive feedback on the work of intermediaries through their ratings of effectiveness. Policy and Funding Advocacy was rated the most effective area of intermediaries’ work.
- Survey findings revealed that the relationship between intermediary activities and systems change—especially ecosystem change—is still emerging. While complex systems change takes time to manifest, intermediaries are finding more immediate success in shifting the organizational culture and practices of sector partners.
DOWNLOAD THE SURVEYS
Staff Survey
Partner Survey
Intermediaries and their partners can use both the framework and the survey data to reflect on their progress in creating more equitable pathways ecosystems.