The HPOG National Implementation Evaluation (NIE) focuses on how HPOG programs are implemented, changes to service delivery systems, and individual outcomes.
Abt Associates conducted the HPOG National Implementation Evaluation (NIE) in partnership with the Urban Institute under contract to the Office of Planning, Research and Evaluation within the Administration for Children and Families (ACF), from 2012 through 2017. The study focuses on the 27 non-tribal Health Profession Opportunity Grants (HPOG) grantees, and addresses three major research questions:
- How are health professions training programs being implemented across the grantee sites?
- What changes to the service delivery system are associated with program implementation?
- What individual-level outputs and outcomes occur—e.g., recruitment, enrollment, retention, completion (accreditation/certification), job entry, employment retention and advancement, and earnings?
To address these research questions, the NIE team completed the following major tasks:
- Developed a detailed evaluation design report. The report describes the HPOG program logic model and the conceptual framework for the NIE, the project’s data collection strategy, detailed research questions, and plans for three related analyses: a Descriptive Implementation Study, a Systems Change Analysis, and an Outcome Study.
- Developed study instruments and administered surveys to HPOG grantees, management and staff, stakeholders and employers.
- Administered a survey to HPOG participants not included in the HPOG 1.0 or Pathways for Advancing Careers and Education (PACE) studies. The evaluation explores outcomes related to: time spent acquiring academic and vocational skills; certificate, degree, license, and credential receipt; employment; earnings; job quality (benefits offered, desired hours, career options, etc.); additional (post-program) training, and self-sufficiency.
- Prepared reports that describe key findings as well as evaluation methods used and study limitations. Reports include one describing interim outcomes, one describing the Descriptive Implementation and Outcome studies, one describing the Systems Change Analysis, and a final report combining results from all three analyses with the addition of outcomes from the participant survey.
- Conducted a special analysis addressing the question “How, and to what extent, did HPOG grantees use performance measurement information to change their programs?”
Timeline for HPOG NIE
Abt Associates began the Health Profession Opportunity Grants (HPOG) National Implementation Evaluation in 2012. The purpose of the project was to conduct a National Implementation Evaluation of the HPOG Program that assesses implementation, systems change, and outcomes and provides valuable information about the operations of these federally funded programs in improving education and employment opportunities for TANF recipients and other low-income individuals. This project was also designed to yield information and lessons about operating such programs and the challenges faced and addressed during implementation and throughout the operation of the program. The HPOG NIE concluded in 2017 and a final report was submitted to ACF in spring 2017.
HPOG Grantees Participating in HPOG NIE Study
- Alamo Community College District and University Health System
- Bergen Community College
- Buffalo and Erie County WDC
- Central Community College
- Central Susquehanna Intermediate Unit
- Community Action Project of Tulsa County, Inc.
- Eastern Gateway Community College
- Edmonds Community College
- Full Employment Council
- Gateway Community and Technical College (KY)
- Gateway Technical College (WI)
- Kansas Department of Commerce
- Milwaukee Area WIB
- New Hampshire Office of Minority Health
- Pensacola State College
- Pima County Community College
- Research Foundation of CUNY-Hostos Community College
- San Diego Workforce Partnership
- Schenectady County Community College
- South Carolina Department of Social Services
- Southland Health Care Forum, Inc.
- Suffolk County Department of Labor
- Temple University, Center for Social Policy
- The WorkPlace
- WDC of Seattle – King County
- WIB SDA-83 Inc. (LA)
- Will County WIB
HPOG NIE Partners and Sponsors
Primary Evaluation Sponsor
The PACE evaluation is funded by The Office of Planning, Research and Evaluation (OPRE). OPRE is the principal advisor to the Assistant Secretary for Children and Families, U.S. Department of Health and Human Services, on increasing the effectiveness and efficiency of programs designed to improve the economic and social well-being of children and families.
Abt Associates is a mission-driven, global leader in research and program implementation in the fields of health, social and environmental policy, and international development. Known for its rigorous approach to solving complex challenges, Abt Associates is regularly ranked as one of the top 20 global research firms and one of the top 40 international development innovators. The company has multiple offices in the U.S. and program offices in more than 40 countries.
- General HPOG page on ACF website
- NIE page on ACF website
- Health Profession Opportunity Grants (HPOG) Impact Study, 2011-2015
- Supporting HHS’s Health Profession Opportunity Grants Program
- Inter-university Consortium for Political and Social Research: Access to Public Use Data
Information for HPOG NIE Participants
HPOG NIE Project Director
Federal Project Officers
Employment & Training Research Team Lead
Senior Social Science Research Analyst
Administration for Children and Families
Social Science Research Analyst
Administration for Children and Families
Other ACF-Sponsored Studies
The Pathways for Advancing Careers and Education (PACE) project is a rigorous evaluation of next-generation education and training strategies for increasing the economic self-sufficiency of low-income individuals and families.