OCTOBER 2011 • NUMBER 11-04 The Youth Transition Demonstration: Interim Findings and Lessons for Program Implementation Thomas Fraker The transition to adulthood for youth especially if they were to move out of their parents’ households. any observed differences in outcomes between them can be attributed to the with disabilities can be especially dif- initiative. The evaluation is tracking ficult. In addition to the host of issues employment, earnings, and benefits, The Youth Transition facing all transition-age youth, young among other outcomes, to assess Demonstration people with disabilities face unique whether YTD helps youth find jobs and issues related to health, social isolation, Recognizing the importance of service reduces their dependency on SSI and service and support needs, and poten- intervention at this critical juncture in DI. The evaluation also includes a study tial loss of benefits. These challenges the lives of young people with disabili- of the implementation of the YTD. complicate their planning for future ties, the Social Security Administration education and work, and often lead to (SSA) initiated the Youth Transition Intervention Components poor employment outcomes, a high risk Demonstration (YTD). Focusing on of dependency on public programs, and youth who are 14 to 25 years old, SSA Because SSA wants to test strong inter- a lifetime of poverty (Davies, Rupp, and is investing considerable resources in ventions grounded in best practices, Wittenburg 2009). developing and evaluating strategies to the YTD components are based on help youth with disabilities become as Guideposts for Success, developed The public cost of dependence on dis- economically self-sufficient as possible. by the National Collaborative on ability benefits by young people is quite YTD projects around the country offer Workforce and Disability for Youth large. In December 2009, 1,066,000 transition services that are intended (NCWD/Y 2005). Guideposts was youth 13 to 25 years old were receiving to lift the barriers facing youth with informed by a review of research, Supplemental Security Income (SSI) disabilities. YTD also includes SSA demonstration projects, and effective benefits totaling nearly $7.5 billion in waivers of disability program rules that practices covering a range of programs. 2009 (Social Security Administration encourage youth to work by allowing It represents the most comprehensive 2010). An additional 196,000 indi- them to retain more of their benefits as information available on “what works” viduals age 25 and under were receiving their earnings increase. in promoting a successful transition to Social Security Disability Insurance adult life for youth with disabilities. (DI) benefits with an aggregate value YTD is being evaluated based on an in excess of $1 billion in 2009 (Social experimental design. Under this design, INTERVENTION COMPONENTS Security Administration 2011). Many youth were randomly assigned to either others are at high risk of receiving SSI or a treatment group that is eligible for • Individualized work-based DI in the future if they do not transition both the waivers and YTD services or experiences to productive adult lives. Some of these to a control group that is under standard • Youth empowerment young people have disabilities that are SSA program rules and may receive currently not severe but have a prognosis only those non-YTD services that hap- • Family involvement for decreased functioning over time. Oth- pen to be available in their communi- • System linkages ers are currently ineligible for benefits ties. Because of random assignment, • SSA waivers and benefits because of their parents’ incomes but the two groups were expected to be counseling might be eligible after reaching age 18, equivalent at baseline; consequently, Mathematica’s Center for Studying Disability Policy provides rigorous, objective disability policy research, collects data from the people disability policy aims to serve, and supplies the nation’s policy- makers with the information they need to navigate the transition to 21st-century disability policy. For more information, visit our website at www.DisabilityPolicyResearch.org. Foremost among the intervention ers expand eligibility for three additional 2007. The top panel in Table 1 provides components are individualized work- work incentives for SSI beneficiaries: the basic information about these projects. based experiences. These include plan for achieving self-support, individual YTD project services ended in the fall of volunteer work, subsidized jobs, and, development accounts, and the student 2009 in Colorado and Erie County and in most notably, competitive paid employ- earned income exclusion. the spring of 2010 in the Bronx. ment in integrated settings where YTD project staff typically deliver ser- Three additional projects participated in individuals with disabilities work vices directly to participating youth and phase two of the evaluation. They were alongside able-bodied individuals. The their families. This approach contrasts selected from a group of five projects that literature identifies competitive paid with a case management model, in which were funded by SSA through its contract employment in secondary school as project staff help participants to access with Mathematica to deliver YTD ser- the strongest predictor of post school services provided by other organizations. vices on a pilot basis in 2007. The selec- employment success (Benz, Yovanoff, Project staff engage youth in person- tion criteria included the achievement of and Doren 1997; Luecking and Fabian centered planning, provide job develop- youth recruitment targets, the strength of 2000). Youth empowerment refers to the ment and job placement services, foster services delivered and their fidelity to the acquisition of the skills and knowledge family support for transition efforts, intervention design, and the size of the that allow youth to chart their own and counsel youth and their families on target population. The projects selected courses and advocate for themselves. In issues surrounding SSA benefits. On the to fully implement their YTD interven- the context of YTD, empowerment is orther hand, project staff typically refer tions are located in Miami-Dade County, fostered primarily by engaging youth in youth to other organizations for health Florida; Montgomery County, Maryland; person-centered planning that focuses and education services. and 19 counties in West Virginia. Youth on education, employment, health care, in these locations began to enroll in the and independent living. Family involve- Another noteworthy feature of the YTD evaluation in March 2008, and the proj- ment is important because of the critical design is the technical assistance that ects are scheduled to end in March 2012. role that families play in helping youth was provided to projects. TransCen, Inc., The bottom panel in Table 1 provides to manage their disability benefits and a leader in the design and implementa- basic information about these projects. formulate plans for employment. The tion of employment interventions for intervention fosters this involvement youth with disabilities, delivered assis- through family-focused training activi- tance focused largely on helping project Enrollment of Youth in the ties, support for parent networking, and staff network with employers to identify Evaluation and YTD Services the provision of transition-related infor- paid competitive jobs and match youth In all of the evaluation sites except Mont- mation. YTD also facilitates the system with appropriate jobs. gomery County, enrollment in the evalua- linkages that youth may need to access tion was restricted to youth who were SSI health services, education programs, YTD Projects or DI beneficiaries. In these sites, Math- transportation assistance, and accommo- ematica survey interviewers conducted dations for education and employment. SSA signed cooperative agreements with extensive outreach to youth on the disabil- seven organizations in September 2003 SSA’s waivers for YTD and the benefits ity benefit rolls to enroll them in the study. to operate YTD projects in California, counseling that youth need to understand A young person was considered to have Colorado, Iowa, Maryland, Mississippi, them are also central to the intervention been enrolled upon completing a baseline and New York (one in Bronx County because they enhance five standard SSI interview and returning to Mathematica a and the other in Erie County). Two years work incentives that allow beneficiaries to signed informed consent form affirming later, SSA selected a team of contrac- retain some of their benefits while work- his or her decision to participate in the tors headed by Mathematica to conduct ing. For example, under the waivers, the evaluation. Emancipated youth could sign the random assignment evaluation and SSI earned income exclusion is $65 per the consent form themselves; otherwise, a to provide technical assistance to the month plus three-quarters of any addi- signature by a legal guardian was required. projects. The team also includes MDRC, tional earnings, whereas under standard Following enrollment in the evaluation, a nonprofit corporation that evaluates rules this exclusion is $65 plus one-half Mathematica randomly assigned youth to social welfare programs, and TransCen. of additional earnings (SSA 2011). Also a treatment or a control group. Based on information gathered through under the waivers, the consequences of visits to the seven projects, the contrac- Only in Montgomery County was a negative continuing disability review tors recommended that those in the eligibility for the evaluation restricted or age 18 medical redetermination are Bronx, Colorado, and Erie County par- to youth who had been classified by the delayed for youth enrolled in YTD, thus ticipate in the first phase of the evalua- county’s public school system as having allowing them to continue to receive cash tion. SSA accepted this recommendation, severe emotional disturbances or who and medical benefits for four years after and youth began to enroll in the evalua- were known to have been diagnosed with enrollment or until they reach age 22, tion in Colorado and the Bronx in August a significant mental illness. Approxi- whichever comes later. Finally, the waiv- 2006, and in Erie County in February mately 15 percent of them were receiv- 2 Table 1. to the baseline survey, Mathematica is PROJECTS PARTICIPATING IN THE YTD EVALUATION conducting follow-up surveys one and three years after youth entered the evalu- Sample Size ation, gathering information on service Project Location Target Treatment Cases Control and Name Lead Agency Population (YTD Participants) Cases receipt, educational attainment, employ- Phase-One Projects ment and earnings, attitudes and expecta- Bronx County, John F. Kennedy, Jr. SSI and DI 492 397 tions, and other outcomes. Administrative NY: CUNY Youth Institute for Worker beneficiaries ages (387) data on evaluation enrollees include Transition Education of the 15-19 and their monthly disability benefit amounts and Demonstration City University of families Project New York the use of SSA work incentives. Colorado (4 Colorado WIN SSI and DI 468 387 counties): Partners of the beneficiaries ages (401) Colorado Youth University of 14-25 Interim Findings from WINS Colorado Denver Phase-One Projects Erie Co., NY: Erie 1 Board of SSI and DI 459 384 Transition Cooperative beneficiaries ages (380) The YTD evaluation team is preparing WORKS Educational 16-25 six project-specific reports that present Services interim findings from the process and Phase-Two Projects impact analyses. Interim reports on the Miami-Dade Co., Abilities, Inc. of SSI and DI 460 399 phase-one projects are already available FL: Broadened Florida beneficiaries ages (388) Horizons, Brighter 16-22 at http://www.ssa.gov/disabilityresearch/ Futures interimreports.html. Interim reports on Montgomery Co., St. Luke’s House, High school 422 383 the phase-two projects are scheduled to MD: Career Inc. juniors or seniors (374) Transition with severe be released late in 2012. These reports Program emotional are based on the full data for the imple- disturbances mentation analysis and on one year of West Virginia (19 Human Resources SSI and DI 455 397 counties): West Development Foun- beneficiaries ages (388) follow-up survey data and administra- Virginia Youth dation, Inc. 15-25 tive data for the impact analysis. Key Works findings for the Colorado project and for Note: Martinez et al. (2008) provide full descriptions of the six projects participating in the YTD evaluation. the Bronx and Erie county projects are summarized below. ing disability benefits; the others were Of these, 805 completed the baseline For each phase-one project, approximately considered to be at high risk of receiving survey and were randomly assigned to a two-thirds of the treatment group youth benefits in the future, absent effective treatment or control group. The project used some type of employment service, intervention. For youth who met these staff convinced 374 of the 422 treatment from either YTD or other programs, during criteria, project staff conducted the initial group members (89 percent) to partici- the 12 months after they enrolled in the outreach, primarily through presenta- pate in the intervention. evaluation, as shown in the first row of tions to students in high school transition Table 2. This finding reflects impacts, rela- classes. Mathematica then followed up Data Sources tive to what these youth would have expe- with the youth to complete the base- rienced in the absence of YTD, ranging line interview, obtain written informed The YTD evaluation includes an analysis from 12.4 percentage points in Colorado consent, and randomly assign them to a of the implementation of the demonstra- to 16.2 percentage points in the Bronx. treatment group or to a control group. tion projects and an analysis of their Notwithstanding these positive impacts, the impacts on employment and related intensity of YTD employment services was Mathematica attempted to contact 21,774 outcomes. The implementation analysis low in two of the projects. Our analysis of youth in the 5 sites in which recruitment relies primarily on qualitative data col- ETO data found that the average amount was based on the SSA disability rolls. lected during three visits to the projects of YTD employment services received Twenty percent of these youth (4,298) by the evaluation team over a period of by participants in the Colorado and Erie enrolled in the evaluation and were ran- two years. In addition, Efforts-to-Out- projects who had actually used any such domly assigned to a treatment or control comes (ETO), the web-based manage- services was just 4.0 hours and 5.8 hours, group. The staff of the respective YTD ment information system used by the respectively. Youth in the Bronx project projects convinced 1,944 of the 2,334 YTD projects, is the source of quantita- received substantially more YTD employ- treatment group members (83 percent) to tive data on service delivery. ment services, averaging 20.2 hours. participate in their interventions. The impact analysis is based on data from Given the low intensity of YTD employ- In Montgomery County, 930 youth con- surveys of enrollees and administrative ment services received by participants in sented to participate in the evaluation. files for SSA benefit programs. In addition 3 the Colorado and Erie projects, it is not Table 2 surprising that these projects had no sta- SELECTED ONE-YEAR IMPACTS OF THE PHASE-ONE YTD PROJECTS (PERCENTAGES) tistically significant impacts on the pro- Bronx County, NY Colorado Erie County, NY portion of youth who were employed in Treatment Treatment Treatment a paid job at some point during the year Outcome Measure Mean Impact Mean Impact Mean Impact following random assignment (Table 2, Used any 68.0 16.2*** 61.7 12.4*** 66.3 13.7*** row 2). The project in the Bronx, which employment service provided more intense employment Employed in a 30.5 9.0*** 34.4 1.3 43.6 2.9 services that included direct placement paid job in paid summer jobs, had a significant Used any SSA work 16.5 9.3*** 24.5 1.9 31.7 6.9** incentive positive impact on employment of 9 Note: A regression model was used to estimate impacts while controlling for baseline differences percentage points. These results suggest between treatment and control group members. that the SSA waivers for YTD may need */**/***Impact estimate is significantly different from zero at the .10/.05/.01 level using a two-tailed t-test. to be combined with intense employment services in order to affect short-term employment outcomes. closely monitor both the delivery of these YTD participants and on employment Two of the phase-one projects had services by project staff and the employ- outcomes for these youth. The reports are positive impacts on the use of SSA work ment outcomes of project participants. discussed during monthly project-specific incentives. The final row in Table 2 teleconferences with project directors and While services such as resolving issues shows that the projects in the Bronx and managers. Those discussions center on the with SSA benefits and facilitating enroll- in Erie County had statistically signifi- intensity of employment services and on ment in education programs may promote cant impacts of 9.3 and 6.9 percentage paid employment outcomes. the well-being of YTD participants, Trans- points, respectively, on the use of any Cen reiterated to the staff of the phase- The refinements to YTD technical work incentive during the year following two projects that the YTD initiative is assistance were designed to increase the enrollment in the evaluation. It is likely about competitive paid employment and likelihood that the phase-two projects that the SSA waivers for YTD contrib- self-sufficiency. Accordingly, TransCen would provide the services envisioned uted to these impacts, but the evaluation advised the projects to emphasize job for the YTD initiative—services focused design does not allow the impacts of development—reaching out to employers on competitive paid employment. the waivers to be disentangled from the to inform them about the YTD project and These refinements are likely to result in impacts of YTD services. The Colorado to identify employment opportunities for interventions that are stronger than those project had no significant impact on the YTD participants—and job placement— that were implemented by the phase-one use of SSA work incentives. working with youth to identify their skills projects and, thus, in stronger results. and interests in order to match them with Implications of Findings for appropriate jobs. TransCen provided References Phase-Two Projects employment-focused technical assistance through workshops at annual confer- For the full list of references, go to Refinements to the technical assistance www.disabilitypolicyresearch.org/ ences attended by all YTD project staff, provided to the phase-two projects brief11_04_ref.asp. repeated visits to each project to assist were prompted by the finding that the with job development and job placement, two phase-one projects that provided For more information, contact Thomas monthly webinars on topics pertaining to Fraker at tfraker@mathematica-mpr.com. participating youth with few hours of the delivery of employment services, and employment services had no impacts on This brief was prepared with support from telephone consultation on employment paid employment during the initial post- the National Institute on Disability and issues concerning specific youth. Rehabilitation Research, U.S. Department enrollment year. TransCen’s technical Empirical monitoring of employment- of Education, through its Rehabilitation assistance for all projects had concen- Research and Training Center on Employ- trated on employment services and the focused staff efforts and actual employ- ment Policy grant to Cornell University achievement of positive employment ment outcomes for YTD participants (No. H133B040012). Mathematica Policy outcomes. However, the interim find- complements the sharpened focus of tech- Research is a subcontractor under this grant. ings for the phase-one projects revealed a nical assistance on employment. Based The contents of this brief do not necessarily need not only to sharpen the focus of the on data that project staff enter into ETO, represent the policies of the U.S. Depart- ment of Education or any other federal technical assistance on services directly the evaluation team prepares monthly agency [Edgar, 75.620 (b)]. The author is linked to paid employment but also to reports on services delivered by staff to solely responsible for all views expressed. Princeton, NJ • Ann Arbor, MI • Cambridge, MA • Chicago, IL • Oakland, CA • Washington, DC Visit our website at www.mathematica-mpr.com Mathematica® is a registered trademark of Mathematica Policy Research, Inc.