Contract Number: Addressing Teen Pregnancy HHSP23320082911YC Risks for Youth Living in Out-of- Mathematica Reference Number: Home Care: Implementing 06549.094 POWER Through Choices 2010 Submitted to: Implementation Report Office of Adolescent Health Department of Health and Human Services December 15, 2014 1101 Wootton Parkway, Suite 700 Rockville, MD 20852 Alicia Meckstroth Project Officer: Amy Farb Megan Barry (Child Trends) Betsy Keating Submitted by: Ellen Kisker (Twin Peaks Partners) Mathematica Policy Research Kristine Andrews (Child Trends) P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Brian Goesling This page is left blank for double-sided printing. CONTENTS I. INTRODUCTION .................................................................................................1 II. THE POWER THROUGH CHOICES APPROACH TO SERVING HIGH-RISK YOUTH .............................................................................................4 III. BUILDING A PROGRAM INFRASTRUCTURE: ESTABLISHING HIGH-LEVEL PARTNERSHIPS AND RECRUITING GROUP HOMES AND YOUTH........................................................................................................9 IV. PREPARING FACILITATORS TO DELIVER THE CURRICULUM AND BUILD RAPPORT WITH YOUTH ..............................................................17 V. COLLABORATING WITH GROUP HOME STAFF AND FITTING PTC INTO THE GROUP HOME CONTEXT ...............................................................25 VI. YOUTHS’ ATTENDANCE, ENGAGEMENT, AND FEEDBACK.......................... 29 VII. LOOKING FORWARD: LESSONS TO INFORM FUTURE IMPLEMENTATION ...........................................................................................33 REFERENCES .........................................................................................................................37 APPENDIX A: IMPLEMENTATION STUDY DATA SOURCES AND METHODOLOGY ................................................................................................... A.1 APPENDIX B: SUMMARY OF POWER THROUGH CHOICES LESSONS............................. B.1 APPENDIX C: KEY PROGRAM DOCUMENTS ...................................................................... C.1 iii This page is left blank for double-sided printing. TABLES III.1. Group Home Characteristics, by State ......................................................................12 III.2. Key Baseline Characteristics for PTC Participants, Total and by State ..................... 16 FIGURES I.1. POWER Through Choices Program—A Snapshot ......................................................1 I.2. PPA Evaluation of POWER Through Choices—A Snapshot .......................................2 I.3. Key Implementation Study Questions .........................................................................3 II.1. Logic Model for POWER Through Choices .................................................................5 II.2. POWER Through Choices Curriculum ........................................................................6 II.3. POWER Through Choices: Examples of Activities ......................................................7 III.1. Mission and Key Activities of Partner Oganizations (Other Than PTC) ..................... 10 III.2. State and Local Context: Existing Services and Education Outside of PTC .............. 13 III.3. Group Home Staff Perception of Need for PTC, Overall Percentage and by State .........................................................................................................................14 IV.1. Professional Backgrounds of PTC Facilitators .........................................................17 IV.2. Desired Characteristics of PTC Facilitators ...............................................................18 IV.3. Initial Training for PTC Facilitators ............................................................................19 IV.4. Youths’ Ratings of PTC Facilitators, Overall Percentage and by State ...................... 23 V.1. Group Home Staff Enthusiasm for PTC, Overall Percentage and by State ............... 28 VI.1. Attendance in PTC Sessions, Overall Percentage and by State ............................... 30 VI.2. Youths’ Feedback on the Value of PTC, Overall Percentage and by State ............... 31 v This page is left blank for double-sided printing. I. INTRODUCTION Some of the most vulnerable youth in the United States are those living apart from their families in residential care overseen by the foster care or juvenile justice systems. Many such youth have experienced abuse and neglect, face mental health and substance abuse challenges, and struggle with serious behavioral problems, all factors found to increase sexual risk-taking behavior (James et al. 2009; McGuinness et al. 2002). Youth in foster care and other out-of-home care settings are considered a priority population for efforts to prevent teen and unplanned pregnancy. Such youth report having first sexual intercourse at earlier ages and more sexual partners compared with other youth (Belenko et al. 2009; James et al. 2009; Kelly et al. 2003; McGuinness et al. 2002; Carpenter et al. 2001). Moreover, teen girls in foster care are 2.5 times more likely to become pregnant by the age of 19 and 1.5 times more likely to have a subsequent teen pregnancy than their peers outside the foster care system (Dworsky and Courtney 2010; Bilaver and Courtney 2006). Among boys in foster care, by age 21 about 50 percent reported impregnating someone compared with 19 percent of their peers not in foster care (Courtney et al. 2011). POWER Through Choices 2010 (PTC) is a comprehensive, skill-building sexual health education curriculum focused on pregnancy prevention and risk reduction and designed specifically to address the unique risks of youth in foster care and other out-of-home care settings. While the curriculum has not previously been rigorously evaluated, an earlier study reported positive effects of the curriculum on knowledge, attitudes, and intentions related to safe sex (Becker and Barth 2000). Since 2011, the Oklahoma Institute for Child Advocacy (OICA) has led PTC implementation in three sites: Oklahoma (statewide), California (two counties), and Maryland (seven counties). This PTC implementation is funded through the Personal Responsibility Education Innovative Strategies grant program by the Administration on Children, Youth, and Families, U.S. Department of Health and Human Services. Figure I.1 highlights key characteristics of the program and its implementation. Figure I.1. POWER Through Choices Program—A Snapshot • Teens in out-of-home care are at increased risk of engaging in risky sexual behaviors, becoming pregnant, Why and having repeat teen pregnancies. PTC is designed to reduce these outcomes among youth living in out-of-home care. • Participants include 44 group homes that serve youth involved with the child welfare (foster care) and/or juvenile justice systems and a total of 1,038 youth, aged 13 to 18, served by those systems. • The Oklahoma Institute for Child Advocacy (OICA) administers the program and provides leadership, training, and oversight. A local partner organization leads the implementation of the program in three sites: Who OICA in Oklahoma, the Kern County Superintendent of Schools in California, and Planned Parenthood in Maryland. • Key staff include a project director based in Oklahoma, a project coordinator in each of the three sites (California, Oklahoma, and Maryland), and two to four trained curriculum facilitators in each site. • Ten 90-minute interactive sessions are delivered by a team of two trained facilitators to single-gender groups of 8 to16 youth. • Sessions are typically delivered twice per week for 5 weeks (but sometimes once per week for 10 weeks). What • Key topics include anatomy, conception, and reproductive health; planning for the future; HIV/sexually transmitted disease prevention; identifying community resources; developing communication skills; and making choices to achieve goals • The three program sites are group homes in Oklahoma (statewide), California (Kern and San Luis Obispo counties), and Maryland (Baltimore, Carroll, Frederick, Howard, Montgomery, Prince Georges, and Where Washington counties). • The curriculum is typically delivered within the group home. In California, it is sometimes delivered at one of the partner organizations’ facilities. • Youth are enrolled on a rolling basis, as new group homes are recruited or the resident population of When participating homes turns over. 1 PTC is an updated and expanded version of the original POWER Through Choices curriculum developed in the mid-1990s by the Family Welfare Research Group (FWRG) at the University of California, Berkeley. Beginning in 2005 and concluding in 2010, the Oklahoma Institute for Child Advocacy (OICA) updated, expanded, and piloted the curriculum—now formally called POWER Through Choices 2010—with support from the National Campaign to Prevent Teen and Unplanned Pregnancy and the Centers for Disease Control and Prevention’s Promoting Science-Based Approaches to Teen Pregnancy Prevention grant program. The updated version of PTC maintains the format, goals, and interactive nature of the original curriculum. The changes and improvements made to the curriculum are described in Chapter II. PTC is currently being evaluated as part of the Evaluation of Adolescent Pregnancy Prevention Approaches (PPA) in collaboration with OICA and its local evaluator, the University of Oklahoma Health Sciences Center (OUHSC) (see Figure I.2). PPA is a national evaluation funded by the Office of Adolescent Health within the U.S. Department of Health and Human Services to study the effectiveness of seven teen pregnancy prevention approaches. The PPA evaluation, which is conducting random-assignment experiments, is intended to provide rigorous evidence about program impacts, document implementation of the program, and generate insights about the successes and challenges of delivering innovative teen pregnancy prevention programs. PTC is the only program studied through PPA that focuses on reducing pregnancy and associated sexual risk behaviors among youth living in out-of-home care. Moreover, the PPA evaluation is the first rigorous test of PTC’s effectiveness in improving behavioral outcomes. PTC is designed and appropriate for youth living in all types of out-of-home care settings; however, the implementation of PTC described in this report is exclusive to youth living in group homes overseen by the child welfare (foster care) and/or juvenile justice systems. “Out-of-home care” generally refers to the placement of a young person into a residential setting other than the youth’s family home. This placement may occur through a child welfare agency, a juvenile justice agency, or voluntary placement by a parent or guardian. Out-of-home care settings encompass a broad set of residential arrangements, among them family foster homes, kinship foster care, maternity homes, detention centers, group homes, psychiatric treatment facilities, transitional living programs, and emergency shelters. A “group home” is considered a congregate care residential facility operated or contracted by a state child welfare agency, a state juvenile justice agency, or by a private care provider. Figure I.2. PPA Evaluation of POWER Through Choices—A Snapshot • POWER Through Choices is being evaluated through the national multiyear Evaluation of Adolescent Pregnancy Prevention Approaches (PPA). The aim of PPA is to assess the effectiveness of seven teen pregnancy prevention programs. • The PPA evaluation is funded by the Office of Adolescent Health, U.S. Department of Health and Human Services and is being conducted by Mathematica Policy Research with Child Trends and Twin Peaks Partners, LLC. • The PPA team is conducting the evaluation of PTC in partnership with the University of Oklahoma Health Sciences Center (OUHSC) and the Oklahoma Institute for Child Advocacy (OICA). OUHSC is leading the random assignment process, the impact study data collection, and the surveys of group home staff. OICA is primarily responsible for recruiting group homes to the evaluation and delivering the curriculum. • PTC program impacts will be assessed using an experimental design and a cluster random assignment approach, with group home youth cohorts as the unit of random assignment. Impacts will be measured by three follow-up surveys with youth—upon program completion and both 6 and 12 months after program completion. • This report focuses on PTC implementation; the impact evaluation will be detailed in a later report. 2 The PPA evaluation is noteworthy in the scale of its research design and the complexity of study implementation. The evaluation involves the implementation of PTC in 44 group homes in Oklahoma, California, and Maryland. These group homes serve youth who are involved with the child welfare (foster care) and/or juvenile justice systems. The group homes were eligible for random assignment multiple times during the study period, as their resident population turned over, and new cohorts of youth sample members could be defined. A total of 97 cohorts of youth within the study homes were randomly assigned either to a treatment group that received PTC or a control group that did not, and included in the study sample. The sample enrollment period began in January 2012 and ended in June 2014. All youth between the ages of 13 and 18 residing in the study homes were eligible for participation. The study sample (a total of 1,038 youth, including 518 treatment group members and 520 control group members) constitutes those youth for whom consent to participate in both the program and the evaluation was provided. This report addresses several questions about the development and implementation of PTC (see Figure I.3). To address these questions, the PPA evaluation team drew on semistructured interviews, observations of program sessions, surveys of participants and staff, and program participation data. The team conducted semistructured interviews, mostly in-person with PTC staff and key stakeholders, including several group home administrators. Interviews covered the history, development, and implementation of the program. Two rounds of interviews (one in-person and a second by telephone) were conducted with the program director and the program coordinators in each of the three sites. These interviews were transcribed, and the research team used qualitative analysis software to conduct descriptive and inductive analyses. The data from program staff interviews were supplemented with observations of a PTC session in each of the three sites, program participation data on attendance, and surveys with participants and group home administrators and staff. In order to provide timely lessons to the field, this report focuses on OICA’s experiences implementing PTC during the first two-thirds of the evaluation period. Appendix A presents details about the implementation study methods and data sources. Figure I.3. Key Implementation Study Questions • How does the PTC program address the needs of youth in out-of-home care? What theories and practices support the program model? What topics and activities are included in the curriculum? • Which organizations collaborated in implementing PTC? How were they recruited into the study, and how did they, in turn, recruit group homes to participate? What are the key characteristics of the group homes and the youth the program served? • Who are the PTC facilitators? How were they recruited, trained, and supported? How did they deliver the curriculum and connect with the youth participants? • How did PTC staff collaborate with group home staff and build support for PTC within the group homes? • How did youth respond to PTC? To what extent did they participate? Were they engaged by the material? • What lessons have been learned from this study of PTC, and how are they relevant for future program implementation? 3 II. THE POWER THROUGH CHOICES APPROACH TO SERVING HIGH-RISK YOUTH POWER Through Choices 2010 (PTC) is an innovative curriculum that addresses the particular needs of youth living in out-of-home care settings. Although these youth commonly face many challenges and risk factors, they often do not receive much sexual health education (Hudson 2012; Becker and Barth 2000). PTC aims to fill this gap by providing interactive, skill- building sessions to prepare youth to recognize and make decisions related to sexual behavior, find and use local resources, and communicate effectively. POWER Through Choices addresses the needs of youth living in out-of-home care—a high-risk population with limited exposure to sex education. The PTC curriculum addresses the unique risk factors faced by youth living in out-of-home care. It is one of the only research-based sexual education curricula designed specifically for these youth. Through interactive activities, PTC addresses the youths’ strong need for affection, desire for a support network, and higher likelihood of being exposed to sexual abuse or violence, which all may lead them to engage in risky sexual behaviors (Oklahoma Institute for Child Advocacy and the University of Oklahoma National Resource Center for Youth Services 2010). The curriculum aims to provide information and build skills that may help youth in group home settings avoid risky sexual behaviors and prevent pregnancy. Youth living in out-of-home care typically have limited access to sexual health education about contraception and pregnancy prevention. Overall, research has found that teens living in foster care have relatively low levels of knowledge about contraception and reproductive health (Hudson 2012). 1 These youth commonly experience disruptions in schooling and may even be removed from public school completely, so they often do not have access to the sex education provided in public schools (Becker and Barth 2000). Moreover, youth living in group homes have reported difficulty accessing reproductive health information and services, and some studies have indicated a lack of sexual health education provided within group home settings (Freundlich 2003; Crottogini et al. 2008). This lack of access to sex education puts youth in out of home care at risk of higher rates of sexual risk-taking behavior and teen pregnancy. Whether youth enter group homes through child welfare (foster care) or the juvenile justice system, their experiences in the group home are similar. 2 Youth from both systems often reside together in the same group home. They may cross over between the foster care and juvenile justice systems and be served by both. Moreover, youth from these two systems have often experienced similar issues and challenges, such as abuse and neglect, substance abuse, mental health issues, and serious behavioral problems (National Campaign to Prevent Teen and Unplanned Pregnancy 2012). Over 130,000 teenagers nationwide are estimated to live in institutional group home settings (U.S. Department of Health and Human Services 2013; U.S. 1 As part of the evaluation’s upcoming impact analyses, information on youths’ knowledge at baseline and several points of followup will be examined. 2 Foster care is part of the child welfare system. 4 Department of Justice 2010). Group homes typically focus on providing services to address the underlying causes of the conditions that brought youth to the facility while also providing a safe environment. While the homes may vary in length of stay, level of security and control, and the array of services offered, they typically offer such services as individual and family therapy, case management, substance abuse counseling and treatment, and various types of education and training. Typically, sexual health education is not provided in group homes, largely because of scheduling constraints and limited group home staff expertise in sexual health education. We provide more information about the group homes in this evaluation in Chapter III. Based in theory, PTC aims to build goal-setting, communication, and decision making skills to help empower youth to make healthy choices for themselves. PTC anticipates that youth will be motivated, prepared, and empowered to make healthy sexual and other choices for themselves by learning about sexual health, reproduction, and contraceptive options; setting personal goals and planning for the future; and developing communication and other life skills. The logic model for PTC is grounded in four psychosocial theories of behavior—the health belief model, self-regulation theory, theory of reasoned action, and social and cognitive learning theory. The model presented in Figure II.1 summarizes PTC program components, their relation to targeted mediating factors, and key intermediate and long-term outcomes. Informed by the theories, PTC focuses on (1) participants’ understanding of pregnancy and sexually transmitted infection (STI) risks and their perceived ability to reduce these risks, (2) goal-setting and planning to achieve future goals and avoid pregnancy, (3) the impact of behavioral choices on achieving future goals, and (4) opportunities for practicing positive behaviors. Figure II.1. Logic Model for POWER Through Choices Overall, PTC aims to help youth recognize their own power through choices. By emphasizing self empowerment, the curriculum’s objectives are to enable participating youth to 5 recognize and make healthy, positive choices related to sexual behavior; build contraceptive knowledge and skills; develop and practice effective communications skills; and learn how to locate and access available resources (Oklahoma Institute for Child Advocacy and the University of Oklahoma National Resource Center for Youth Services (NRCYS) 2010). Throughout the program, PTC challenges youth to envision their future and recognize the importance of making healthy choices in order to accomplish their goals. PTC’s interactive approach engages youth through role-playing activities, group discussion, and other hands-on activities. A team of two professionally trained facilitators delivers PTC to small groups of youth. They typically delivered it to single-gender groups of 8 to 16 youth, in 10 90-minute sessions typically held twice a week for five weeks. Key topics addressed in the curriculum are female and male reproductive anatomy, STI and HIV transmission and prevention, contraceptive methods, communication styles, making choices that fit your goals and lifestyle, and reproductive health resources available to participants. Figure II.2 provides a summary of the purpose of each session. Appendix B provides additional detail about the content of each session. Figure II.2. POWER Through Choices Curriculum Session Title Purpose 1 Introduction to PTC Introduce curriculum, assess participants’ knowledge regarding pregnancy prevention and sex education, and demonstrate role playing 2 Adolescent Reproductive Increase knowledge of male and female reproductive anatomy, the Health Basics process of fertilization and conception, and the menstrual cycle 3 Creating the Future You Identify planning involved in practicing positive sexual behaviors, Want outline individual choices involved in sexual decision making, and discuss abstinence as a viable choice 4 Making Choices Clear Help participants to build assertiveness and communication skills related to sexual activity 5 Understanding STIs and Increase knowledge and understanding of STI/HIV transmission and HIV and How to Reduce prevention Your Risk 6 Increasing Contraceptive Increase knowledge about contraceptive methods Knowledge 7 Practice Makes Perfect Discuss the level of risk associated with various sexual behaviors, use role playing to demonstrate the importance of dual methods, and practice using a condom 8 Using Resources to Discuss ways to improve communication about contraception with Support Your Choices foster parents, guardians, and group home staff members and how to access local sexual and reproductive health resources 9 Making Choices That Fit Help participants to develop a plan for avoiding unwanted pregnancies Your Lifestyle and STIs, set short- and long-term goals, and identify choices needed to attain goals 10 Plan + Prepare + Practice = Reinforce themes and messages of the curriculum POWER Each PTC session engages youth in interactive activities that emphasize the importance of self-empowerment and individual choices. The curriculum uses a combination of role-playing 6 demonstrations; individual reflection and group discussion; and other interactive demonstrations, games, and activities to increase participants' knowledge of sexual and reproductive health and their decision-making and communication skills (see Figure II.3). For example, in the “Designing My Saturday Night” activity in Session 3, facilitators describe up to 6 fictional couples, many of whom live in out-of-home care settings or experience other situations familiar to youth in such settings. Participants are then asked to suggest ways that these couples can have a healthy and safe Saturday night, including what decisions about contraception should be made before and during their night out. This activity presents situations relevant to the participants’ own lives—part of the overall effort to engage them in the program. Figure II.3. POWER Through Choices: Examples of Activities “Designing My Saturday Night” “Express Yourself” “Risky Business” (Session 3 (Creating the Future (Session 4 (Making Choices (Session 7 (Practice Makes You Want), 55 minutes) Clear); 15–25 minutes) Perfect); 15–25 minutes) 1. Facilitators introduce participants 1. Facilitators ask participants to 1. Facilitators pass out four Risky to 6 fictional couples with various define passive, aggressive, and Business cards, each with a backgrounds. assertive communication styles. different risk level printed on it Facilitators explain key (No risk for pregnancy, STIs, or 2. Participants brainstorm as a HIV; Little risk for pregnancy, components of communication group ways that each couple STIs or HIV; At risk for STIs or (content of message, eye could have a healthy and safe HIV; and At risk for STIs or HIV contact, tone of voice, body Saturday night. and pregnancy). language). 3. Facilitators ask participants to 2. Facilitators read out different 2. Facilitators demonstrate the identify steps and choices that scenarios or behaviors (e.g., three communication styles. each couple needs to make to “abstinent now, but had protect themselves from STIs 3. Facilitators give each participant unprotected sex in the past”). and unwanted pregnancy. a statement to read. Each decides the style in which he or 3. Participants rank the risk level of she reads the statement. The each scenario using their Risky group votes on which Business cards. communication style was used, 4. Facilitators share the correct and discusses their reasoning. answer, and the group 4. Facilitators ask each participant discusses the explanation for to read their statement again, each correct answer. using assertive communication, to encourage participants to use assertive communication in their lives. PTC in its revised form makes several enhancements over the original curriculum. Based on feedback from facilitators and youth, OICA added two sessions (Session 2 on reproductive health and Session 5 on STIs). It also added new role-playing activities and improved upon existing activities to promote inclusivity of all teens and enhance PTC’s relevance to the group home context. For example, role-playing scenarios were added on gay and lesbian relationships, talking to group home counselors about birth control options, and calling adolescent health care providers. The revised curriculum also integrated updated data and statistics and new information on resources, and it eliminated one session from the original curriculum (“Parents: Making an Informed Choice”). 7 PTC is structured to guide program facilitators and promote consistency in implementation and fidelity to the curriculum. PTC is a structured and systematic curriculum. For each lesson, it provides detailed information and, in some cases, specific language for facilitators to use in their presentation of the material. OICA expects facilitators to adhere with high fidelity to the curriculum (see Chapter IV). To support program and group home staff in implementing PTC, OICA developed support materials including a revised manual for training PTC trainers, a fidelity monitoring tool, and outcome evaluation tools, and NRCYS developed a handbook to support program orientation for group home staff. 8 III. BUILDING A PROGRAM INFRASTRUCTURE: ESTABLISHING HIGH-LEVEL PARTNERSHIPS AND RECRUITING GROUP HOMES AND YOUTH After receiving funding to implement and test the new PTC curriculum, OICA faced the challenge of recruiting a large number of group homes to participate in a random-assignment evaluation. To ensure that the evaluation would have adequate statistical power to detect program impacts, the evaluation design called for recruiting approximately 40 group homes. To meet this goal, OICA established high-level partnerships with organizations that could help recruit group homes and implement the PTC curriculum and evaluation. OICA and its partners succeeded in recruiting 44 group homes for the evaluation. These homes serve a racially diverse group of youth with a history of risky sexual behavior. OICA recruited three partner organizations for the implementation of PTC. OICA recognized it would need to partner with several organizations to reach the scale required for the evaluation since there were not enough group homes in Oklahoma to carry out a rigorous evaluation. While OICA would lead PTC implementation in Oklahoma, relying on partners to lead implementation in other states was an efficient way to expand the scale of the overall effort. OICA selected partners that had an established relationship with the child welfare system in their state, knowing that these relationships would facilitate the recruitment of individual group homes to the evaluation. Senior leaders at OICA had many professional relationships in the child welfare and juvenile justice systems, in Oklahoma and nationwide, they could draw on in the recruitment effort. OICA drew on these connections to identify partner organizations that could help recruit group homes. OICA carefully vetted potential partners, and three organizations were selected because they all had an existing relationship to the foster care system, history and experience with sexual health education, and experience working collaboratively with other organizations in their states and communities. These three partner organizations—the Kern County Superintendent of Schools (KCSOS) in California, Planned Parenthood of Maryland (PPM), and the Illinois Caucus for Adolescent Health (ICAH)—were selected to recruit group homes and deliver the PTC curriculum in their state. 3 OICA leads the PTC implementation site in Oklahoma and provides general oversight of the cross-site implementation effort. Figure III.1 highlights characteristics of the PTC partner organizations. In addition to delivering the curriculum, these partner organizations were responsible for identifying and recruiting group homes, obtaining consent from the legally authorized representatives (LARs) of youth for the program and the evaluation, and administering baseline and follow-up surveys. 4 Each partner identified a site coordinator to oversee these activities. Program facilitators delivered the curriculum and assisted with group home recruitment. In addition, OICA funded one or two local evaluation data collectors in each site, who performed 3 As described below, OICA later discontinued its partnership with ICAH. 4 A legally authorized representative may be a state-authorized case worker or lawyer, a state agency worker, or the youth’s biological parent. 9 evaluation-related data collection tasks in collaboration with OUHSC, including obtaining consent and administering surveys. 5 Figure III.1. Mission and Key Activities of Partner Oganizations (Other Than PTC) Oklahoma: California: Maryland: Oklahoma Institute for Kern County Planned Parenthood of Child Advocacy Superintendent of Schools Maryland Mission: Mission: Mission: Create awareness, take action, As advocates for children, Ensure access to high-quality, and change policy to improve the provide leadership, education, affordable reproductive health health, safety, and well-being of and support students, school services for all; provide medical Oklahoma’s children. districts, and the community services, education, training, and through programs, services, and advocacy to help individuals Sample Activities: fiscal accountability make informed decisions about Inform and educate state reproductive health, family Sample Activities: policymakers and citizens about planning, and sexuality. the importance of investing Serve, support, and monitor local Sample Activities: wisely in children and families schools and districts Operate eight health centers to Work with Healthy Teens Assist with staff development and provide medical services and Oklahoma to prepare teen birth training support data, provide health educator trainings, and encourage Assist with offering special Support program efforts to community collaboration education, alternative education, educate youth and others about and other special-interest sexual health, protect themselves Promote Oklahoma After-School programs from STIs, prevent unintended Network pregnancies, and make healthy Help develop new curricula decisions Early in the evaluation period, OICA decided to discontinue its partnership with ICAH. When a key leader from ICAH left the organization, ICAH lost its critical connections to the foster care system in general and group home administrators in particular. OICA determined that ICAH was no longer well-positioned nor sufficiently committed to the evaluation to successfully recruit group homes and cancelled its subcontract agreement with ICAH. At the time, ICAH had not yet recruited any group homes for the evaluation. OICA and KCSOS subsequently increased the number of group homes they recruited in Oklahoma and California, respectively, in order to reach the target sample size for the evaluation. OICA provided leadership, training, guidance, and oversight across the partner organizations. It was responsible for ensuring procedural consistency and fidelity to the curriculum; developing and coordinating PTC facilitator training; and ensuring that the target number of homes was recruited for the evaluation. (We discuss facilitator training and fidelity monitoring in the next chapter.) 5 These local evaluation data collectors were separate from and operated independent of the staff who delivered the PTC program. 10 The PTC partner organizations drew on relationships in the child welfare and juvenile justice systems to recruit group homes across three states. Recruiting group homes to the study required relationship-building and collaboration. Before reaching out to individual group homes, the site coordinator from each partner organization created momentum and buy-in for PTC among the agencies and organizations (called “supervising agencies” in this report) in each state that are responsible for overseeing, supporting, operating, and/or funding group homes. This included coordinating with the child welfare and juvenile justice agencies, state and local departments of social or human services (which house these agencies), group home coalitions, state licensing boards, and private contractors that operate group homes. In each state, the partner organizations reached out to key contacts in the relevant state and local supervising agencies to market PTC and develop support for its implementation and evaluation. The supervising agencies typically recognized that the curriculum offered an important educational service to youth in foster care and juvenile justice. Moreover, implementing PTC would enable group homes, in some cases, to meet their independent living skills (ILS) requirements. These requirements are typically met through classes delivered to youth on various life skills to support their eventual transition out of institutional care and into independent, self-sufficient adult life. OICA and its partners drew on existing connections and relationships to recruit group homes for the evaluation. In Oklahoma, OICA worked closely with the NRCYS to identify contacts in the foster care community and drew on these contacts to cultivate support for PTC and the evaluation among the supervising agencies. In California, KCSOS’s existing relationships with local school districts facilitated both cooperation from the supervising agencies and recruitment of group homes. In Maryland, PPM drew on relationships with foster care agencies, group home staff, and youth that were developed through an earlier program effort. However, some group homes in Maryland were not sympathetic to PPM’s overall family planning mission and thus were reluctant to participate. PPM cultivated its relationship with the Maryland Department of Human Resources to facilitate group home recruitment, and, overall, most homes welcomed an opportunity to offer sexual health education to the youth in their care. Supervising agencies typically could not require group homes to implement PTC and participate in the evaluation, but their support, formal and informal, helped in recruitment. As one site coordinator explained, the support of the supervising agencies reassured group home administrators that participating in the PTC effort “has been blessed...so [they’re] not out on a ledge…for doing it.” OICA and its partners developed formal agreements with participating group homes. The recruitment of individual homes typically involved a meeting, possibly two, between the partner organization’s site coordinator and the group home’s lead administrator to discuss the curriculum and the various steps involved in participating in the evaluation. Then the group home administrator signed a memorandum of understanding (MOU) documenting the roles and responsibilities of both the partner organization and the home. By signing the MOU, group homes agreed to support the implementation of PTC and assist with the necessary functions of the evaluation, such as participating in staff surveys and facilitating access to youth for their surveys. For most group homes, it generally took about six months to go from an initial contact with the supervising agency to the signing of an MOU. An MOU had to be in place before sample enrollment could start. Group homes were recruited and MOUs signed on a rolling basis from Summer 2011 to 11 Fall 2013. Recruitment of the later homes in the sample proceeded more quickly. For homes recruited later, an MOU was typically signed within one or two months after the state’s partner organization made the initial contact with the home. The 44 group homes in the PTC study vary by size, type, and sponsoring agency. The 44 group homes in the study sample serve youth from both the child welfare and juvenile justice systems. In Oklahoma, the selected group homes serve youth from one system or the other, while in California and Maryland the homes typically served youth from both systems. Ultimately, a total of 44 group homes were recruited for the PTC study, including 19 in California, 15 in Oklahoma, and 10 in Maryland (see Table III.1). Table III.1. Group Home Characteristics, by State California Maryland Oklahoma Total Total Number 19 homes 10 homes 15 homes 44 homes Contracted to Serve Youth from Child welfare (CW) system 0 0 8 8 Juvenile justice (JJ) system 2 0 7 9 Both CW/JJ 17 10 0 27 Gender Serveda Male 11 5 10 26 Female 7 3 4 14 Both 1 2 1 4 Size (Number of Beds) 8 or fewer 16 5 0 21 9 to 16 1 0 14 15 17 to 31 0 4 0 4 32 or more 2 1 1 4 Primary Setting or Type Cottage home setting 17 5 0 24 Campus/dorm setting 1 4 15 18 Academy 1 1 0 2 Schooling On site 3 0 12 15 Off site (public schooling) 16 7 0 23 Both 0 3 3 6 Unsupervised Leave Allowedb Yes (contingent on approval/ 17 10 0 27 behavioral status) No (not at all) 2 0 15 17 a Homes that serve pregnant and parenting teens were excluded from the evaluation. b Excluding a pass to visit home. “Unsupervised leave” refers to leave within the community where the group home is located. Almost all group homes in the sample serve youth of a single gender. Over half (26 homes) serve males, while close to one-third (14 homes) serve females. The remaining four homes serve both males and females. In all cases, however, PTC was delivered to single-gender groups. Most homes are small, and over three-quarters (36 homes) have 16 or fewer youth at any given time. Almost half of participating group homes (24 homes) are small “cottage” homes of 8 or fewer youth. Cottage homes were free-standing small housing units in a community. A smaller number of group homes operate in campus or dormitory settings (18 homes) or are academies (2 homes) serving youth in the juvenile justice system in larger institutional care settings. 12 Schooling and unsupervised leave options for youth in the group homes also vary. Usually, youth in group care in this study attend public schools. However, most youth in the Oklahoma homes receive schooling within the group home setting. In most of the California, and all of the Maryland group homes in the study, youth can earn passes allowing them unsupervised leave within the area near the group home. In Oklahoma and in some California homes, youth were not permitted this privilege. Schooling and leave options may influence youth participation in the PTC sessions, as discussed in Chapter VI. OICA succeeded in recruiting homes to the study partly because PTC filled an important gap in services. Youth in out-of-home care often have limited access to sexual health education, as discussed in Chapter II, and those in this evaluation are not a likely exception. None of the three states in the study require that schools teach sexual health education (other than HIV/AIDS education) to all students (see Figure III.2). This reduces the likelihood that youth in group homes in these states receive such education in school, either before or during their stay at the group home. In addition, the qualitative interviews with both program staff and group home staff also suggest that very little, if any, sexual health education is provided through the group homes. In some homes, a single class session related to sex education may periodically be provided to youth by a local public health department or private organization, or on-site counselors may address sexual health issues with youth on an individual, as- needed basis. However, other than the occasional short class or discussion, sexual education targeted to youth in the evaluation’s group homes seemed rare. Figure III.2. State and Local Context: Existing Services and Education Outside of PTC Existing Services and Supports California Maryland Oklahoma State Policy on California does not require schools Maryland requires schools to Oklahoma does not require Sexual Health to teach sexual health education. It teach sexual health schools to teach sexual health Education, in brief does require education on education, though such a education. It does require HIV/AIDS and other STIs. If class can be offered as an education on HIV/AIDS. If schools teach comprehensive sex elective. The state also schools teach sex education, education, they must meet state requires HIV/AIDS education materials must be approved for content requirements for such for all students. medical accuracy. education. Group Home Very little or no sexual health Very little or no sexual health Very little, if any, sexual health Context education. In some homes, a education. In some homes, a education. In some homes, a single session related to sex single session related to sex single session related to sex education may be offered. education may be offered. education may be offered. Community The Dream Center and Coffee The Maryland Foster Youth Very few, if any, community- Services for Youth House in Bakersfield (Kern Resource Center in based services for youth in in Out-of-Home County), California serves as a Baltimore provides support out-of-home care (current or Care centralized “one-stop shop” where services and assistance to former) youth in foster and other out-of- foster and other out-of-home home care (current and former) care youth in Maryland, and can access a wide range of specifically targets youth services and assistance. transitioning out of care. STI = sexually transmitted infection. Group home staff perceived a strong need for PTC among the youth in their care. Over 9 in 10 staff from study group homes reported that they agreed with the statement that “PTC is very much needed by youth living in this home,” and nearly 7 in 10 strongly agreed (Figure III.3).The view that PTC was meeting an educational need among youth made it easier to sell the program and evaluation to group home administrators and staff during the recruitment phase. 13 Figure III.3. Group Home Staff Perception of Need for PTC, Overall Percentage and by State The PTC program is very much needed by youth living in this home. All Sites 69.0 24.3 4.8 1.9 California 68.2 25.8 6.1 Maryland 62.2 32.4 2.7 2.7 Oklahoma 72.0 20.6 4.7 2.8 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Strongly agree Somewhat agree Somewhat disagree Strongly disagree Source: Data are from the Direct Care Staff Survey, analyzed and tabulated by the University of Oklahoma Health Sciences Center (OUHSC). The survey was administered prior to the delivery of PTC in the group homes. Note: Based on a sample of 257 group home staff. Once youth leave the group home setting, other support services are available in some of the target areas. In Bakersfield, California, and Baltimore, Maryland, resource centers serve foster care and other youth transitioning out of group or other out-of-home care (Figure III.2). The Dream Center in California and the Maryland Foster Youth Resource Center are designed to ease the transition to independent living for youth in out-of-home care and to provide a range of support services, some of which may involve sexual health education and family planning support. Study youth in these states may receive these types of services once they leave the group care setting. By contrast, few community-based support services are available to youth in group or other out-of-home care settings in Oklahoma. Youth in cohorts assigned to the evaluation, including both treatment and control groups, can access any existing community and group home services available to them, some of which may address risk factors related to teen sexual activity and pregnancy. The PTC evaluation will estimate the impact of participating in PTC versus receiving any existing services already available in group homes or communities. Adhering to the evaluation’s consent requirements for group home youth required substantial efforts from PTC staff. Once homes were recruited and an MOU signed, all youth ages 13 to 18 were considered for PTC participation, contingent on obtaining consent for their participation in the program and evaluation. Group home participation in the evaluation proceeded after consent was obtained for 80 percent of the youth in the home (or a minimum of six youth). Requirements for the protection of human subjects in research mandated that the legally authorized representative (LAR) for each youth provide active consent for the youth’s participation in the evaluation. The procedures for obtaining active consent varied depending on the law and procedures in the host state. For example, the process in Oklahoma was relatively 14 streamlined since officials from the state’s Department of Human Services could complete much of the paperwork themselves. A similar process was used in Maryland. In California, in contrast, KCSOS staff had to reach out to each youth’s social worker to move through the consent process. Working with social workers to contact the appropriate LAR and obtain consent for each California participant was often a time-consuming and challenging process for PTC staff. To begin, PTC staff typically had to connect with social workers to obtain the name and contact information of the LAR for each youth. Social workers in California often have large caseloads and many competing priorities, which complicated this step. PTC staff then contacted the LAR to obtain consent. PTC staff spent considerable time and energy trying to coordinate with the busy schedules of the LARs (lawyers, probation officers or, in some cases, the biological parents) to obtain consent. In order to expedite this process, PTC staff arranged to meet LARs in places that would be convenient for the LAR to get consent documents signed. Since lawyers required verification of the youth’s willingness to participate in the program, PTC staff requested youth to obtain their lawyer’s consent signatures. KCSOS’s staff showed persistence and organizational skill in completing this process, which typically spanned a one to two-month period for a given cohort of youth. PTC serves a racially diverse group of youth with a history of risky sexual behavior. At the time of enrollment into PTC, the average study participant living in the group homes was 16 years old; with nearly 80 percent of the participants being male and over 20 percent female (see Table III.2). California served relatively more males (86 percent) than Oklahoma (77 percent) or Maryland (63 percent). Race and ethnicity also varied by state. The highest proportion of African Americans was served in Maryland (46 percent), while relatively more youth of Hispanic ethnicity were served in California (58 percent). The participants from Oklahoma were the most racially and ethnically diverse of the three states, with relatively similar proportions of youth self-reporting as Caucasian, African American, Hispanic, and Multiple Race. Risky sexual behavior was prevalent among PTC participants. Overall, 89 percent of participants reported ever having sexual intercourse, and just fewer than half (49 percent) of youth across the three states reported having practiced safe sex (either never having had sex or having used birth control the last time they had sex). Roughly one-quarter of youth in each state reported having had sex in the past three months without using any birth control. 15 Table III.2. Key Baseline Characteristics for PTC Participants, Total and by State California Maryland Oklahoma Total Demographic and Background Characteristics Mean Age (years) 16.2 16.4 16.1 16.2 Median Age Entering Group Care 14.0 14.0 14.0 14.0 Gender Male 86.4 62.9 76.6 78.8 Female 13.6 37.1 23.4 21.3 Race White/Caucasian 21.2 19.1 29.3 23.8 Black/African American 14.0 46.1 21.3 22.2 Multiple Races 11.0 27.0 29.8 20.7 American Indian/Alaskan Native 6.8 2.3 9.0 6.8 Native Hawaiian/Pacific Islander 3.0 1.1 1.1 2.0 Asian 0.9 0.0 0.0 0.4 Unknowna 43.2 4.5 9.6 24.2 Ethnicity Hispanic 57.5 18.2 21.0 37.3 Not Hispanic 42.5 81.8 79.0 62.7 Past Sexual Activity Ever Had Intercourseb 89.8 81.6 91.5 89.0 Safe Sex: Never had sex or used birth control the last time had sex 52.5 55.1 42.0 49.1 At Risk: Had sex in the last 3 months without birth control 23.7 29.2 35.9 24.5 Sample Size 236 89 188 513 Source: Youth surveys administered to youth upon program completion. The surveys were administered by the local evaluation data collectors and analyzed and tabulated by OUHSC. Data included for 513 of the 518 treatment group members in the evaluation sample. a This category likely includes participants of Hispanic ethnicity, particularly in California. Data reviewed by the Oklahoma University Health Sciences Center (OUHSC) suggest that a large fraction of participants of “Unknown” race in California are Hispanic youth who did not identify with any of the race categories. b This variable measures intercourse that was consensual in nature. That is, in the youth survey the question on intercourse was introduced with a note that said: “Questions on this survey only mean behaviors that you choose to participate in—do not count behaviors that you were forced to do against your will.” 16 IV. PREPARING FACILITATORS TO DELIVER THE CURRICULUM AND BUILD RAPPORT WITH YOUTH Hiring well-qualified facilitators to deliver the PTC curriculum to the youth participants was critical to implementation success. Facilitators came from a variety of backgrounds, but all had previous experience working with at-risk youth. OICA provided pre-service and ongoing training for facilitators, with an emphasis on hands-on practice before program delivery and fidelity to the curriculum. Facilitators needed to be able to connect with youth, manage group dynamics, present the curriculum in an engaging way, and answer questions from youth openly and clearly. To a large extent, it appears that they generally succeeded; most program youth rated their PTC facilitators very highly compared with other teachers in their lives. OICA and its partners valued hiring facilitators who had past experience working with at- risk youth. Typically, 8 to10 facilitators at any given time were delivering PTC across the three states. A team of two facilitators, typically one male and one female, delivered each PTC lesson. In California, KCSOS, and in Oklahoma, OICA initially employed two facilitators each, and then expanded to four midway through the study period after additional group homes were recruited to the study. In Maryland, PPM employed two facilitators. In all states, the site coordinator was also trained as a facilitator and served as a substitute when needed. Although all facilitators had some previous experience working with youth, they had different professional backgrounds and training (see Figure IV.1). All PTC facilitators had a bachelors’ degree (predominantly in education, counseling, or social work), along with some prior professional work experience. At least two facilitators also had a master’s degree. Three of 10 were sexual health educators by training. Figure IV.1. Professional Backgrounds of PTC Facilitators • 3 youth program coordinators or staff members (with a focus on at-risk youth) • 3 sexual health educators • 2 teachers (middle or high school) • 2 case managers for at-risk youth and families • 1 counselor in a youth group home • 1 life skills educator • 1 youth theatre educator and performer • 1 foster care parent (both group home and private home) Note: Counts reflect 10 facilitators, several of whom had multiple types of professional experiences prior to PTC. 17 Since youth living in group homes and other out-of-home care settings often come from stress-filled, challenging backgrounds, OICA and its partners hired facilitators who had experience and enthusiasm for working with at-risk youth. OICA and its partners considered it more important that facilitators have experience working with at-risk youth and a “big heart for kids” than that they have experience with sexual health education. OICA believed it was easier to teach facilitators the PTC curriculum content than it was to teach facilitators how to successfully engage youth. OICA and its partners sought staff who could connect with youth, make them feel comfortable, and encourage their active participation in the group. The most skilled facilitators were able to earn the trust and respect of youth in their group, manage group dynamics to encourage active participation by all youth, and maintain control of the classroom. Since youth in the group homes typically had a history of socio-emotional and behavioral problems, difficult and disorderly situations could arise, and skillful management of group dynamics was important. In California, facilitators were bilingual in English and Spanish since Spanish was the first language of many youth and staff in the participating group homes. Comprehensive facilitator training emphasized the value of youth empowerment and prioritized hands-on practice before program delivery. At the time they were hired, facilitators possessed many of the skills of successful facilitators (see Figure IV.2). Training was clearly important, however. The PTC training, described below, sought to further develop and cultivate these skills and help facilitators apply them to the implementation of the PTC curriculum. Figure IV.2. Desired Characteristics of PTC Facilitators • Engaging and energetic presence • Strong relationship-building skills, including the ability to o Build rapport and establish personal trust o Make youth feel comfortable o Relate to and identify with youth o Act in an empathetic, compassionate, and nonjudgmental manner o Engage youth with varying capabilities and comprehension levels • Strong classroom and time management skills, including the ability to o Establish and use ground rules o Seat group appropriately o Include all group members o Help youth focus and “leave problems at the door” o Establish consistent boundaries without being punitive o Act flexibly and creatively o Demonstrate a “thick skin” • Mastery of substantive content in PTC curriculum OICA invested heavily in training and technical assistance for PTC facilitators. Facilitators and other staff (PTC site coordinators and local evaluation data collectors) attended an initial four-day (30-hour), in-person training retreat in spring 2011 led by OICA’s PTC Project Director. (see Figure IV.3). The training emphasized the importance of fidelity to the curriculum as designed. Overall, the training, like the PTC curriculum, was engaging and interactive, identified personal strengths, and built teamwork and camaraderie among attendees through 18 icebreaker and energizer activities. At the end of the training, OICA expected that participants would be prepared to plan for and implement the curriculum effectively with youth in group care. 6 Figure IV.3. Initial Training for PTC Facilitators Key Training Components • Overview of PTC 2010 curriculum and evaluation • Group home staff orientation and direct care staff handbook • Values clarification: qualities of a good facilitator • How to answer sensitive questions posed by youth • Group management methods and practices • Sessions 1 through 10 o Review of content o Observations of curriculum delivery o “Teach-backs” (Practice facilitating curriculum, along with constructive feedback from program leaders) • Acceptable adaptations and enhancements • Brainstorming and group question and answer o Incentives, celebrations o Youth advisory groups • Guidance for staff on using “self-care” strategies to manage stress that may result from hearing about traumatic experiences of the PTC youth • Overview of evaluation components o Importance of teaching to fidelity o Facilitator role in documenting fidelity and attendance To deliver the curriculum in a manner designed to empower youth to make their own sexual and contraceptive choices, facilitators learned to incorporate a referential power approach (as opposed to a positional power approach). Referential power is the ability of a leader to influence a follower based on a high level of respect, loyalty, or admiration for the leader or desire to gain approval from that leader. Influence and collaboration, rather than command and control, generally foster referential power. To earn respect from youth, PTC facilitators were taught that everyone in the classroom (youth and staff) is equal and deserving of respect. To expand their positive influence, facilitators were encouraged to invest time and energy into building rapport with youth (described in further detail later). Role-playing and participant-centered activities created collaboration. Concepts of referential power laid the groundwork for PTC’s efforts to foster youth empowerment. During the initial training, facilitators took turns delivering parts of the curriculum and receiving feedback from project leadership and their colleagues. This “teach-back” process 6 Separate two-day data collection training was held in fall 2011, just before sample enrollment began, to train the local evaluation data collectors to administer surveys, track and retain youth participants during the study period, and assist with fidelity monitoring. All PTC staff attended both trainings, regardless of their role, to ensure that all staff understood both the PTC program and the importance of the various evaluation-related tasks. 19 provided a key opportunity for facilitators to actively learn and observe the curriculum and to both empower others and be empowered. For facilitators who joined the PTC team partway through the study period, OICA provided the same initial training, either individually or in small groups, along with opportunities for lesson teach-backs, observations by program leadership, and individualized feedback and assistance. Before delivering PTC to youth in the evaluation, the facilitators who participated in the first round of training practiced delivery of the curriculum to in-state youth not part of the evaluation. The practice sessions were designed to increase facilitators’ familiarity and comfort with the curriculum. The site coordinator in each state observed the practice sessions and provided feedback and technical assistance. To reinforce fidelity to the curriculum and strengthen overall project performance, OICA also convened three-day in-person training retreats with PTC staff from all three states in Fall 2012 and 2013. At these annual training retreats, staff shared successes and challenges in implementing PTC; discussed ideas, activities, and techniques that worked well in delivering the lessons and interacting with group homes; received training on additional topics (for example, trauma-informed care for youth in group homes); and conducted additional “teach-backs” for some of the more challenging curriculum activities. 7 During one of the follow-up retreats, facilitators also learned how to implement two educational modules that OICA developed for group home staff. (We describe these in Chapter V). OICA also periodically organized supplemental trainings and group discussions via conference call. The PTC staff across the sites felt well-prepared to deliver the curriculum as a result of the training and technical assistance they received. Detailed fidelity monitoring helped promote consistency in delivering PTC. The PTC program relies on two sets of monitoring tools to maximize fidelity to the curriculum, post-session facilitator feedback forms and the program observation form. Using the post-session facilitator feedback forms, one for each of the 10 PTC lessons, facilitators record detailed information on the completion of key activities in the lesson (Appendix C). They also record their feedback on implementation successes and challenges, as well as suggestions for further training and assistance. PTC staff also use the program observation form to assess the fidelity and quality of PTC lessons as delivered and to provide feedback to facilitators. A local evaluation data collector from each state observed and completed an observation form for one of the 10 sessions each time the curriculum was delivered. In addition, the site coordinators periodically observed lessons delivered in their state. A systematic examination of fidelity to the curriculum, which is based on an examination of the post-session facilitator feedback and program observation forms, is being conducted by OUHSC, under contract to OICA. 7 The follow-up training retreats also included local evaluation data collectors and covered issues relevant for supporting the evaluation. 20 The OICA project director and site coordinators used the information from these monitoring forms to support ongoing technical assistance and promote continual program improvement. The OICA project director regularly reviewed the completed forms, synthesized findings and lessons, and shared the results with the site coordinators in regular telephone meetings (monthly at first, then less often as the project progressed). The site coordinators, in turn, shared the information they received with facilitators during regular (usually weekly) meetings in each state. The information from the post-session facilitator feedback and program observation forms stimulated discussions of challenges and ways to improve curriculum delivery and classroom management. Over time, PTC leadership and staff reported that facilitators became more comfortable with the content of the curriculum and more skilled in implementing it. At first, facilitators focused on mastering the content of the curriculum. As they gained experience and confidence, their focus shifted somewhat toward mastering their presentation skills and improving student engagement. For example, more seasoned facilitators could focus more on “not giving the person with the loudest voice, that is most disruptive, the most power, (and) seating (youth) appropriately, (and) establishing ground rules effectively,” in addition to focusing on covering the content of the curriculum and responding to questions accurately. Use of facilitator teams strengthened program delivery and helped PTC manage transitions when staff turnover occurred. OICA and its partners tried to pair facilitators with complementary strengths and perspectives to form successful facilitator teams. Teams were coached to work together and divide the workload in delivering the lessons to take advantage of each of their strengths. In the process, facilitators often learned from each other and strengthened their own skills. For example, in one team, one facilitator had a strong biological science and health background and was very calm while the other had experience managing and teaching groups and was more dynamic. The facilitator teams almost always included both a male and a female facilitator, as OICA felt it was important to offer youth the perspectives of both men and women. Pairing facilitators strategically was also helpful when staff turnover occurred. When a new facilitator joined the PTC team, he or she was paired with an experienced facilitator, making it easier to continue implementation while the new facilitator gained experience. During the two- and-half-year study enrollment period, two facilitators from each state were replaced, most during the first half of the period. OICA monitored facilitators closely and held them to high expectations in terms of their commitment to the program and delivery of the lessons. In two cases (one each in two states), an unsuitable facilitator was fired; in the other cases, staff resigned to take other positions. When turnover in facilitators occurred, the remaining facilitator played a larger role and was usually paired with a substitute facilitator until a replacement could be hired and trained. The site coordinators in each of three sites were trained as facilitators and typically served as the back-up facilitator as needed. Facilitators used creative strategies to engage youth and build rapport. Facilitators focused on establishing positive, supportive relationships with the youth participants. Continuity of facilitators throughout the 10 PTC lessons for a given cohort helped facilitate rapport and relationship-building. For most cohorts, the two facilitators remained constant during the 10-session period. Depending on the number of youth in the home, PTC 21 groups ranged in size from 6 to 20 youth. Facilitators had to work harder to get to know and form strong relationships with youth in the larger groups. PTC staff noted that facilitators at each partner organization developed unique, effective ways of engaging youth and establishing a comfortable PTC environment. This was especially important with relatively large groups. In Maryland, for example, the facilitators used a large poster with pictures of all the youth. Upon entering a PTC session, a youth could move his or her picture into different categories indicating how he or she was feeling that day (happy, sad, excited, and so on). This allowed the facilitators to quickly gauge the mood of the group. Facilitators also encouraged youth to “leave all of their concerns and disagreements at the door” during PTC sessions. In Oklahoma, facilitators used creative ice breaker games to begin each session. This helped transition the youth from their other activities into PTC and set their mindset toward PTC. In California, facilitators used buttons with pictures and sayings that youth wore during the lessons to highlight the unique nature of each individual and to help build a sense of cohesion and unity among participants. The buttons were also used to acknowledge birthdays or special events. Each of these strategies also helped establish each youth as a unique individual. PPA evaluation staff visited a PTC session in each state and observed strengths in how facilitators related to youth. For example, one facilitator successfully used humor and pop culture references to engage participants. Another shared her own personal experiences with the youth, which seemed to foster a connection between the facilitator and participants. In all cases, PPA evaluation staff noted that the participants seemed very comfortable with the facilitators during the lessons. Facilitators fostered an open and accepting environment where youths’ questions were answered and discussed. A critical aspect of PTC for participants was the opportunity to get answers to questions. Facilitators encouraged youth to ask questions of any type, and part of every session was devoted specifically to question and answer (Q&A) time. Question boxes that were accessible to participants allowed youth to pose questions anonymously. Q&A time was an opportunity for facilitators to correct misinformation and address any previously unanswered questions. Many questions related to sexual and contraceptive practices, and many called for clarification of slang or jargon phrases. Q&A time also helped build the youths’ trust in facilitators’ honesty and foster open communication. Most facilitators reported that Q&A time was extremely popular, especially among female students, and most facilitators wished they had extra time to address questions. Facilitators strongly believed that answering questions completely, honestly, and in a nonjudgmental manner was essential to encouraging additional questions and making youth feel safe. In the words of one staff member: “…we say this is an open place where you can ask questions and we’ll answer them as honestly as we possibly can, and you should feel safe here.” The opportunity to ask questions and receive honest answers is a fairly unique combination for youth in out-of-home care, who often do not have many trusted adults in their lives. PTC conversations were very open and candid. For example, one facilitator noted: “(These are) very blunt conversations. . . . The kids are generating a lot of the questions, and they want the information so we’re just trying to implement and answer their questions honestly without judgment or being shocked.” 22 Most youth rated their PTC facilitators very highly. At the end of the PTC program, most youth rated their facilitators highly in comparison to other teachers they have had (see Figure IV.4). Overall, nearly 75 percent of participants rated their PTC facilitators as “outstanding” and an additional 14 percent rated their facilitators as “above average” in comparison to other teachers in their lives. Only a small fraction did not rate their PTC facilitators highly. Across the three sites, approximately 5 percent of participants rated their facilitators as “poor” or “below average” compared to other teachers. Overall, based on a rating of “above average” or better from youth, facilitators were successful in engaging and connecting with approximately 88 percent of the high-risk youth targeted by PTC. Figure IV.4. Youths’ Ratings of PTC Facilitators, Overall Percentage and by State Compared to other teachers you have had, how would you rate the instructor who presented the Power Through Choices program? All sites 2.9 2.5 7.0 13.7 73.9 California 2.9 2.9 4.8 10.5 79.0 Maryland 15.4 13.8 70.8 Oklahoma 4.1 2.9 6.5 17.6 68.8 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Poor Below average Average Above average Outstanding Source: Post-test survey administered to youth upon program completion. The surveys were administered by the local evaluation data collectors and analyzed and tabulated by OUHSC. Note: At the time of the analysis, responses were available for 445 youth (210 in California, 65 in Maryland, and 170 in Oklahoma), or 86 percent of the treatment group members in the evaluation sample. 23 This page is left blank for double-sided printing. V. COLLABORATING WITH GROUP HOME STAFF AND FITTING PTC INTO THE GROUP HOME CONTEXT To connect effectively with youth, PTC facilitators had to work closely with group home staff. Coordination with and assistance from group home staff were critical to schedule the PTC sessions, support the program’s objectives, encourage youths’ attendance, and complete evaluation tasks related to consent-gathering and survey data collection. As one PTC staff member explained: “You have to have a good relationship with the group homes in order to get to the kids. Period.” Good relationships and communication with group home staff were especially important when program implementation difficulties arose. Although PTC staff oriented group homes prior to program delivery about the curriculum’s structure, content, objectives, and practices, differences between PTC and group home approaches to working with youth sometimes interfered with youths’ attendance or participation. When this occurred, PTC staff communicated with group home staff to discuss the PTC approach and seek their support. To increase group home staff knowledge and further improve relationships, PTC developed two training modules for group home staff. Frequent group home staff turnover made ongoing communication essential to maintaining good relationships and implementing PTC successfully. In the end, despite some conflicts between the PTC and group home cultures, group home staff retained their original strong enthusiasm for PTC. PTC staff provided ongoing information and support to increase group home staff understanding of PTC. PTC staff communicated with group homes to increase their understanding of the curriculum and prepare the staff to reinforce PTC’s learning objectives with youth outside of the PTC sessions. In most homes, PTC staff held an orientation meeting with group home staff prior to implementation to explain the topics covered in upcoming lessons; highlight the types of questions that might arise after the lessons; and coach them on how to have honest, productive conversations with youth about sexual health topics. These conversations drew on material provided in PTC’s Direct Care Staff Handbook, a resource for PTC staff to use in their interactions with group home staff. In addition, PTC facilitators provided staff in all homes with a list of local medical and family planning resources to which they could refer youth when necessary. In Oklahoma, facilitators created eye-catching information cards with candy attached to distribute to group home staff to advertise each upcoming lesson, both as a way to remind them to encourage youth to attend and to help reinforce the curriculum’s objectives. Later, these “candy cards” were also used in the other two states. Building on the orientation for group home staff, PTC facilitators in numerous homes conducted a follow-up meeting with group home staff on reproductive health topics. As one PTC facilitator explained: “group home staffers felt ill-prepared to handle the biological and reproductive health questions that PTC sometimes prompted from youth.” These short meetings, provided on request, helped better equip group home staff to answer difficult questions from youth, and also created opportunities for ongoing communication and relationship-building. 25 Group home policies and practices sometimes interfered with PTC implementation. PTC staff occasionally had to address conflicts between the PTC approach and the more regulated and sometimes punitive group home approach to managing youth. The group home environment is typically very structured, highly scheduled, and restrictive for youth, with few opportunities for individual choice. In contrast, PTC’s approach is relaxed and informal; PTC aims to provide a safe space where youth can learn, discuss, and ask questions about sensitive topics in an open and fun environment. PTC also emphasizes individualized decision making and encourages youth to see the power they have to make their own choices related to their health, relationships, and future goals. One important conflict arose in group homes where staff remained in the classroom to monitor youths’ behavior during PTC sessions. Staff presence sometimes appeared to inhibit youth from engaging in discussions and asking questions. When this happened, PTC facilitators explained the importance of offering a private space for PTC without group home staff present. Usually group home staff were receptive to this feedback, and over time they attended fewer PTC sessions. In the few homes where staff were required to be present in the room to maintain “line of sight” supervision of youth, PTC staff asked that they sit in the back and observe without making comments or asking questions themselves. The presence of group home staff in PTC sessions sometimes led to interference with curriculum delivery. The explicit nature of some aspects of the PTC lessons and the candidness of the question and answer sessions occasionally made group home staff uncomfortable. PTC facilitators reported staff statements, such as “I can’t believe you all are talking like this to these kids.” In a few isolated incidents, group home staff threatened to “shut down the question box” if the youth asked any questions that they deemed inappropriate. In general, this type of interference was limited, and PTC staff communicated with group home staff to prevent future occurrences. Over time, as group home staff became more comfortable with the PTC curriculum and facilitators, concerns about the question box and the resulting discussions diminished. PTC participation also suffered when youths’ behavior led group home staff to punish participants by preventing them from attending PTC sessions. Since group home staff generally viewed PTC as a fun opportunity for youth, they sometimes prevented them from attending as punishment for behavior issues outside of PTC. When this occurred, PTC staff reminded group home staff that PTC was part of a research study and that youths’ attendance was important not just for the youth, but for the study as a whole. Other times, well-intentioned group home staff tried to help PTC facilitators by preventing attendance by youth who were having behavior issues on the day of the session. When this happened, facilitators would ask that the youth be allowed to come to PTC for 10 minutes to see if they could calm down and settle in. If behavior did not improve after 10 minutes, then the youth would be told to leave the session. Most of the time, however, youth were able to regulate their behavior and participate in the full PTC session. Group home staff turnover and competing priorities added to the challenges PTC facilitators faced in delivering the curriculum. PTC staff needed to be mindful of the everyday challenges facing group home staff and appreciative of their support and assistance. Group homes are a challenging work environment; some homes seemed understaffed to PTC staff, and youth living in these settings have myriad 26 needs that can make the jobs of direct care staff difficult and stressful. Adding to this, the group home staff often have limited education and training and low salaries. These challenges combine to make working in a group home a “tough, tough job,” according to PTC leadership, and frequent turnover is not uncommon. Frequent staff turnover in the group homes complicated the efforts of PTC staff to develop positive relationships with group home staff and heightened the need for ongoing communication. Because of this, PTC staff reported working strategically to develop good relationships with longer-term staff. In the words of one PTC staff member: “[At] one home a key staff (member) has been there for 16 years.… These long-term people are the ones we want to get to know, but then there are lots of people who are cycling through every 3 months. Old timers or cyclers ... not a lot in between.” Because group home staff juggle priorities and coordinate many activities for youth, PTC staff had to be both persistent and flexible to maintain their support. Group home staff coordinate, schedule, and manage various programs and opportunities, including schooling, job skills training, therapy, life skills training, tutoring, interest groups, meals, chores, recreation, appointments, court appearances, and self-help groups like Alcoholics Anonymous or Narcotics Anonymous. When asked about fitting PTC into the schedules of busy group homes, one facilitator stated “We have to be ‘on it’ with [the group home staff], which I think makes sense since [PTC is] our priority and it’s not their one and only priority.” Overall, PTC facilitators needed to be professional, courteous, and flexible in working with group home staff, as shifting home priorities sometimes led to last-minute schedule changes for individual youth or for the whole PTC class. To strengthen collaboration with group home staff, PTC leadership developed new group home staff training modules midway through the evaluation period. Recognizing a need for additional support for group home staff, midway through the evaluation period OICA developed two one-hour training modules for group home staff. These training modules are intended to make the training that group home staff receive related to PTC more systematic. The modules focus on adolescent brain development and strategies for talking with adolescents using a nonjudgmental communication style and positive reinforcement. The training modules have been used by PTC staff in several group homes where a special need or interest was identified. The additional trainings also allowed group home staff an opportunity to share youths’ feedback on the PTC lessons with the facilitators. In addition, they were helpful when group home staff turnover occurred because they gave PTC facilitators the opportunity to meet new staff and orient them to PTC. Since these modules were developed after the evaluation began, they took place in less than one-quarter of the homes in the study during the study period. However, future replications of PTC can benefit from these training modules. Overall, group home staff expressed enthusiasm for PTC. According to PTC facilitators, as group home staff became more familiar with the PTC facilitators and curriculum, their comfort with and commitment to PTC seemed to increase. Support and enthusiasm for PTC from group home staff were important because they could translate into youth enthusiasm and participation. PTC staff noted that when group home staff were enthusiastic, they spoke positively about PTC and encouraged youth to attend the sessions. 27 Staff survey data show that most group home staff across the three states expressed enthusiasm for PTC, both before and after curriculum delivery. Before curriculum delivery, group home staff recognized the importance of the PTC program and expressed enthusiasm about hosting it. However, as described earlier, group home policies, competing priorities, and differences in approach between group homes and PTC made delivery of the curriculum challenging at times. Still, after PTC was delivered in a home, over 9 in 10 group home staff either strongly or somewhat agreed with the statement that “I’m enthusiastic about the PTC program in this home” (see Figure V.1). More than half strongly agreed, and one-third somewhat agreed. These percentages were similar to those reported before curriculum delivery. The continued high staff enthusiasm, in spite of implementation challenges, likely reflects the importance of the PTC curriculum for the youth in the homes as well as the relationship- building, training, and support efforts of the PTC staff. Figure V.1. Group Home Staff Enthusiasm for PTC, Overall Percentage and by State I’m enthusiastic about the PTC program in this home. All Sites 58.6 34.3 6.2 1.0 California 57.6 30.3 12.1 Maryland 56.8 32.4 5.4 5.4 Oklahoma 59.8 37.4 2.8 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Strongly agree Somewhat agree Somewhat disagree Strongly disagree Source: Data are from the Direct Care Staff Survey, analyzed and tabulated by the University of Oklahoma Health Sciences Center (OUHSC). The survey was administered after the delivery of PTC in the group homes. Note: Based on a sample of 210 group home staff. 28 VI. YOUTHS’ ATTENDANCE, ENGAGEMENT, AND FEEDBACK PTC uses an interactive curriculum that encourages youth participation and engagement; however, the curriculum is a great time commitment for youth, with ten 90-minute sessions. Since youth in group home care have highly scheduled and structured lives, ensuring that they could attend all 10 sessions was challenging. Overall, however, participants generally attended the majority of sessions; and participants, group home staff, and PTC program staff alike all reported that youth valued the program and its messages. Despite serious scheduling constraints, four-fifths of youth attended at least 8 of 10 PTC sessions. Scheduling PTC so that youth were available for all ten 90-minute sessions was a significant challenge. The sessions had to fit into the highly structured, busy schedules of youth in group care. Teenagers, in general, have large time commitments to school, jobs, and extracurricular activities. In addition, youth in group homes are often required to attend weekly counseling and basic living and social skills classes as well as maintain appointments with social workers, lawyers, and/or probation officers. Some youth may also have the opportunity to spend the weekend away from the group home on home visits or take other approved, unsupervised leave. To address scheduling constraints and minimize the burden on group home staff and participants, PTC staff typically delivered the program in the group home and worked closely with each home to identify the best schedule for that location. 8 Group home administrators reported that scheduling sessions was not always easy; but facilitators worked with them to find the most convenient time for their staff and participants, including weekday evenings, Friday afternoons, and (less commonly) Saturday mornings. Most of the group homes treated PTC as a mandatory activity for their youth. This approach, along with careful scheduling, promoted high levels of attendance. A majority of youth (66 percent) across the three states attended all 10 sessions, and 83 percent attended at least 8 sessions. Youth missed sessions for various approved reasons, such as a doctor’s appointment, court date, or other scheduling conflicts. Unapproved absences, while they did occur, were less common. In some homes, group home staff used attendance at the PTC sessions as a reward for good behavior and barred youth with poor behavior or other infractions from attending. Although facilitators asked group home staff to stop this practice, it may have lowered the attendance rate, particularly early in the study. With one important exception, facilitators did not make up sessions if participants missed them. The exception was Session 2, on the reproductive health system and anatomy. If youth missed Session 2, facilitators provided a full make-up session since this session is essential for understanding all subsequent content in the PTC curriculum. When youth missed other sessions, facilitators left 8 In California, staff from some group homes transported participants to the office of the Kern County Superintendent of Schools for implementation. 29 handouts or other materials so that youth would still receive some of the main messages from that session. Attendance in PTC varied by state. In California, 80 percent of youth attended all 10 sessions, compared with 60 percent in Oklahoma and 42 percent in Maryland (see Figure VI.1). All the study homes in Maryland allowed unsupervised leave for youth (contingent on approval); these approved leaves may have contributed to the higher absence rate in Maryland. Moreover, early in the evaluation, the first cohort of homes in Maryland, in contrast to those in California and Oklahoma, scheduled PTC over a 10-week period, rather than the preferred 5-week period. While PTC was easier for group homes to schedule over a 10-week period, PTC staff generally found that youth became more engaged in the program when it was offered twice a week over 5 weeks. As a result, attendance tended to be higher in that format than the 10-week format. After the first cohort, PTC staff scheduled the remaining cohorts over a 5-week period in all three states. Maryland’s PTC staff also noted that as the evaluation progressed, they increased their emphasis on coordinating with group home staff to ensure high attendance at all sessions. Figure VI.1. Attendance in PTC Sessions, Overall Percentage and by State 100 96 89 91 80 87 83 80 79 80 71 Percentage 60 66 60 40 42 20 0 All Sites California Maryland Oklahoma Attended all Attended at least Attended at least 10 sessions 8 sessions 6 sessions Source: Attendance log data collected by program staff at the end of each PTC session, and analyzed and tabulated by OUHSC. Note: Based on a total sample size of 518 (235 in California, 95 in Maryland, and 188 in Oklahoma). The sample represents 100 percent of the evaluation’s treatment group members. PTC’s interactive format engaged youth as participants; however, there were some challenges. Facilitators and group home administrators noted that participants were generally very interested and engaged in the topics covered in the curriculum and wanted to learn more. As described in Chapter II, active role-playing and other hands-on activities engaged youth in the material. The PPA evaluation observers noted that the PTC sessions were very interactive, and facilitators kept participants engaged through a quick pace and a variety of activities. Observers in all three states noted that participants seemed comfortable with the facilitators, had good rapport with them, and were eager to ask questions to gain further information. 30 Although participation was high overall, some youth did not engage in all activities. Facilitators reported that participants varied in their comfort with the topic of sex, and sometimes youth did not want to talk about sex at all, which limited their participation in some discussions and activities. Facilitators also noted that some youth, particularly boys, were uncomfortable with the role playing. The role-playing scenarios required youth to act with their same-gender classmates, and in keeping with PTC’s inclusive philosophy, some role playing features same- sex couples. The facilitators reported that some participants held homophobic attitudes and were especially reluctant to do these role-playing activities. The length of the PTC sessions sometimes made it difficult for youth to remain engaged in program activities. Several group home administrators reported that youth occasionally struggled to remain attentive and engaged throughout the entire session (sessions are intended to last 90 minutes, but many homes scheduled the session over a two-hour period to allow additional time for transitions and Q&A). Some participants had low comprehension levels and poor retention skills, which may have led to disengagement with the material during the long sessions. Additionally, a few group home administrators noted that some youth became overwhelmed with the number of activities they were required to attend each week at the group home, so occasionally would get frustrated with attending and participating in PTC. This frustration could lead the youth to not fully engage with the material or not attend the session at all. A great majority of youth viewed the program as valuable. On the whole, participants believed PTC sessions provided valuable, useful information that was applicable to their lives. Both group home administrators and program staff noted that most participants were unlikely to receive comprehensive sexual health education outside of PTC, and believed that youth appreciated receiving this information. Indeed, the youth themselves reported that the material presented through the program was beneficial to them. As Figure VI.2 shows, approximately 87 percent of participants across the three states reported that PTC was either “very helpful” or “extremely helpful” to them. Just two percent of youth reported that the program was not helpful to them. Figure VI.2. Youth Feedback on the Value of PTC, Overall Percentage and by State How helpful do you feel the material presented in POWER Through Choices has been to you personally? All Sites 2.2 11.1 29.4 57.3 California 1.9 6.5 27.4 64.2 Maryland 2.3 21.8 31.0 44.8 Oklahoma 2.6 11.5 30.7 55.2 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Not helpful Neutral Very helpful Extremely helpful Source: Post-test survey administered to youth upon program completion. The surveys were administered by the local evaluation data collectors and analyzed and tabulated by OUHSC. Note: At the time of the analysis, responses were available for 494 youth (215 in California, 87 in Maryland, and 192 in Oklahoma), or 95 percent of the treatment group members in the evaluation sample. 31 This page is left blank for double-sided printing. VII. LOOKING FORWARD: LESSONS TO INFORM FUTURE IMPLEMENTATION POWER Through Choices aims to educate and prepare high-risk youth to make good decisions for themselves based on the power of their own choices. The program teaches youth about sexual health, reproduction, contraceptive options, and healthy relationships. It also seeks to develop youths’ goal-setting, planning, communication, and other life skills to prepare and empower them to make healthy sexual and other choices for themselves as they enter adulthood. PTC appears to fill a gap in sexual health education and related support for youth in the foster care and juvenile justice systems. Given the success of the PTC implementation, the experience can provide guidance for future replications of PTC in other places. Overall, during the period of implementation examined in this report, PTC staff in Oklahoma, California, and Maryland appeared to implement the program model and curriculum largely as planned. 9 PTC was led and delivered by a program director, site coordinators, and facilitators with a genuine passion for and commitment to the program and the youth it serves. Moreover, the PTC partnership benefitted from the project director’s strong, proactive, and well-coordinated leadership, along with OICA’s emphasis on thorough and ongoing training and technical assistance for staff. In addition, participation data show that PTC sessions were well-attended—over four-fifths of youth attended at least 8 of 10 sessions. The feedback from youth about the program was also positive—over 8.5 in 10 youth reported that the program was either “very helpful” or “extremely helpful” to them. The assessment of program implementation highlights several lessons, below, which are related to interacting with high-risk youth; working within an institutional, group home setting; and delivering the PTC curriculum. Working with high-risk youth like those in the foster care and juvenile justice systems requires an inclusive curriculum that will actively engage and interest participants. The PTC curriculum was carefully designed to be relevant to the lives of high-risk youth living in group homes and other out-of-home care settings. Prior to the PPA evaluation, OICA revised the original PTC curriculum and pilot-tested it with youth in out-of-home care to ensure that the lessons and activities were inclusive of and relevant to them. The PTC role playing and other activities use life experiences that youth living in out-of home-care encounter. This makes the curriculum relevant for these youth, helping them see its purpose and encouraging their attendance and engagement. The process of revising the curriculum and pilot testing it with youth was informed by insights from a youth advisory board comprised of youth and young adults who had previously lived in an out-of-home care setting. While PTC’s main messages and format remain useful for other groups of youth, the specific examples and role playing are particularly relevant for youth in out-of-home care. Anyone considering using PTC with a different population may wish to adapt the role-playing and other activities to mirror the life experiences of the new target population. 9 The detailed and systematic examination of fidelity to the curriculum, to be conducted by OUHSC based on the post-session facilitator feedback and program observation forms, will provide an important addition to the overall analysis of PTC implementation. 33 PTC facilitators worked to create an open, supportive environment in which youth could feel comfortable enough to participate actively. Active engagement, according to the program's theoretical foundation, is the path by which youth can gain knowledge and skills applicable to their lives. PTC staff found that one fairly simple way to foster active engagement was to schedule the lessons twice a week over 5 weeks rather than once a week over 10 weeks. Staff found that it was easier to build rapport within the group when they met more often across a shorter timeframe, and this rapport helped to facilitate youths’ engagement in the lessons. PTC facilitators found that many youth were particularly interested in the question and answer (Q&A) time, when they could raise the frankest of questions or concerns. Programs that serve high-risk youth should recognize the importance of setting aside a specific time for participants to ask questions in an inclusive, nonjudgmental environment where they can be assured that their questions will be answered openly and honestly. Given the value of Q&A time, some PTC staff suggested that the program might incorporate more time for it. This suggestion, however, is complicated by other feedback suggesting that the 90-minute lessons were already too long to hold the attention of some youth and keep them engaged. Given this, PTC and others might consider two options: (1) providing separate, additional time for discussion and Q&A, either individually or group-based; and/or (2) spreading existing lessons across more days with shorter durations while allowing additional time for Q&A (for example, twelve or thirteen 75-minute sessions, rather than ten 90-minute sessions). In any case, additional time for follow-up discussion and Q&A time may be an adaptation worth considering for subsequent implementations of PTC. Organizations that wish to serve youth living in group care settings will benefit from guidance on building relationships and collaborating with group homes. Group homes in the child welfare and juvenile justice systems provide a suitable and convenient setting to reach and educate youth who are at high risk of unintended teen pregnancy and its related risks. With a PTC program and staffing infrastructure in place, once PTC reaches an implementation agreement with a new group home and develops a solid, collaborative relationship with its staff, the time required for trained PTC facilitators to deliver the curriculum is relatively modest (15 hours over five weeks). Moreover, once an agreement with a home is established, an upfront investment in relationship development could pay off quickly if PTC is offered to new cohorts of youth on an ongoing basis. Additionally, the highly structured and scheduled environment typical of group homes is beneficial since, with careful planning and coordination, it may translate to relatively high session attendance rates. Also, the group homes general treatment of PTC as a mandatory activity helps to promote high attendance. An upfront investment in relationship-building with youth-serving agencies is an essential first step toward providing programming in group homes. At the outset, OICA and its partners invested a lot of their time in networking and building relationships with administrators from the state-level foster care and juvenile justice systems. When OICA and its partners reached out to stakeholders and administrators within these systems, they generally found that PTC was recognized as an important educational opportunity for youth living in group homes. In particular, the support of the administrators of the supervising agencies—those that oversee individual group homes—was essential to securing the eventual support and cooperation of the homes. PTC’s success depends on the acceptance of and support from the group homes. 34 Winning that support, however, takes a concerted effort. Fitting a youth-centered program like PTC into a group home setting requires overcoming a clash between the program’s tolerant, empowering tone and the restrictive and highly regulated environment of the typical group home. PTC emphasizes individual decision making, future planning, and the empowerment that youth can develop from making choices for themselves. PTC provides messages about healthy relationships, communication, and planning that youth in group homes might not otherwise hear. In contrast, many homes impose a structure that limits choices for these youth, and staff may not be accustomed to open conversations with youth about sexual health. Since PTC relies on creating a safe space where youth can comfortably discuss sexual health, its approach can be at odds with the typical group home environment. PTC staff overcame such differences in culture by developing relationships with group home administrators and frontline staff. As facilitators delivered PTC repeatedly in the same home, the group home staff became more comfortable with and accepting of the PTC messages and approach. By investing early in building relationships with group home staff, outside organizations can pave the way for smooth implementation of youth programming in this setting. PTC’s Direct Care Staff Handbook is a useful supplement that could be implemented into the curriculum for future replications. Such a handbook provides structured, step-by-step guidance and materials on collaborating with and integrating programming into group homes. The handbook could be expanded to include a formal orientation training for group home staff that would incorporate training on adolescent development and communication strategies using the modules developed midway through this study. The handbook could also be enhanced with specific lessons learned from implementation experiences in Oklahoma, California, and Maryland. Adherence to at least some of the guidelines in the handbook could be specified as a required element of participation in future PTC curriculum implementation efforts. Enthusiasm for and experience with at-risk youth can be the top-priority criteria for hiring facilitators, as long as they are given thorough training and technical assistance. The PTC program hired experienced professionals from diverse backgrounds as facilitators, but hiring decisions were driven most of all by the priority given to finding people who demonstrated passion for and experience working with at-risk youth. That passion and experience are important in all youth programs, but are especially critical for programs serving youth in out-of-home care, who do not have many trusted adults in their lives. Many of PTC’s youth participants have experienced abuse or neglect, been involved in the criminal justice system, and come from disadvantaged neighborhoods—all of which are linked to early and risky sexual behaviors. Successful PTC implementation requires facilitators who are able to connect with youth, build their trust, and keep their attention. The continuous program improvement and heavy investment in training and technical assistance for PTC facilitators appear to have been important for implementation success. The initial four-day training and follow-up three-day training retreats supported facilitators in several ways. They developed facilitators’ content expertise in sexual and reproductive health topics, guided them through the process of delivering the curriculum with fidelity, and prepared them for collaborating with group home staff. “Teach-back” sessions were valuable in strengthening facilitators comfort and skills by providing practice implementing the curriculum, followed by suggestions for improvement. The training also emphasized program messaging, helping to foster and reinforce facilitators’ support for and commitment to the PTC themes and approach. 35 Training supported facilitators in managing challenging classroom dynamics and drawing on their personal strengths to engage and empower participants. Along with the training retreats, the ongoing fidelity monitoring process allowed for regular observations of facilitators as they delivered lessons, which were coupled with follow-up suggestions for improvement. Any program delivered in small, dispersed group homes is likely to rely on facilitators who can operate professionally and independently. Hiring professional staff and training them well is therefore critical. The facilitator teams were required to function quite independently and exercise judgment and collegiality as they moved around to a variety of settings to deliver the PTC curriculum. Their job required them to be self-directed, resourceful, skilled, and very reliable and organized in cultivating relationships with group homes and delivering the curriculum to youth. Any future efforts to hire staff for PTC implementation should include careful selection and training of facilitators to ensure they are well prepared for the diverse tasks they will encounter. 36 REFERENCES Becker, M. G., and R. P. Barth. “POWER Through Choices: The Development of a Sexuality Education Curriculum for Youths in Out-of-Home Care.” Child Welfare, vol. 79, no. 3, 2000, pp. 269–282. Belenko, S., R. Dembo, M. Rollie, K. Childs, and C. Salvatore. “Detecting, Preventing, and Treating Sexually Transmitted Diseases among Adolescent Arrestees: An Unmet Public Health Need.” American Journal of Public Health, vol. 99, no. 6, 2009, pp. 1032–1041. Bilaver, L.A., and M.E. Courtney. “Foster Care Youth.” Science Says #27. Washington, DC: The National Campaign to Prevent Teen Pregnancy, 2006. Carpenter, S. C., R. B. Clyman, A. J. Davidson, and J. F. Steiner. “The Association of Foster Care Or Kinship Care with Adolescent Sexual Behavior and First Pregnancy.” Pediatrics, vol. 108, no. 3, 2001, pp. e46–e46. Courtney, Mark E., A. Dworsky, J. Hook, A. Brown, and others. “Midwest Evaluation of the Adult Functioning of Former Foster Youth.” Chicago: University of Chicago at Chapin Hall, 2011. Crottogini, J., J. Fajardo, E. Villasenor, and K. Ward. “Understanding the Sexual Health Needs of Bay Area Foster and Former Foster Care Youth: A Collaborative Project Between Health Initiatives for Youth (HIFY) and San Francisco State University. San Francisco State University, 2008. Dworsky, A. and M. E. Courtney. “The Risk of Teenage Pregnancy among Transitioning Foster Youth: Implications for Extending State Care Beyond Age 18.” Children and Youth Services Review, vol. 32, no. 10, 2010, pp. 1351–1356. Freundlich, M. Time Running Out: Teens in Foster Care. New York: Children’s Rights, 2003. Hudson, A. L. “Where Do Youth in Foster Care Receive Information About Preventing Unplanned Pregnancy and Sexually Transmitted Infection?” Journal of Pediatric Nursing, vol. 27, no. 5, 2012, pp. 443–450. James, S., S. B. Montgomery, L. K. Leslie, and J. Zhang. “Sexual Risk Behaviors among Youth in the Child Welfare System.” Children and Youth Services Review, vol. 31, no. 9, 2009, pp. 990–1000. Kelly, P. J., J. D. Champion, J. Morgan-Kidd, and R. Wood. “Sexuality Knowledge, Attitudes, and Practices of Young Women in the Juvenile Justice System.” Pediatric Nursing, vol. 29, no. 4, 2003, pp. 271–275. McGuinness, T. M., M. Mason, G. Tolbert, and C. DeFontaine. “Becoming Responsible Teens: Promoting the Health of Adolescents in Foster Care.” Journal of the American Psychiatric Nurses Association, vol. 8, no. 3, 2002, pp. 92–98. 37 National Campaign to Prevent Teen and Unplanned Pregnancy. Preventing Teen Pregnancy Through Outreach and Engagement: Tips for Working with Foster Care and Juvenile Justice. Washington, DC: 2012. Oklahoma Institute for Child Advocacy and the University of Oklahoma National Resource Center for Youth Services. “POWER Through Choices: Sexuality Education for Youth in Foster and Group Care.” Oklahoma City, Oklahoma: OICA and NRCYS, 2010. Scientific Software Development. “Atlas.ti: Visual Qualitative Data Analysis Management Model Building in Education Research and Business.” Berlin, Germany: SSD, 1997. Smith, K., and S. Colman. “Evaluation of Adolescent Pregnancy Prevention Approaches: Design of the Impact Study.” Princeton, NJ: Mathematica Policy Research, 2012. U.S. Department of Health and Human Services. “The AFCARS Report: Preliminary Estimates for Fiscal Year 2012.” Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau, 2013. U.S. Department of Justice. “Juveniles in Corrections: Custody Data (1997–Present),” Washington, DC: U.S. Department of Justice, Office of Juvenile Justice and Delinquency Prevention, 2010. Available at http://www.ojjdp.gov/ojstatbb/corrections. Accessed on September 26, 2013. 38 APPENDIX A IMPLEMENTATION STUDY DATA SOURCES AND METHODOLOGY This page is left blank for double-sided printing. The findings from this implementation study were drawn from four data sources: (1) semistructured interviews with staff and stakeholders, conducted in person and by telephone, (2) observations of PTC sessions, (3) program attendance data, and (4) surveys of program participants and group home staff. The implementation study data were collected by members of the PPA evaluation team, in collaboration with the Oklahoma Institute for Child Advocacy (OICA) and its local evaluator, the University of Oklahoma Health Sciences Center (OUHSC). The PPA evaluation team collected the first two types of data, PTC facilitators collected the attendance data, and local evaluation data collectors affiliated with OICA and OUHSC administered the participant and group home staff surveys. OUHSC compiled and tabulated the program attendance data and survey data. Semistructured Interviews with Staff and Stakeholders The primary source of data for this report was semistructured qualitative interviews conducted with PTC site coordinators and facilitators, group home administrators, and other stakeholders during the annual PTC staff training meeting in Oklahoma City, Oklahoma, in September 2012 and telephone interviews in spring 2013. A team of two PPA qualitative researchers completed the interviews. Interview respondents and timing. To provide a broad picture of PTC program development and implementation, we identified five groups of people to interview: (1) centralized program leadership (the OICA program director), (2) site coordinators from each state, (3) facilitators in each state, (4) group home administrators, and (5) community stakeholders (people from other programs or service providers that worked with youth in out-of- home care). Overall, we interviewed 20 program staff and stakeholders, some individually and others in small groups. 10 The program director and site coordinators were interviewed twice: once in Oklahoma City in September 2012 and a second time by telephone in spring 2013. The interviews typically lasted between 60 and 90 minutes. The annual PTC staff training in Oklahoma City in September 2012 provided a convenient location for the PPA research team to meet with people from the three study states. Most of the key staff were present at the training, and more than half the interviews were conducted at this time. The PPA site visit team also observed parts of the staff training. The remaining interviews were conducted by telephone during spring 2013. During these interviews, researchers obtained updates on aspects of program implementation, filled gaps in understanding of the program and its operations, and interviewed new staff and community stakeholders who were not present at the training in Oklahoma City. Key topics. Researchers developed a semistructured interview protocol for each group of respondents. Semistructured interviews were appropriate because they provide a basic structure 10 We also interviewed a local evaluation data collector from each state, to understand the role these staff played in gathering consent and administering surveys. These local evaluation data collectors were separate from and operated independent of the staff who delivered the PTC program. A.3 for discussing key topics and also allow for follow-up probes and questions that arise from interview responses. Researchers based PTC implementation interview protocols on a broad PPA implementation study master protocol previously approved by the Office of Management and Budget and PPA’s Institutional Review Board. The questions were tailored to the PTC implementation study based on specific features of the program. Protocols included questions on the topics highlighted in Table 1. Table 1. Key Interview Topics Topic Description Respondent Background PTC staff member or community stakeholder’s professional background and experiences working with youth in out-of-home care Program Overview and PTC goals and key activities Components Program Context The states and localities in which PTC is being implemented and information on the development of PTC Program Counterfactual Existing education and services for youth in out-of-home care in the three states and relevant localities Population Served The youth in out-of-home care served by PTC (or, for stakeholders, information on the youth in out-of-home care served by their programs) Training PTC training provided to staff Group Home Recruitment Recruitment of group homes and program participants, as well as coordination with and Coordination group home administrators and staff Youth Attendance and Youth attendance and engagement in program sessions, as well as facilitator rapport Rapport with youth participants Program Implementation On-the-ground program implementation and monitoring, including quality and fidelity of implementation Lessons Overall experiences implementing PTC, including successes and challenges Analysis approach. Qualitative analysis of the semistructured interview data involved an iterative process using thematic analysis and triangulation of data sources (Patton 2002; Ritchie and Spencer 2002). Because of the number of interviews conducted, we used a qualitative analysis software package, Atlas.ti (Scientific Software Development 1997), to help in organizing and synthesizing the data. First, we developed a coding scheme for the analysis, organized according to key topic. The initial coding scheme was based on a general scheme used across PPA sites, with overarching “parent” topic codes and more specific subcodes. That coding scheme was refined for PTC using an iterative approach. Specifically, when data collection ended, PPA staff engaged in a collective and individual process of open coding. Open coding is an inductive approach to qualitative analysis that allows unanticipated codes or themes to emerge. During this process, PPA staff coded a portion of the notes using Atlas.ti, identifying codes that did not work for PTC and those that needed to be adapted. The resulting refined coding scheme supported a systematic process of analysis and interpretation. Second, we applied the refined codes to passages in all the interview notes to facilitate data analyses. To ensure accurate and consistent coding, two analysts independently coded site visit data, and two researchers (both members of the site visit team) reviewed the coded documents and reconciled any differences in coding. To address the research questions, we used the Atlas.ti A.4 software to retrieve relevant passages in the qualitative data, analyze the patterns of responses across respondents, and identify emergent and important themes. Program Observations Observations of the delivery of PTC lessons, although limited by design, enhanced the PPA research team’s understanding of the program. A researcher on the PPA team and two research assistants from Mathematica observed one PTC session in each state during spring and summer 2013. These observations were intended to provide a point-in-time “snapshot” example of what PTC implementation looked like in the three study states. The PPA observers completed a program observation form for the session they observed. This is the form used regularly by the local evaluation data collectors to collect data on the fidelity and quality of PTC program delivery (Appendix C). The form provides instructions and guidelines on how to rate the facilitators on dimensions of implementation. It also describes the important characteristics of “excellent” versus “poorly delivered” sessions. In addition to completing this form, the observers participated in a debriefing discussion with one of the PPA researchers who conducted the semistructured interviews. The observers appraised the comfort level of facilitators and youth participants with the PTC material, as well as the types of interactions between facilitators and youth. The observers also provided open-ended comments on the strengths and challenges of the session observed. Separate from the PPA observations, OUHSC plans to conduct a systematic examination of the observational data collected by the local evaluation data collectors as part of PTC’s fidelity monitoring process. The local evaluation data collectors observed and completed an observation form for one of the 10 PTC sessions each time the curriculum was delivered. OUHSC plans to tabulate and examine these data. Program Attendance Data PTC staff in all three sites used uniform log sheets to track the attendance of each participant in each of the 10 sessions. These data were reviewed and tabulated by OUHSC and provided to the PPA evaluation team. The attendance data presented in this report represent 100 percent of the PPA evaluation’s sample of treatment group members. Survey Data Population served by PTC. Data on the population served by the PTC program were gathered from a baseline survey administered to program participants by the local evaluation data collectors at the time of their enrollment into PTC. The survey instrument included questions on demographic and background characteristics; risk-taking behavior; previous receipt of sex education; knowledge, attitudes, and intentions toward sexual activity; sexual activity and contraceptive use; and pregnancy and sexually transmitted infections. The survey was administered to consented youth on a rolling basis, beginning in early 2012. The data in the report are based on 513 youth who participated in PTC (or 99 percent of the evaluation’s sample of treatment group members.) A.5 Participant experiences in PTC. Qualitative data on participant experiences in PTC were gathered as part of a post-test survey administered by the local evaluation data collectors to participants upon their completion of the program. The survey collected information on how helpful participants found the PTC program, as well as participants’ opinions about the quality of the PTC instruction. The data were analyzed and tabulated by OUHSC. The data in the report represent the responses of over 85 percent of the total number of treatment group members. Group home staff views on, and perceptions of, PTC. Qualitative data on group home staff views on, and perceptions of, the PTC program were collected as part of the “Direct Care Staff Survey.” This survey was administered to frontline group home staff by the local evaluation data collectors both before and after the delivery of PTC in a group home. The survey collected information on staff views about the value and usefulness of PTC for youth living in the home, as well as information on staff communication and teamwork within the home. A total of 257 group home staff responses were included in the analysis, representing over three-quarters of all such staff working in the study homes. OUHSC plans to conduct a complete analysis of the group home staff survey data as part of future reporting on PTC program experiences in California, Maryland, and Oklahoma. Additional information on the study’s group homes, including context and staff turnover, was collected through a short pre- and post-program survey of the group home administrators. OUHSC will also analyze and report on these data. A.6 APPENDIX B SUMMARY OF POWER THROUGH CHOICES LESSONS This page is left blank for double-sided printing. Table B.1. Summary of POWER Through Choices Lessons Session Name Objectives Materials Activities Length 1 Introduction to • Learn about the content and Flip chart, chalkboard, or Introductions and Ground Rules 15–20 minutes PTC 2010 themes of the curriculum. white board • Understand the reasons for Markers Discussion of the Program 15 minutes setting ground rules. Masking tape Reasons Why 15–20 minutes • Understand the uniqueness Anonymous Question Box a. Worksheet 1A – Reasons of this program for youth in Index cards (two different Why out-of-home care. colors) b. Worksheet 1B – More • Identify reasons some teens Pencils or pens Reasons Why choose to have sex or to Candy, spoons, and cups abstain (not have sex). (optional) Introduce and Demonstrate Role 20 minutes • Practice communication skills Participant worksheets a. Role Play 1C - Friendly through role play. Advice – Demonstration Practice Role Playing 10 minutes a. Role Play 1D - Friendly Advice – Practice Session Wrap-Up: Discussion 10 minutes 2 Adolescent • Learn the name and Flip chart, chalkboard, or 1. Review of Session 1 and 10 minutes Reproductive functions of male and female whiteboard Introduction of Session 2 B.3 Health Basics reproductive anatomy. Markers • Understand the process of Masking tape Male Reproductive Anatomy 20–25 minutes fertilization and conception. Anonymous Question Box a. Worksheet 2A – Male • Learn about the female Reproductive Anatomy Index cards (two different menstrual cycle. colors) Female Reproductive Anatomy 20–25 minutes Pencils or pens a. Worksheet 2C – External Overhead projector or Female Reproductive computer/projector Anatomy Overhead transparencies or b. Worksheet 2E – Internal PowerPoint slides of Female Reproductive worksheets Anatomy Participant worksheets 10–15 minutes Fertilization and Conception Menstrual Cycle a. Worksheet 2G – The 15–20 minutes Menstrual Cycle Wrap-Up of Session 2 10 minutes Session Name Objectives Materials Activities Length 3 Creating the • Identify the planning involved Flip chart, chalkboard, or 1. Review of Session 2 20 minutes Future You in practicing healthy sexual whiteboard Want behavior. 2. Designing My Saturday Night 55 minutes • Markers • Outline some of the individual a. Couple Profiles 1–6 • Masking tape choices in the sexual • Anonymous Question Box Session Wrap-Up: Discussion 15 minutes decision-making process. • Index cards (two different • Recognize abstinence as a colors) viable option and choice. • Ground rules (posted) • Pencils or pens • Posters 4 Making Choices • Practice the important Flip chart, chalkboard, or 1. Review of Session 10 minutes Clear elements of assertive whiteboard 3/Introduction communication and 15–25 minutes • Markers distinguish between 2. Express Yourself • Masking tape assertive, passive, and a. Statement Sheets • Anonymous Question Box 15 minutes aggressive communication Anniversary Night • Index cards (two different styles. a. Role Play 4A – Anniversary colors) • Demonstrate (through role Night • Ground rules (posted) 15–25 minutes playing) knowledge of the • Pencils or pens It Takes Two reasons why it is important to • Participant worksheets B.4 use a condom and other a. Worksheet 4B – It Takes forms of protection to protect Two against HIV and other STIs. b. Discussion Questions: It • Demonstrate (through role Takes Two 15–20 minutes playing) condom use negotiation. Talking It Out 10 minutes • Identify techniques for a. Role Play 4C – Talking It effective communication with Out a partner. Session Wrap-Up: Discussion • Identify how miscommunication can lead to potentially dangerous situations. • Demonstrate (through role playing) effective communication techniques with a foster parent, guardian, or group home staff member regarding contraception. Session Name Objectives Materials Activities Length 5 Understanding • Identify the most common Flip chart, chalkboard, or 1. Review of Session 5 minutes STIs and HIV STIs, related symptoms, whiteboard 4/Introduction and How to outcomes, and treatment. • Markers Reduce Your • Gain a clearer understanding • Masking tape 2. STI PowerPoint presentation: 55–65 minutes Risk of the examination process “STIs: How Much Do You • Anonymous Question Box Know?” for STI checks and/or pelvic • Index cards (two different colors) exams. a. Worksheet 5A: STIs: How • Ground rules (posted) • Demonstrate basic Much Do You Know? • Pencils or pens knowledge of HIV and STI • Computer/projector 3. Condom Demonstration 10 minutes transmission and prevention, with an emphasis on • STI PowerPoint presentation a. Do’s and Don’ts for Correct condoms. • Speculum for explanation of Condom Usage • Demonstrate the steps in female STI check 4. Session Wrap-Up: Discussion 10 minutes correct condom usage. • Urethral swab for explanation of male STI check • Condoms/lubricants (various brands/types) • Moist towelettes • Penis model • Participant worksheets 6 Increasing • Become familiar with various Flip chart, chalkboard, or 1. Review of Session 5/Introduction 10 minutes B.5 Contraceptive methods of contraception. whiteboard Knowledge • Demonstrate a basic 2. Contraceptive Advertisements 40–45 minutes • Markers understanding of how the a. Worksheet 6A – • Masking tape various contraceptive Contraceptive • Anonymous Question Box Advertisement Information methods function. • Index cards (two different colors) • Ground rules (posted) 3. Doing It Right 20–25 minutes • Pencils or pens 4. Session Wrap-Up: Discussion 10 minutes • Paper bags • Contraceptive methods (as many as available) • Pamphlets explaining contraceptive methods • Newsprint • Miscellaneous craft supplies for decorating advertisements • Penis models • Moist towelettes • Condoms • Participant worksheets Session Name Objectives Materials Activities Length 7 Practice Makes • Identify the degree of risk Flip chart, chalkboard, or 1. Review of Session 6/Introduction 10 minutes Perfect associated with various whiteboard sexual behaviors. 2. Risky Business 15-25 minutes • Markers/chalk • Demonstrate (through role a. Sexual Behaviors/Activities • Masking tape playing) knowledge of the and Risks • Anonymous Question Box reasons why it is important to • Index cards (two different b. Risky Business Cards use a condom in addition to colors) other forms of protection to 3. Lunchtime Conversation • Pencils or pens 20 minutes help prevent HIV and other a. Role Play 7A - Lunchtime • Ground rules (posted) Conversation STIs. • Condoms • Experience, through • Penis models 4. Condom Comfort 15–20 minutes simulation, the effects of drug and alcohol use, as well as • Moist towelettes 5. Condom Race 10–15 minutes darkness, on effective • Stop watch contraceptive use. • Blindfolds 6. Finding Adult Resources 10 minutes • Participant worksheets a. Worksheet 7B - Finding • Gain a realistic Adult Resources understanding of the amount of time required to put on a 7. Session Wrap-Up: Discussion 10 minutes condom. 8 Using • Identify adults who can serve Flip chart, chalkboard, or 1. Review of Session 7/Introduction 10 minutes as resources in locating B.6 Resources to whiteboard Support Your information regarding sexual 2. Finding Adult Resources Review 15–25 minutes • Markers Choices health issues. • Masking tape 3. Adolescent Health Care 40–55 minutes • Identify at least one resource • Anonymous Question Box Providers: What Can They Do in their area that provides • Index cards (two different for You? free or low-cost adolescent colors) sexual and reproductive a. Worksheet 8A – Adolescent • Pencils or pens Health Providers health services. • Ground rules (posted) • Identify at least one resource b. Handout 8C – Going to an • Computers with internet access in their area where they can Adolescent Health Care or phone books, transit maps, obtain free or low-cost Provider: Feeling and/or local clinic brochures for contraceptives. Empowered! Your Rights identifying/locating health care • Learn the steps involved in resources accessing a local family 4. Session Wrap-Up: Discussion 10 minutes • Participant worksheets planning resource. • Identify teen rights in accessing family planning resources. Session Name Objectives Materials Activities Length 9 Making Choices • Demonstrate an Flip chart, chalkboard, or 1. Review of Session 8/Introduction 10 minutes That Fit Your understanding of how whiteboard Lifestyle personal lifestyle affects 2. You Decide 25–35 minutes • Markers/chalk contraceptive choices. • Masking tape 3. How Will You Avoid 15–20 minutes • Develop a plan for avoiding • Anonymous Question Box Pregnancy/STIs? an unplanned pregnancy. • Index cards (two different • Develop a plan for avoiding 4. Going for My Goals Session 20–30 minutes colors) STIs. • Pencils or pens 5. Wrap-Up: Discussion 10 minutes • Identify a short-term or long- • Ground rules (posted) term goal. • Photographs to represent each • Identify the series of choices character in the You Decide that must be made to attain a activity goal. • Participant worksheets 10 Plan + Prepare • Develop a plan to protect Flip chart, chalkboard, or 1. Review of Session 9/Introduction 10 minutes + Practice = oneself from an unplanned whiteboard POWER pregnancy, from HIV, and 2. Steps to Protection 30–40 minutes • Markers/chalk from other STIs. a. Worksheet 10A – The • Masking tape • Review the key concepts Steps to Protection • Anonymous Question Box presented in the curriculum. • Index cards (two different 3. Sex Bingo 20–25 minutes • Identify skills and information colors) B.7 learned from the curriculum. 4. Curriculum Wrap-Up: Discussion 20 minutes • Pencils or pens • Ground rules (posted) • Tokens for Bingo Cards (poker chips, beans, etc.) • Certificates of Completion • Optional: Bingo prizes • Participant worksheets This page is left blank for double-sided printing. APPENDIX C KEY PROGRAM DOCUMENTS This page is left blank for double-sided printing. Post Session Facilitator Feedback: Session 1 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): Complete: ____/____/____ ____ Minutes Introduction ___:___ AM/PM (circle one) No. Participants: Session Location: Facilitator Background: Male: Health Educator Social Worker Foster Parent Group Home Staff Other ________________________ Female: Session 1 Completeness of Coverage * Reasons: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Introductions and Ground Rules 2. Discussion About the Program 3. Reasons Why 4. Introducing & Demonstrating Role Playing 5. Practicing Role Playing 6. Session Wrap-Up: Option 1 Option 2 Other:_____________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 2 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): Complete: ____/____/____ _____ Minutes Adolescent Reproductive ___:___ AM/PM (circle one) No. Participants: Session Location: Facilitator Background: Health Basics Health Educator Social Worker Foster Parent Male: Group Home Staff Other ________________________ Female: Session 2 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session I/Intro to 2 2. Male Reproductive Anatomy 3. Female Reproductive Anatomy 4. Fertilization and Conception 5. Menstrual Cycle 6. Session Wrap-Up: Option 1 Option 2 Other:_______________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 3 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Complete: Group Home: Facilitator Name(s): ____/____/____ _____ Minutes Creating the Future ___:___ AM/PM (circle one) No. Participants: Session Location: Facilitator Background: You Want Male: Health Educator Social Worker Foster Parent Group Home Staff Other _______________________ Female: Session 3 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). Topics/Activities 1 2 3 4 5 You may write the number(s) corresponding with the list of reasons below. If for a reason not included on list, please explain: 1. Review of Session 2 2. Intro to 3 3. *Designing My Saturday Night: Carlos/Tanya Stephanie/Rodney Thomas/Janice Myeshia/Frank Jeff/Danny Jessica/Meredith 4. Session Wrap-Up: Option 1 Option 2 Other:______________________________ *Please note for Topic #3: Covering 3 vignettes is considered 100 % complete for this area. * Completeness ratings: Possible reason(s) topic/activity not fully covered: A. Not enough time 1 = activity or topic presented exactly as written in manual B. Group interest in topic minimal 2 = made minor changes which did not alter the content of the activity or topic C. More time spent on other topics/activities 3 = made major changes which altered the content of the activity or topic D. Facilitator instructions not clear 4 = mentioned the activity or topic, but spent no meaningful time on it E. Not familiar with topic/activity 5 = did not present the activity or topic F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ ___________________________________________________________________________________________________________________________ _______________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 4 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): Complete: ____/____/____ _____ Minutes Making Choices Clear ___:___ AM/PM (circle one) No. Participants: Session Location: Facilitator Background: Male: Health Educator Social Worker Foster Parent Group Home Staff Other ________________________ Female: Session 4 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 3/Intro to 4 2. Express Yourself 3. Anniversary Night 4. It Takes Two 5. Talking It Out 6. Session Wrap-Up: Option 1 Option 2 Other:___________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities D. Facilitator instructions not clear 4 = mentioned the activity or topic, but spent no meaningful time on it E. Not familiar with topic/activity 5 = did not present the activity or topic F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 5 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): ____/____/____ Complete: Understanding _____ Minutes STIs and HIV & ___:___ AM/PM (circle one) How to Reduce No. Participants: Session Location: Facilitator Background: Your Risk Health Educator Social Worker Foster Parent Male: Group Home Staff Other ______________________ Female: Session 5 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 4/Intro to 5 2. STI PowerPoint and “STIs: How Much Do You Know?” 3. Condom Facts, Issues, and Demonstration 4. Session Wrap-Up: Option 1 Option 2 Other:_________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Do you feel overall the session objectives were achieved through the activities covered? Yes No Revised April 2012 POWER Through Choices 2010 – all rights reserved If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 6 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): Complete: ____/____/____ Increasing _____ Minutes Contraceptive ___:___ AM/PM (circle one) Knowledge No. Participants: Session Location: Facilitator Background: Male: Health Educator Social Worker Foster Parent Group Home Staff Other ________________________ Female: Session 6 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 5/Intro to 6 2.Contraception Advertisements 3.Condom Practice “Doing It Right” 4. Session Wrap-Up: Option 1 Option 2 Other:____________________________ *Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 7 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): ____/____/____ Complete: _____ Minutes Practice Makes ___:___ AM/PM (circle one) Perfect No. Participants: Session Location: Facilitator Background: Male: Health Educator Social Worker Foster Parent Group Home Staff Other _____________________ Female: Session 7 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 6/Intro to 7 2. Risky Business 3. Lunch Time Conversation 4. Condom Comfort 5. Condom Race 6. Finding Adult Resources 7. Session Wrap-Up: Option 1 Option 2 Other:_____________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 8 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Group Home: Facilitator Name(s): Complete: Using Resources ____/____/____ _____ Minutes to Support Your ___:___ AM/PM (circle one) No. Participants: Session Location: Facilitator Background: Choices Male: Health Educator Social Worker Foster Parent Group Home Staff Other _____________________ Female: Session 8 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 7/Intro to 8 2. Review “Finding Adult Resources” 3. Adolescent Health Care Providers: What can they do for you? 4. Going to an Adolescent Health Care Provider: Feeling Empowered! Your Rights 5. Session Wrap-Up: Option 1 Option 2 Other:_____________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 9 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Complete: Group Home: Facilitator Name(s): ____/____/____ _____ Minutes Making Facilitator Background: Choices That No. Participants: ___:___ AM/PM (circle one) Session Location: Health Educator Social Worker Foster Parent Fit Your Male: Group Home Staff Other _____________________ Lifestyle Female: Session 9 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 8/Intro to 9 2. *You Decide: Stephanie Christina Andre Sonya and Ryan Carlos and Tanya Chris and Terry 3. How Will You Avoid Pregnancy/STIs? 4. Going for My Goals 5. Session Wrap-Up: Option 1 Option 2 Other:______________________________ *Please note for Topic #2: Covering 3 vignettes is considered 100 % complete for this area. * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Do you feel overall the session objectives were achieved through the activities covered? Yes No Revised April 2012 POWER Through Choices 2010 – all rights reserved If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Post Session Facilitator Feedback: Session 10 Instructions: Please complete both pages of this form following completion of each session. All facilitators must complete a separate form. Date/Time of Session: Total time to Complete: Group Home: Facilitator Name(s): _____ Minutes Plan + ____/____/____ Facilitator Background: Prepare + No. Participants: Session Location: Health Educator Social Worker Foster Parent Practice = ___:___ AM/PM (circle one) Male: Group Home Staff Other _____________________ POWER Female: Session 10 Completeness of Coverage * Reason: If topic/activity not completely covered, identify reason(s). You may write the number(s) corresponding with the list of reasons Topics/Activities 1 2 3 4 5 below. If for a reason not included on list, please explain: 1. Review of Session 9/Intro to 10 2. Steps to Protection 3. Sex Bingo 4. Curriculum Wrap-Up: Review 5. Curriculum Wrap-Up: Questions? 6. Curriculum Wrap-Up: Congratulations! 7. Curriculum Wrap-Up: Certificate Other:_____________________________ * Completeness ratings: Possible reason(s) topic/activity not fully covered: 1 = activity or topic presented exactly as written in manual A. Not enough time 2 = made minor changes which did not alter the content of the activity or topic B. Group interest in topic minimal 3 = made major changes which altered the content of the activity or topic C. More time spent on other topics/activities 4 = mentioned the activity or topic, but spent no meaningful time on it D. Facilitator instructions not clear 5 = did not present the activity or topic E. Not familiar with topic/activity F. Not comfortable delivering topic/activity material Revised April 2012 POWER Through Choices 2010 – all rights reserved Do you feel overall the session objectives were achieved through the activities covered? Yes No If no, why? ____________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ How would you describe your preparation to deliver the material? Very well prepared Moderately Prepared Poorly Prepared If not fully prepared, what could be done to improve preparation?________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Do you feel that additional training would have benefited you? Yes No If yes, in what area(s) would you have liked more training? _____________________________________________________________________________ _____________________________________________________________________________________________________________________________ Comments/recommendations regarding this session (What went well? What did not go well? What, if any, changes did you make to the curriculum?) _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________ Revised April 2012 POWER Through Choices 2010 – all rights reserved Program Observation Form POWER Through Choices 2010 Oklahoma Institute for Child Advocacy and University of Oklahoma Health Sciences Center Introduction: The purpose of the observation form is to measure the fidelity and quality of implementation of the program delivery. Please use the guidelines below when completing the observation form and do not change the scoring provided; for example, do not circle multiple answers or score a 1.5 rather than a 1 or a 2. You should complete the observation form after viewing the entire session, but you should read through the questions prior to the observation. It is also helpful to take notes during your viewing; for example, for Question 1, each time an implementer gives explanations, place a checkmark next to the appropriate rating. One form should be completed for each implementer. Instructions: The following questions assess the overall quality of the program session and delivery of the information. Use your best judgment and do not circle more than one response. 1. In general, how clear were the program implementer’s explanations of activities? 1 2 3 4 5 Not clear Somewhat clear Very clear 1 - Most participants do not understand instructions and cannot proceed; many questions asked. 3 - About half of the group understands, while the other half ask questions for clarification. 5 - 90-100% of the participants begin and complete the activity/discussion with no hesitation and no questions. 2. To what extent did the implementer keep track of time during the session and activities? 1 2 3 4 5 Not on time Some loss of time Well on time 1- Implementer does not have time to complete the material (particularly at the end of the session); regularly allows discussions to drag on (e.g., participants seem bored or begin discussing non-related issues in small groups). 3 - Misses a few points; sometimes allows discussions to drag on. 5 - Completes all content of the session; completes activities and discussions in a timely manner (using the suggested time limitations in the program manual, if available). 3. To what extent did the presentation of materials seem rushed or hurried? 1 2 3 4 5 Very rushed Somewhat rushed Not rushed 1- Implementer doesn’t allow time for discussion; doesn’t have time for examples; tells participants they are in a hurry; body language suggests stress or hurry. 3 - Some deletion of discussion/activities; sometimes states but does not explain material. 5 - Does not rush participants or speech but still completes all the materials; appears relaxed. 4. To what extent did the participants appear to understand the material? 1 2 3 4 5 Little understanding Some understanding Good understanding Use your best judgment based on participant conversations and feedback. Roughly: 1 - Less than 25% seem to understand; 3 - About half; 5 - 75-100% understand. 5. How actively did the group members participate in discussions and activities? 1 2 3 4 5 Little participation Some participation Active participation Use your best judgment based on listening to the discussions and feedback. Roughly, 1 - Less than 25% participate; 3 - About half participate. 5 - 75-100% participate 6. On the following scale, rate the implementer on the following qualities: a) Knowledge of the program 1 2 3 4 5 Poor Average Excellent 1 - Cannot answer questions, mispronounces names; reads from the manual. 5 - Provides information above and beyond what’s in the manual; seems very familiar with the concepts and answers questions with ease. b) Level of enthusiasm 1 2 3 4 5 Poor Average Excellent 1 - Presents information in a dry and boring way; lacks personal connection to material; appears “burned out.” 5 - Makes clear that the program is a great opportunity; gets participants talking and excited; outgoing. c) Poise and confidence 1 2 3 4 5 Poor Average Excellent 1 - Appears nervous or hurried; does not have good eye contact. 5 - Does not hesitate in addressing concerns. Well organized, not nervous. d) Rapport and communication with participants 1 2 3 4 5 Poor Average Excellent 1 – Doesn’t remember names; does not “connect” with participants; acts distant or unfriendly. 5 - Gets participants talking and excited; very friendly; uses people’s names when appropriate; seems to understand the community and its needs. e) Effectively addressed questions/concerns 1 2 3 4 5 Poor Average Excellent 1 - Engages in “power struggles”; responds negatively to comments; gives inaccurate information; doesn’t direct participants elsewhere for further info. 5 - Answers questions of fact with information, questions of value with validation; if doesn’t know the answer, is honest about it and directs them elsewhere. 7. Rate the overall quality of the program session. 1 2 3 4 5 Poor Average Excellent Summary measure of all the preceding questions. Assesses both the extent of material covered and the performance of the implementer. Excellent sessions looks like: • Participants are doing rather than talking about activities • Non-judgmental responses to questions • Answering questions of fact with information, questions of value with validation • Good time management and well organized • Adequate pacing—not too fast and did not drag • Using effective checks for understanding. Poor sessions look like: • Lecture-style of presenting the content • Reading the content from the notebook • Stumbling along with the content and failing to make connections to what has been discussed previously or what participants are contributing. • Uninvolved participants • Getting into power struggles with participants about the content. • Judgmental responses • Flat affect and boring style • Unorganized and random • Loses track of time. Note: The following questions (8, 9, and 10) are for grantee’s internal use only for program improvement purposes. These questions are optional and will not be reported to OAH or ACYF for performance measurement purposes. 8. Briefly describe any implementation problems you noticed, including any major changes to the content or delivery of the material; time wasted in getting the session started or finished, etc: _____________________________________________________________________________________________ _____________________________________________________________________________________________ _____________________________________________________________________________________________ _____________________________________________________________________________________________ 9. Please note at least one major strength of the session and/or facilitator’s delivery of the material: _____________________________________________________________________________________________ _____________________________________________________________________________________________ _____________________________________________________________________________________________ _____________________________________________________________________________________________ 10. Other Comments: Use the space below for additional comments regarding strengths or weaknesses of the session, particularly if there is anything that affected your ratings above. _____________________________________________________________________________________________ _____________________________________________________________________________________________ _____________________________________________________________________________________________ _________________________________________________________________________________ This page is left blank for double-sided printing.