Meta-Analysis of Federally Funded Teen Pregnancy Prevention Programs: Technical Supplement November 2019 Randall Juras and Meredith Kelsey Abt Associates Mark Lipsey and Katarzyna Steinka-Fry Vanderbilt University Jean Layzer Belmont Research Associates Submitted to: Amy Farb The Office of Population Affairs Lisa Trivits and Sarah Oberlander The Office of the Assistant Secretary for Planning and Evaluation Tia Brown, Jessica Johnson, and Kenyatta Parker Administration for Children and Families U.S. Department of Health and Human Services Contract Number: HHSP233201500069I Project Director: Randall Juras Abt Associates 5001 S. Miami Blvd #210 Durham, NC 27703 This report is in the public domain. Permission to reproduce is not necessary. Suggested citation: Juras, Randall, Katarzyna Steinka-Fry, Meredith Kelsey, Mark Lipsey, and Jean Layzer (2019). Meta-Analysis of Federally Funded Teen Pregnancy Prevention Programs: Technical Supplement. Washington, DC: Office of Population Affairs, Office of the Assistant Secretary for Planning and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Disclaimer: This publication was supported by Award No. HHSP233201500069I from the Office of Population Affairs (OPA). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of OPA or the U.S. Department of Health and Human Services. CONTENTS CONTENTS Introduction ........................................................................................................................................... 1 1. Eligibility, Cleaning, and Coding ................................................................................................ 2 1.1. Eligibility Criteria ............................................................................................................ 2 1.1.1 Eligibility Criteria ............................................................................................... 2 1.1.2 Eligible Research Designs................................................................................... 2 1.2. Screening and Coding Procedures ................................................................................... 3 1.2.1 Study Screening and Coding Procedures ............................................................ 3 1.2.2 Meta-Analysis Coding Manual ........................................................................... 3 1.3. Individual Participant Data Request............................................................................... 17 1.3.1 Overview ........................................................................................................... 17 1.3.2 Instructions for Providing Individual Participant Data ..................................... 17 1.4. Calculation of Effect Sizes and Standard Errors ............................................................ 20 1.5. Moderator Definitions and Coding ................................................................................ 22 2. Analysis Plan ............................................................................................................................... 25 2.1. Methodological Specifications ....................................................................................... 25 2.1.1 Aggregate Data Meta-Analysis ......................................................................... 25 2.1.2 Individual Participant Data Meta-Analysis ....................................................... 26 2.1.3 Analysis of Program Attendance and Retention ............................................... 27 2.2. Deviations from Pre-Specified Analysis Protocol ......................................................... 27 3. Additional Results and Sensitivity Analyses ............................................................................ 30 3.1. Distribution of Synthesized Effect Sizes and Statistical Findings by Outcome ............. 30 3.2. IPD Analysis Detailed Results ....................................................................................... 34 3.3. Subgroup Effects from IPD Meta-Analysis ................................................................... 40 3.4. Sensitivity Analyses Examining Robustness of Mean Effect Size Estimates ................ 66 3.5. Bivariate Correlations between Moderators................................................................... 68 3.6. Additional Meta-Regression Model Specifications for Associations between Moderators and Effect Sizes .......................................................................................... 71 3.7. Meta-Analysis Using All Effect Sizes ........................................................................... 76 3.8. Relationships between Study Methods and Analysis Results ........................................ 79 3.8.1 Meta-Analysis of Randomized Experiments..................................................... 79 3.8.2 Post-Test Assessment Timing ........................................................................... 82 3.9. Overall Effects of Programs That Did and Did Not Report Effect Sizes for Recent Pregnancy....................................................................................................................... 84 References ............................................................................................................................................ 86 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs i CONTENTS LIST OF TABLES Table 1.5.1: Moderators Related to RQ1, Program Design ................................................................. 22 Table 1.5.2: Moderators Related to RQ2, Program Implementation ................................................... 23 Table 1.5.3: Moderators Related to RQ3, Participant Characteristics ................................................. 23 Table 1.5.4: Moderators Related to RQ4, Study Methods ................................................................... 24 Table 3.3.1: Subgroup Effects by Participant Gender.......................................................................... 41 Table 3.3.2: Subgroup Effects by Ethnicity ......................................................................................... 48 Table 3.3.3: Subgroup Effects by Race................................................................................................ 57 Table 3.4.1: Sensitivity Analyses Examining Robustness of Mean Effect Size Estimates for binary outcomes ........................................................................................................................ 67 Table 3.5.1: Bivariate Correlations between Moderators .................................................................... 69 Table 3.5.1: Bivariate Correlations between Moderators (Continued) ................................................ 70 Table 3.6.1: Program Design Moderators of Effects: Unstandardized Coefficients and 95% Confidence Intervals from Meta-Regression Models .................................................... 71 Table 3.6.2: Program Implementation Moderators of Effects: Unstandardized Coefficients and 95% Confidence Intervals from Meta-Regression Models .................................................... 72 Table 3.6.3: A Participant Characteristic Moderators of Effects: Unstandardized Coefficients and 95% Confidence Intervals from Meta-Regression Models ............................................ 72 Table 3.6.4: Study Method Moderators of Effects: Unstandardized Coefficients and 95% Confidence Intervals from Meta-Regression Models ........................................................................ 73 Table 3.6.5: Regression Models Examining Moderators of Participant Attendance Rates ................. 73 Table 3.6.6: Regression Models Examining Moderators of Program Retention Rates ....................... 74 Table 3.7.1: Relationships between Program Design Features and Average Effect Sizes.................. 76 Table 3.7.2: Relationships between Program Implementation Features and Average Effect Sizes.... 77 Table 3.7.3: Relationships between Participant Characteristics and Average Effect Sizes ................. 78 Table 3.8.1: Overall Effects of TPP Programs on Confirmatory Outcomes ........................................ 79 Table 3.8.2: Relationships between Program Design Features and Average Effect Sizes.................. 79 Table 3.8.3: Relfationships between Program Implementation Features and Average Effect Sizes .. 81 Table 3.8.4: Relationships between Participant Characteristics and Average Effect Sizes ................ 81 Table 3.8.5: Relationships between Study Methods and Average Effect Sizes ................................... 82 Table 3.8.6: Relationship between Post-test Assessment Timing and Average Effect Sizes (for timing coded as a categorical variable) ..................................................................................... 82 Table 3.8.7: Relationship between Post-test Assessment Timing and Average Effect Sizes (for timing coded as a binary variable) ............................................................................................. 83 Table 3.9.1: Overall Effects of TPP Programs for Studies with reported Recent Pregnancy Outcome84 Table 3.9.2: Overall Effects of TPP Programs for Studies without reported Recent Pregnancy Outcome ......................................................................................................................... 84 Table 3.9.3: Relationships between Report of Recent Pregnancy Outcome and Average Effect Sizes 85 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs ii CONTENTS LIST OF FIGURES Figure 3.1.1: Distribution of Program Effects for Odds of Ever Having Sex ...................................... 30 Figure 3.1.2: Distribution of Program Effects for Odds of Recent Sexual Activity ............................ 31 Figure 3.1.3: Distribution of Program Effects for Odds of Recent Unprotected Sexual Activity ....... 32 Figure 3.1.4: Distribution of Program Effects for Odds of Any Pregnancy ........................................ 33 Figure 3.1.5: Distribution of Program Effects for Odds of Recent Pregnancy .................................... 34 Figure 3.2.1: Ever Had Sex: Program Effects for Participant Subgroups ............................................ 36 Figure 3.2.2: Recent Sexual Activity: Program Effects for Participant Subgroups .............................. 37 Figure 3.2.3: Recent Unprotected Sexual Activity: Program Effects for Participant Subgroups ......... 38 Figure 3.2.4: Ever Pregnant: Program Effects for Participant Subgroups........................................... 39 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs iii INTRODUCTION Introduction This document is a technical supplement to the final report for the Meta-Analysis of Federally Funded Teen Pregnancy Prevention Programs. It provides additional detail on the meta-analysis’s design and implementation, including reproductions of documents used by the Abt team to determine study eligibility and clean and code data. The technical supplement is divided into three chapters. The first chapter of the supplement provides additional information on how studies were screened, how data were coded from reports, and how individual participant data were cleaned and coded. The second chapter provides a detailed discussion of methods that were pre-specified prior to data analysis and then discusses deviations from that pre-specified protocol. The last chapter provides detailed results for the full sample and for subgroups that supplement those discussed in the report, as well as sensitivity analyses. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 1 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING 1. Eligibility, Cleaning, and Coding This chapter provides additional information on how studies were screened, how data were coded from reports, and how individual participant data were cleaned and coded. 1.1. Eligibility Criteria This section provides additional detail on study eligibility criteria. 1.1.1 Eligibility Criteria To be eligible for inclusion in the meta-analysis, studies had to meet each of the following criteria: • Evaluated a teen pregnancy prevention program, broadly defined.1 • Included a comparison condition—no treatment, an alternative treatment such as driving skills training, or some form of business as usual (i.e., what participants would have received absent the evaluation study). 2 • Used an experimental or controlled quasi-experimental design that compared participants receiving one pregnancy prevention program with at least one valid comparison condition. See below for descriptions of eligible designs. • Assigned at least 10 study participants to the intervention and comparison group(s). • Measured and reported on at least one sexual behavior or sexual risk behavior. There were no other restrictions on the type of measure, reporter, or scale used for these outcome measures. 1.1.2 Eligible Research Designs To be eligible for inclusion in the meta-analysis, a study must have used one of the following research designs: • A randomized design where participants were randomly assigned to intervention and comparison conditions. Randomization could occur at the individual or larger cluster (group) level. • A quasi-randomized design where participants were assigned by a quasi-random procedure plausibly equivalent to randomization (e.g., alternation, date of birth, case record number). • A quasi-experimental design with matching where participants were not randomly assigned to conditions, but participants were matched on at least one baseline measure of prior sexual behavior or a close proxy risk factor for sexual behavior. Baseline measures were required to have been measured prior to the receipt of the intervention. 1 We define such a program as an intervention that involved actions performed with the explicit expectation that services would reduce pregnancy and/or reduce the rate of sexually transmitted infections. 2 Studies that compared two active teen pregnancy prevention programs were excluded from the meta-analysis because they only provided information about the relative effects of two active programs and did not measure the absolute effect of a teen pregnancy prevention program compared with usual practice. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 2 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING 1.2. Screening and Coding Procedures This section gives an overview of the study’s screening and coding procedures, followed by an exact reproduction in Section 1.2.2 of the coding manual used by the Abt team. 1.2.1 Study Screening and Coding Procedures Eligibility screening was conducted by doctoral-level researchers. Any disagreements about study eligibility were resolved via discussion with the Co-Principal Investigator and Project Director. Aggregate Data Sample. We used standard systematic reviewing and meta-analysis procedures (Lipsey and Wilson 2001) to extract data for the aggregate data (AD) meta-analysis. Data were extracted from the study evaluation reports by two master’s- or doctoral-level researchers, each of whom participated in several weeks of initial training followed by weekly coding meetings. A doctoral-level researcher reviewed all study coding and resolved any coding disagreements via discussion with the coders and the Data Collection Lead. All data extraction followed a standardized coding protocol (see Section 1.2.2). Individual Participant Data Sample. Through the Office of Adolescent Health (OAH) within the U.S. Department of Health and Human Services (HHS), we requested individual participant data (IPD) via e- mail for each eligible study that was completed prior to October 31, 2016. Each e-mail included a set of instructions (see Section 1.3), an Excel data shell, and a username and password for uploading the data to Abt’s secure data-transfer site. We requested that grantees provide specific participant-level outcome and demographic variables, including group assignment (treatment vs. control/comparison), demographic characteristics (e.g., age, race, gender), baseline sexual risk and behavior measures, sexual risk and behavior outcomes at follow-up, and study design variables (e.g., weights and random assignment block dummies). Each grantee was assigned a Data Liaison from the Abt team who was available by phone and e-mail to answer any questions about the data request. Once received, each data set was reviewed by a doctoral-level researcher. When necessary, Data Liaisons sent follow-up questions to grantees to resolve unclear data labels or values. 1.2.2 Meta-Analysis Coding Manual Meta-Analysis Coding Manual [Variable Names Shown in Brackets] Study Level Study identification number. The “unit” you will code here consists of a study, i.e., one research investigation of a defined subject sample or subsamples compared to each other, and the treatments, measures, and statistical analyses applied to them. Sometimes there are several different reports about a single study. In such cases, the coding should be done from the full set of relevant reports, using whichever report is best for each item to be coded; be sure you have the full set of relevant reports before beginning to code. Sometimes a single report describes more than one study sample, e.g., evaluations at three separate sites. In these cases, each study sample will have a unique study identification number and each study should be coded separately as if it had been described in a separate report. [studyid] Each study has its own study identification number, or StudyID (e.g., 619). Each report also has an identification number (e.g., 619.01), which you will find in the FileMaker bibliography. The ReportID has two parts; the part before the decimal is the StudyID, and the part after the decimal is used to distinguish the reports within a study. (These two types of ID numbers, along with bibliographic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 3 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING information, are assigned and tracked using the bibliography.) When coding, use the study ID (e.g., 619) to refer to the study as a whole, and use the appropriate report ID (e.g., 619.01) when referring to an individual report. Coder’s initials [coder] State in which the prevention program was implemented (check all that apply). [state] 1. Alabama 2. …. 3. … 51. District of Columbia 52. Single state (unspecified) 53. Multiple states (unspecified) Group Identification and Selection At this stage, you will need to identify the groups in the study for which effect size statistics can be computed. Note that for any group comparison coding, the two groups involved must be from the same experiment or quasi-experiment; that is, they must have been involved in the same randomization, matching, etc. from the same design. If two or more experiments or quasi-experiments are presented in the same report, each must be handled separately. Intervention Groups Write in Name [txa-d] 1-4 __________________________________ Comparison Groups Write in Name [cta-d] 1-4 __________________________________ Study Design and Methods Method of assignment to groups. This item focuses on the initial method of assignment to groups regardless of subsequent degradations due to attrition, refusal, etc. prior to treatment onset. These latter situations are coded elsewhere. [design] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 4 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Random or near-random: 1. Randomly assignment at the individual level. Individual participants are randomly assigned to conditions. In some cases random assignment may be done after individuals have been matched or blocked. 2. Random assignment by group; that is, intact groups such as classrooms are assigned. 3. Regression discontinuity design: quantitative cutting point defines groups on some continuum (this design will be rare). 4. Quasi-randomized procedure presumed to produce comparable groups. This applies to groups which have individuals assigned by some naturally occurring process that is apparently random, e.g. alternation, date of birth, medical record number. The key here is that the procedure used to select groups is not strictly random, but the method of allocation should not create nonequivalence between groups. Non-random, but matched or statistically controlled: Note: Matching refers to the process by which individuals are selected for conditions (e.g., treatment and comparison) in a manner that ensures that the individuals in one group are matched on certain relevant characteristics in the other group. Comparing the characteristics of the groups after they have been assigned to experimental conditions does not constitute matching. 5. Matched individually, through sampling, on one or more baseline measures of sexual behavior, sexual behavior risk factors, demographic characteristics, or other measures. 6. Statistical controls used to equate individuals on one or more baseline measures of sexual behavior, sexual behavior risk factors, demographic characteristics, or other measures (e.g., through regression control, ANCOVA, analysis of covariance, propensity score methods). 7. Matched at a larger group level; that is, intact groups were matched on their means for some set of characteristics; e.g., the mean ages of the groups are similar, but each subject in one group has not been individually matched to a subject in the other group on age. Please list all of the variables used in the matching and/or statistical controls. [matchedvarlist] For cluster randomized trials, please enter the average cluster size (i.e., average number of youth in each cluster). Code -9 for cannot tell. Code -8 for not applicable. [m] What is the risk of selection bias due to inadequate generation of a randomized sequence? [rob_sg] 1. Low risk. The investigators describe a random component in the sequence generation process such as referring to a random number table, using a computer random number generator, coin tossing, shuffling cards/envelopes, throwing dice, drawing of lots, or minimization. 2. High risk. The investigators describe a non-random component in the sequence generation process. This might involve some systematic non-random approach such as odd/even birth dates, rules based on dates of admission, rules based on some sort of record number. Other non- random approaches might include allocation by judgement (e.g., teacher, practitioner ratings), allocation by participant preferences, or allocation by availability of the intervention. By definition, any quasi-experimental design where participants self-select into conditions is at high risk of bias. 3. Unclear risk of bias. Insufficient information is provided about the sequence generation process to permit judgement of low or high risk. Provide a description of the information used to code the risk of bias due to sequence generation. [rob_sg_t] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 5 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING What is the overall attrition rate (across all groups) in the study between the time of assignment to groups to the first follow-up? This item refers to overall attrition in the study; more detailed attrition calculations will be estimated using the assigned and observed sample sizes coded in the effect size section. [attrf_o] What is the overall attrition rate (across all groups) in the study between the time of assignment to groups to the last follow-up? Again, this item refers to overall attrition in the study; more detailed attrition calculations will be estimated using the assigned and observed sample sizes coded in the effect size section. [attrl_o] Did the authors use an intent-to-treat (ITT) analysis? Intent-to-treat analysis refers to situations where researchers ‘analyze as randomized’, meaning that all individuals that were randomized to the intervention/control groups are included in the final outcome analysis, regardless of whether they actually attended the intervention. Note, that it is possible for a study to conduct an ITT analysis even if they have attrition, as long as they had intended to include any non-compliers in their final model. [itt] 1. Yes – Explicitly stated 2. No -9. Cannot tell How did the authors handle missing data in their analysis? NOTE: If the authors use multiple methods choose the method used for missing data on the dependent variables. [missdata] 1. Listwise deletion 2. Pairwise deletion 3. Mean or mode imputation 4. Single regression imputation 5. Dummy variable approach (imputed value at zero with dummy variable) 6. Multiple imputation 7. Full information maximum likelihood (FIML) 8. Other method 9. Not applicable – no missing data 10. Cannot tell Was there bias due to selective outcome reporting? [bias] 1. Low Risk of Bias. All baseline, pretest, and outcome measures outlined in the Methods section (or specified elsewhere in the report) are reported in the Results section. 2. High Risk of Bias. Code if any one of the following is true: a. Not all of the study’s pre-specified baseline/pretest measures or primary outcomes have been reported. b. One or more baseline/pretest or outcome measures is reported using measurements, analysis methods, or subsets of the data (e.g., subscales) that were not pre-specified. c. One or more pre-specified baseline/pretest or outcome measures are reported incompletely so that they cannot be entered into meta-analysis. d. The report fails to include results for a key outcome that would be expected to have been reported for such a study. e. Evidence that analyses and other method choices may have been manipulated to bias the findings reported (e.g., choice of model fit, omission of key confounders). 3. Unclear/Cannot Tell. Insufficient information to permit judgment of “Low Risk” or “High Risk.” Provide a description of the information used to code the risk of bias due to selective reporting. [bias_t] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 6 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Intervention and Comparison Groups Create one record in this database for each of the intervention and/or comparison groups you selected earlier for coding. For example, studies with a single intervention group and a single comparison group will have two records in this section of the database. Number each group consecutively within a study, starting with 1. [groupid] Select the type of group you are coding. [tvc] 1. Intervention group 2. Control/comparison group What type of services does this group receive? [type] 1. Focal/primary intervention program. There may be several focal programs in a study, as when two different types of programs are compared, both of which are expected to be effective. 2. Active treatment that is not a pregnancy prevention program. This is a group that receives a sham treatment (e.g., watches a video on nature, receives nutrition information, diet intervention) intended to take the same duration as the focal intervention program, but does not involve any active teen pregnancy prevention components. 3. Inactive treatment. This is a group that receives no prevention program and gets only assessments. 4. Active business as usual. This is a group that receives “usual” active treatment (e.g., sex education, teen pregnancy prevention) that may be effective in preventing teen pregnancy but is not the focal treatment of the study. This treatment must be limited to services that the youth would receive whether or not the research study was implemented (e.g., mandated school-based sex education). 5. Other (please specify). Program name. Write in the program name or label for this group. [name] Program description. Write in a brief description of the treatment this group receives. As much as possible, quote or give a close paraphrase of the relevant descriptive text in the study report; always include page numbers to the report when appropriate. It is acceptable to copy and paste directly from the article as long as you include the information in quotations and provide a page number for the quotation. [descrip] Participant Characteristics Enter the percent of males in this group. Use -9 for cannot tell. [permale] Enter the percent of White participants in this group. Use -9 for cannot tell. [perwhite] Enter the percent of Non-White participants in this group. Use -9 for cannot tell. [pernonwhite] Enter the percent of Black participants in this group. Use -9 for cannot tell. [perblack] Enter the percent of Hispanic participants in this group. Use -9 for cannot tell. [perhisp] Enter the average age of the group using number of years. Use -9 for cannot tell. [age] Enter the age range of the group using “XX-XX” format. Use -9 for cannot tell. [agerange] Enter the percent of participants in this group who reported ever having had sex at baseline (vaginal intercourse, oral, or anal sex). Use -9 for cannot tell. [anysex] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 7 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Enter the percent of participants in this group who reported ever having had sexual (vaginal) intercourse at baseline. Variables that use the term “sexual intercourse” should be coded here. Use -9 for cannot tell. [anyint] Enter the percent of participants in this group who reported ever having had oral sex at baseline. Use -9 for cannot tell. [anyor] Enter the percent of participants in this group who reported ever having had anal sex at baseline. Use -9 for cannot tell. [anyan] Enter the percent of participants in this group who reported recently having sex (e.g., in the past 3 months) at baseline (intercourse, oral, or anal sex). Use -9 for cannot tell. [recentsex] Enter the percent of participants in this group who reported recently having any unsafe sex (intercourse, oral, or anal sex) at baseline. Use -9 if cannot tell. [recentuns] Enter the percent of participants in this group who reported ever having any unsafe sex (intercourse, oral, or anal sex) at baseline. Use -9 for cannot tell. [unsafesex] Enter the percent of participants in this group who reported ever having any unsafe vaginal sexual intercourse at baseline. Use -9 for cannot tell. [unsafeint] Enter the percent of participants in this group who reported ever having any unsafe oral sex at baseline. Use -9 for cannot tell. [unsafeor] Enter the percent of participants in this group who reported ever having any unsafe anal sex at baseline. Use -9 for cannot tell. [unsafean] Intervention Group Characteristics Is this pregnancy prevention program on the evidence-based program list? Tier 1 programs are evidence- based and Tier 2 programs are not. [rep] 1. Yes 2. No What is the primary focus of this teen pregnancy prevention program? Note that many programs include similar elements in their logic models (e.g., good decision-making, attitudes about risk behavior, development of refusal or negotiation skills). Programs with different goals in mind may all stress that abstinence is the only 100% protection against sexual risk, but that does not necessarily mean that the primary focus of the prevention program is on abstinence. If you are unsure how to code this item, please contact Meredith Kelsey or another content expert. [focus] 1. Abstinence. Abstinence is the only choice. The program provides no discussion of birth control methods. 2. Sexual health. The program may say that abstinence is the one sure way to avoid sexual risk, but also stresses need for protection if you are sexually active. The program always discusses different birth control methods and protection against infection. 3. Youth development. Sexual risk is not the major focus of the program and may not even be addressed explicitly. The program will mention a basis in positive youth development model, and include a broader focus on poor choices (educational, gang activity, drugs and alcohol) as well as possibly sexual risk behavior. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 8 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING 4. HIV/AIDS prevention. If the focus of the original model was as narrow as this, the description will say so, even if material on pregnancy prevention is added. Only code this if the description uses this terminology. 5. Reproductive health services. Possibly delivered in a clinic setting. Could have other elements, such as skills practice, reflective activities, but focus is on direct provision of health services. What types of program components did this group receive? Only code components that are unique to the intervention group (i.e., components that the control group did not receive). Check all that apply for any component present in the program. [progtype] 1. Condom demonstration. This might be hands-on activity or a demonstration with actual models, a video, a mini-lecture, or a comic strip. 2. Service learning. This is a feature of at least one of the more frequently used models. It is not simply community service – it involves group reflection on the experience. Only code if the term “service learning” is used. 3. Role-plays. These are used to develop skills – most often refusal or negotiation skills with respect to sexual risk behavior, but could be to avoid a broader range of risks – gang or other illegal activity, drugs or alcohol, truancy. This component includes skits. 4. Games. Used to practice skills, communicate information, could be group activity or individual with computer. 5. Reflective exercises. Could include journaling, motivational interviewing. 6. Mentoring/tutoring. Individualized mentoring or tutoring. Most likely as part of a youth development program. 7. Individualized counseling. Could be face-to-face, through social media, via text messaging. 8. Direct provision of reproductive health and other health services. Note that many if not most programs provide linkages to health and other services – here their provision is part of the program. 9. Parent activities. Includes: homework for parents, or for parent/child dyad; informational materials distributed to parents; group sessions for parents or for parents with their child; text messaging to parents. 10. Community outreach. Could include media campaigns, public service announcements, rallies, presentations to churches, community groups. 11. Positive role model(s). Opportunities for exposure to positive role models who are not individual mentors. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 9 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Monitoring of treatment implementation. Was the implementation of the program monitored by the author/researcher or program personnel to assess whether it was delivered as intended? [monitored] 1. Yes, but no indication of feedback to treatment providers. Do not infer that monitoring happened. Select “yes” only if the report specifically indicates that implementation was monitored. 2. Yes, with indication that treatment providers received feedback. Do not infer that monitoring happened. Select “yes” only if the report specifically indicates that implementation was monitored. 0. No indication that service delivery was monitored. Implementation quality. Based on evidence or author acknowledgment, was there any uncontrolled variation or degradation in implementation or delivery of treatment, e.g., high dropouts, erratic attendance, low treatment compliance, treatment not delivered as intended, wide differences between settings or providers, etc. Note that this question has to do with variation in treatment delivery, not research contact. That is, there is no “dropout” if all juveniles complete treatment, even if some fail to complete the outcome measures. [impprob] 1. Yes 2. Possible 3. No, apparently implemented as intended Implementation fidelity. Provide a description of any other implementation fidelity measures, assessments, and/or findings including page numbers where appropriate. [impfid] In what setting(s) was the prevention program typically delivered? [setting] 1. Classroom 2. Health clinic 3. Community 4. Other In what format was the prevention program typically delivered? [format] 1. Individual youth with provider 2. Small groups (<10) with provider 3. Large group or whole classrooms with provider 4. Online 5. Other Who typically delivered the prevention program? [provider] 1. Medical professionals (nurses, doctors, clinicians) 2. Health educators (agency staff) 3. Classroom teachers 4. Peer educators 5. Other 6. Mixed (no predominant provider type) What is the sex composition of the intervention group? [mixedsex] 1. Same sex 2. Mixed sex 3. Cannot tell Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 10 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Culturally specific program. Is the program specifically tailored to target a specific cultural, racial, or ethnic group? Only code yes if the report specifically describes the program as targeting a particular group (e.g., racial/ethnic, religious, or SES group, youth whose native language is not English, etc.). [cultural] 1. Yes – explicitly stated 2. No If applicable, provide a brief description of the culturally specific group that the program targets. [culturaldes] Duration of implemented program in weeks. Approximate (or exact) number of weeks for the period over which youth received the program, from first to last treatment contact, excluding follow-ups designated as such. Divide days by 7; multiply months by 4.3; multiply years by 52; round to a whole number. Estimate for this item if necessary and if you can come up with a reasonable order of magnitude number (e.g., take the midpoint of a range if it is all that’s provided). Code -9 if cannot tell. [txwks] Duration of program as intended in contact hours. Approximate (or exact) number of contact hours for the period over which the adolescents were intended to receive the program, from first program contact to last contact, excluding follow-ups designated as such. Code -9 if cannot tell. [txhours] Approximate (or exact) frequency of contact between adolescent and provider or treatment activity. This refers only to the element of treatment that is different from what the control group receives or would have received had a control group been formed in treatment circumstances. [numsessions_cat] 1. Daily contact 2. 3-4 times a week 3. 1-2 times a week 4. Less than weekly 5. One day only -9. Cannot tell Provide page numbers for the information on implemented and intended program duration and dosage (weeks, hours, frequency of sessions). [duration] Provider training, preparation, or qualifications. Describe any information provided about the intervention providers’ training, level of preparation, or instructor qualifications required for delivery. [provid] Incentives for recruitment or participation. Describe any incentives for participant recruitment and/or participation. Provide specific information about incentives (including dollar amounts), when available. [incent] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 11 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Outcomes Study and DV Identification Create one record for each dependent variable that you will be coding. If the study measures sexual activity and pregnancy outcomes, you will have two dependent variable records. This is different from the number of times a dependent variable is measured in a study. For example, if the study measures sexual activity before and after treatment, you will have only one record here – for the sexual activity measure (but you will have two effect sizes for this outcome measure: one at pretest and one at post-test). Variable number. This number is an identification number for the dependent variable you are coding. Each dependent variable is numbered consecutively, within the study you are coding so that each has a unique VarNo for that study. If there is only one dependent measure for this study, you will create only one record in this worksheet, and the variable number will be 1. If there are three dependent measures, they will be numbered 1, 2, and 3. [varid] Description of the dependent measure. Write in a brief description of the dependent measure you are coding. This should include the authors’ label for this variable (e.g., ever has sexual intercourse, had sex within past three months, etc.), the instrument, the direction of scoring (e.g., lower scores are better), and information about what is being measured (e.g., problems associated with sexual behavior, etc.). Quote or closely paraphrase the description that is provided in the original report. For variables for group equivalence coding make sure the label describes successes (e.g., blacks, non-whites, younger age). As an exception (for consistency with the research reports), code sex as proportion of females. When coding race always default to white v. non-white. If the sample is only minority youth then default to black v Hispanic (with black as the success). [dvdes] What type of dependent measures are you coding? [dvmicro] 01 Sexual Activity 1. Ever had sex (yes/no) 2. Recent sexual activity (yes/no) 3. Recent unprotected sex (yes/no) (sexual intercourse without a condom) 4. Number of sexual partners (in last xx days) 5. Number of sexual experiences (in last xx days) 6. Number of unprotected sex experiences (in last xx days) 7. Other sexual activity measure 02 Sexually Transmitted Infections 8. Any STI 9. Specific type of STI 10. Number of STIs 11. Other STI measure 03 Pregnancy and births 6. Ever pregnant (yes/no) 7. Number of pregnancies 8. Ever given birth or fathered a child (yes/no) 9. Other pregnancy measure Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 12 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING 04 Other Characteristics Used for Group Equivalence Effect Sizes 10. Sex/Gender 11. Race/Ethnicity 12. Age 13. Other Pregnancy Risk Factor Type of data collection used for outcome measure. [dvtype] 1. In-person interview 2. Phone interview 3. Pencil & paper questionnaire 4. Online/computer assisted questionnaire 5. Other -9. Cannot tell Number of Days. Enter the number of days over which outcome was counted. Enter -8 for lifetime measures. Enter -9 if cannot tell. Multiply months by 30 (e.g., enter 3 months as 90 days). [dvdays] For cluster randomized trials, please enter the intraclass correlation coefficient (ICC) for each outcome variable coded. Code -8 for not applicable and -9 for cannot tell. [icc] Effect Sizes Although this is the final section of coding, it is a good idea to identify at least one codable effect size before you start coding a study, because studies that appear eligible frequently end up presenting data that cannot be coded into an effect size. Note that effect sizes for breakouts (i.e., subsamples based on gender, race, etc.) are ineligible for coding. The only exception is breakouts of study participants who had not engaged in sex (i.e., vaginal, oral, or anal) at baseline. Effect sizes measuring sexual initiation (vaginal, oral, or anal) among those participants who had not initiated sex at baseline should be coded (with pre-test proportions coded as 1.00 successful). Report ID for this effect size. Indicate the report number (e.g., 2098.01) for the report in which you found the information for this effect size. This is important so that we can find the source information for the effect sizes later on, if necessary, and is especially important for studies with multiple reports. [reportid] Page number for this effect size. Indicate the page number of the report identified above on which you found the effect size data. If you used data from two different pages, you can type in both, but use a comma or dash between the page numbers. [page] Type of effect size you are coding. [estype] 1. Pretest and Post-test 2. Group equivalence There are 3 types of effect sizes that can be coded: pretest, post-test, and group equivalence (or baseline similarity) effect sizes. They are defined as follows: • Group equivalence effect size. Group equivalence effect sizes are used to code the equivalence of two groups prior to treatment delivery on (a) gender, (b) age, (c) race/ethnicity, and/or (d) another risk measure for pregnancy. When multiple racial/ethnic group compositions are reported please report only White/nonwhite proportions (if not available, select another racial/ethnic group). When available, always code gender, age, and race/ethnicity. When multiple other risk factors are reported Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 13 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING select the three deemed to be most relevant (behaviors are more relevant than attitudes/intentions). Cap “other” risk factors at three. • Pretest effect size. This effect size measures the difference between an intervention and comparison group before treatment (or at the beginning of treatment) on the same variable used as an outcome measure. Note: Use pretest data for different analytic samples if available. (e.g., separate pretest data for different follow-up waves). • Post-test/follow-up effect size. This effect size measures the difference between two groups after treatment receipt on some outcome variable. Some post-tests can occur during treatment (after intake), immediately after treatment ends, or any subsequent follow-up period after treatment ends. Group Selection Select the groups being compared in this effect size. Always select the focal prevention program to be ‘group 1.’ [esgroup1] [esgroup2] Dependent Variable Selection Select the dependent variable for this effect size. [varid] Timing of measurement. Approximate (or exact) number of weeks after the end of the intervention when measurement occurred. Divide days by 7; multiply months by 4.3. Enter -9 if cannot tell, but try to make an estimate if possible. [estiming] Effect Size Calculation and Data Entry It is now time to identify the data you will use to calculate the effect size and to calculate the effect size yourself if necessary. You need to determine what effect size format you will use for each effect size calculation. There are two general formats you can use, each with its own section in FileMaker: 1. Compute ES from means, sds, variances, test statistics, etc. 2. Compute ES from frequencies, proportions, contingency tables, odds, odds ratios, etc. Also note that within each of the above effect size formats, effect sizes can be calculated from a variety of statistical estimates; to determine which data you should use for effect size calculation, please refer to the following guidelines in order of preference: 1. Compute ES from regression coefficients with statistical controls for pretest measures and other potential confounding measures at baseline. 2. Compute ES from univariate descriptive statistics (means, sds, frequencies, proportions). 3. Compute ES from test statistics (t, F, Chi square). 4. If significance tests statistics are unavailable or unusable but p-values and degrees of freedom (df) are available, determine the corresponding value of the test statistic (e.g., t, chi-square) and compute ES as if that value had been reported. If you encounter these types of data, please see Emily for guidance. Note that if the authors present both covariate adjusted and unadjusted means, you should use the covariate adjusted ones. If adjusted standard deviations are presented, however, they should not be used. Which group is favored? [esfavor] For intervention-control comparisons, the intervention group is favored when it does “better” than the comparison group. The comparison group is favored when it does “better” than the intervention group. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 14 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING For group equivalence comparisons, the intervention group is favored when it is at lower risk of unsafe sexual activity than the comparison group (i.e., when respondents are male, younger in age, or non- White). Racial/ethnic group risk (from lowest to highest) is American Indian, Black, White, Hispanic (per the 2011 Youth Risk Behavior Survey). Remember that you cannot rely on simple numerical values to determine which group is favored. For example, a researcher might assess the amount of sexual activity, and report this in terms of the number of sexual partners. Fewer sexual partners is better than more, so in this case a lower number, rather than a higher one, indicates a more favorable outcome. Sometimes it may be difficult to tell which group is better off because a study uses multi-item measures in which it is unclear whether a high score or a low score is more favorable. In these situations, a thorough reading of the text from the results and discussions sections usually can bring to light the direction of effect – e.g., the authors will often state verbally which group did better on the measure you are coding, even when it is not clear in the data table. Note that if you cannot determine which group has done better, you will not be able to calculate a numeric effect size. (You will still be able to create an effect size record—just not a numeric effect size.) Select the group that has done “better”: 1. Intervention 2. Comparison 3. Neither, Exactly Equal -9. Cannot tell Effect size derived from what type of statistics? [esdata] 1. N successful/unsuccessful (frequencies) 2. Proportion successful/unsuccessful (percentage successful or not) 3. Means and SDs; means and variances; means and standard errors 4. Independent t-test 5. Chi-square statistic (1 degree of freedom) 6. Effect sizes as reported directly in the study 7. Other statistical approximation For this effect size, did you use adjusted data (e.g., covariate adjusted means) or unadjusted data? If both unadjusted and adjusted data are presented (for post-test measures), you should use the adjusted data for the group means or mean difference, but use unadjusted standard deviations or variances. (If both adjusted and unadjusted data are presented for baseline measures, use the unadjusted data). Adjusted data are most frequently presented as part of an analysis of covariance (ANCOVA). The covariate is often either the pretest or some personal characteristic such as socioeconomic status. If you encounter data that is adjusted using something other than a covariate, please see Emily. [esadj] 1. Unadjusted data 2. Pretest adjusted data (or other baseline measure of an outcome variable construct) 3. Data adjusted on some variable other than the pretest (e.g., socioeconomic status) 4. Data adjusted on pretest plus some other variables Assigned N for the intervention group [estxasn] Assigned N for the comparison group [exctasn] Observed N for the intervention group [estxobn] Observed N for the comparison group [exctobn] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 15 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Mean for intervention group [estxmean] Mean for comparison group [esctmean] Standard deviation for intervention group [estxsd] Standard deviation for comparison group [esctsd] N successful for intervention group [escella] N successful for comparison group [escellc] N failed for intervention group [escellb] N failed for comparison group [escelld] Independent t-value [esindt] χ2 (df=1) [eschisq] Effect size reported by authors [esauth] Odds ratio reported by authors [esor] Final Effect Size Determination Effect size value- standardized mean difference [es_fmsmd] Effect size value- odds ratio [es_fmor] Remember that you cannot rely on simple numerical values to determine which group has done better. For intervention-control comparisons, a positive effect size should indicate that the intervention group did “better” on the outcome measure than the comparison group, while a negative effect size indicates that the comparison group did “better” than the intervention group, and a zero effect size means that the two groups are exactly equal on the measure. You must make sure that the sign of the effect size matches the way we think about direction, such that the effect size is positive when the intervention group (or post-test) is better and negative when the comparison group (or pretest) is better. Effect sizes can range anywhere from around –3 to +3. However, you will most commonly see effect sizes in the –1 to +1 range. Note: If the authors report an effect size, include that in your coding and use it for the final effect size value if no other information is reported. However, if the authors also include enough information to calculate the effect size, always calculate your own and report it in addition to that reported in the study. Any problems coding this effect size? [esprob] Does this effect size measure the difference between two groups on confirmatory outcome variable? Confirmatory outcome variable and measurement timing are designated by the authors. Authors often define the confirmatory outcomes (including a measurement and time period) in the section called “primary research question.” [primaryes] 1. Yes 2. No Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 16 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING 1.3. Individual Participant Data Request This section provides an overview of the request for IPD, followed by an exact reproduction in Section 1.3.2 of the instructions given to grantees on how to provide IPD from the grantee’s evaluation study. 1.3.1 Overview As described in Section 1.2, we requested IPD via e-mail for each eligible study completed prior to October 31, 2016. We offered grantees incentive payments of $1,000 for complying with the request, to compensate for time and effort spent providing the data. The payment was conditional on receipt of the data as well as a signed memorandum of understanding specifying each party’s roles and responsibilities regarding data security and participant anonymity. Grantees were required to de-identify data prior to uploading it to Abt’s data transfer site. 1.3.2 Instructions for Providing Individual Participant Data The instructions reproduced below were provided to grantees, describing how to provide data for the cross-grantee quantitative synthesis. General Information Ideally, you will provide OAH with a single dataset, formatted as in the example Excel spreadsheet that is attached to this e-mail. Each row in the dataset should correspond to an individual participant, and the columns in the dataset should correspond to the variables that are being requested by OAH. There should only be one row for each study participant. This dataset can be in any table-readable format, such as a file created in Excel, R, SAS, SPSS, Stata, or a comma-separated or tab-delimited format. As you will see below, OAH is requesting text or numeric data for several variables. We appreciate that the format of variables will vary across sites, so we request that you label or describe any data value labels that may be unclear (e.g., specify participant gender as 0=Male 1=Female, or Male Female). Providing clear labels in the datasets (and/or providing a codebook with values for each variable) so that we can decipher the data, will prevent follow-up requests from us. In the e-mail to which these instructions were attached, you should have received a username and password for uploading data to the secure file transfer site, [redacted]. At the end of this document are step-by-step instructions for using the site. Once you have uploaded the data, please contact your data liaison by e-mail and/or phone to let them know. If we subsequently have any questions about the data, your data liaison will contact you for clarification. If you do not have a username and password, or if you have any trouble accessing the transfer site, please contact your Abt data liaison. Thank you for helping OAH with this important effort. If you have any questions about the study or about how to provide the data, please do not hesitate to contact OAH or your Abt Associates data liaison. List of Variables Requested Immediately below is a short description of each of the variables that OAH is requesting. Please provide each of these variables for each member of your study’s analytic sample (i.e., the sample you used in your final analysis). If you did not collect data on one or more of these variables, please simply omit that variable from the dataset you provide. Please code any participant-level missing values using whatever Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 17 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING method is most commonly used in your software package (e.g., “.” in Stata or a blank cell in Excel). Please do not provide any identifiable data to OAH (e.g., names or addresses). ID# This should be a non-identifiable identification number for each unique participant. Please do not provide any personally-identifying information such as name, address, or date of birth. Group Assignment This is an indicator for the participant’s treatment status; i.e., whether they were assigned to treatment or control. (For a QED, this would indicate treatment or comparison). Text or numeric values are acceptable, provided the numbers are clearly labeled. Age Please provide each participant’s age (in years) at baseline. You do not need to provide fractions of years if that information is not readily available (e.g., 14 years 6 months or 14.5 years could be coded as 14). Ethnicity Please provide each participant’s ethnicity (i.e., Hispanic or non-Hispanic). Text or numeric values are acceptable, provided all numeric values are clearly labeled. Race Please provide each participant’s race, as you coded it for your analysis (i.e., if you collapsed or cleaned open-ended responses prior to analysis, please provide the final cleaned/collapsed version. However, if you combined race and ethnicity into a single variable for your analysis, please back out ethnicity as a separate variable). Text or numeric values are acceptable, provided all numeric values are clearly labeled. Gender Please indicate each participant’s gender (e.g., male or female). Text or numeric values are acceptable, provided all numeric values are clearly labeled. Ever Had Sex at Baseline This variable or variables should indicate whether the participant has ever had sex (intercourse, oral, and/or anal sex) at baseline (i.e., before the intervention started). Text or numeric values are acceptable, provided all numeric values are clearly labeled. If you have more than one measure of baseline sexual history, please provide each measure (e.g., one variable for “ever had intercourse” and another for “ever had oral sex”) and label the variables accordingly (e.g. baseline_ever_intercourse and baseline_ever_oral). Please make sure that each of these variables is clearly labeled as a baseline measure. Other Baseline Sexual History Please provide any other measures of participants' baseline sexual history or sexual experience. Text or numeric values are acceptable, provided all numeric values are clearly labeled. If you have more than one measure of baseline sexual history, please provide each measure (e.g., one variable for “lifetime number of partners” and another for “had intercourse in the past 90 days”) and label the variables accordingly Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 18 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING (e.g. lifetime_num_partners and baseline_intercourse_90). Please make sure that each of these variables is clearly labeled as a baseline measure. Sexual Risk/Behavior Outcomes Please provide data on any outcomes that you measured at post-test related to sexual risk or sexual behavior, including (but not limited to) those you analyzed in the final report to OAH. These could include outcomes such as abstinence, condom use, or number of partners. Please do not provide non- sexual behavioral outcomes such as school attendance. If you measured outcomes at more than one follow-up time point, please provide outcomes for the time point that was analyzed in your final report. If you are providing more outcomes than were used in your final report, please indicate which measures were included in the report and which were not, either by including this information in the variable labels or by sending an e-mail to your data coordinator. Finally, please clearly identify which variables correspond to the following performance measures: 1. Ever had sexual intercourse 2. Ever been pregnant or gotten someone pregnant 3. Had intercourse in the past 3 months 4. Had intercourse without a condom in the past 3 months 5. Had intercourse without birth control in the past 3 months Intentions Outcomes Please provide any post-test measures of intentions to engage in sexual behaviors. Please clearly label each outcome as a measure of intentions, and identify which variables correspond to the following performance measures: 1. Intention to have intercourse in the next year 2. Intention to use condoms for intercourse in the next year 3. Intention to use birth control for intercourse in the next year Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 19 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING Knowledge/Attitude/Skill Outcomes Please provide data on any other knowledge, attitude, and/or skill outcomes that you measured at post- test, whether or not you reported them in your study (i.e., any non-behavioral outcomes that you measured). Please clearly label each outcome as a measure of knowledge, attitudes, or skills. Study Design Variables If your study used a blocked or stratified random assignment design, please include any blocking or stratification variables that you included in your analysis (e.g., you might have a set of dummy variables representing random assignment blocks). Likewise, if you matched treatment group participants with comparisons in a QED, please include any variables used in the matching process. If observations were weighted for the final analysis, please provide those weights. Please clearly label these variables as study design variables, and send an e-mail to your Abt data liaison explaining what these variables are and how they were used in your analysis. Instructions for Uploading Data [Detailed instructions for accessing Abt Associates’ secure web portal redacted] 1.4. Calculation of Effect Sizes and Standard Errors This section provides additional detail on how we calculated effect sizes and standard errors using aggregate data from study reports. Most studies reported binary measures for the sexual behavior outcomes, so the primary effect size metric we used to measure TPP program effects was the log odds ratio (LOR): where A is the count of “successes” in the intervention group (e.g., number of participants who did not engage in unprotected sex); B is the count of “failures” in the intervention group (e.g., number of participants who engaged in unprotected sex); C is the count of “successes” in the comparison group; and D is the count of “failures” in the comparison group. Log odds ratios were coded such that values greater than zero indicated beneficial TPP program effects relative to the comparison condition (e.g., lower odds of sexual behavior, lower odds of pregnancy). We conducted all analyses using the log odds ratio (unless noted otherwise), translating final results back into the odds ratio metric, for ease of interpretability. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 20 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING When studies measured outcomes on a continuous scale (e.g., mean number of sexual partners), we measured TPP program effects using the small-sample corrected standardized mean difference effect size, or Hedges’ g (Hedges 1981): where d is the standardized mean difference effect size calculated as the difference in post-test means for the intervention and comparison groups divided by the pooled standard deviation; N is the total sample size for the intervention and comparison groups combined; is the sample size for the intervention group; and is the sample size for the comparison group. When synthesizing effect sizes within outcome categories that only included Hedges’ g effect sizes (e.g., number of sexual partners), we conducted all analyses using the Hedges’ g effect size metric, for ease of interpretability. For all other analyses, however (e.g., when combining results across outcome categories), we transformed these standardized mean difference effect sizes into log odds ratio effect sizes using the Cox transformation (Sánchez-Meca, Marín-Martínez, and Chacón-Moscoso 2003): where is the variance (i.e., squared standard error) of the Hedges’ g effect size. Sensitivity analyses excluding these Cox-transformed effect sizes yielded no substantial changes to the findings (see Section 3.4 of this technical supplement), so all main analyses proceeded using the Cox- transformed effect sizes. We examined the distribution of effect sizes and sample sizes for outliers (defined as three times the interquartile ranges above/below upper fence values), identifying only a small number of effect size outliers. Sensitivity analyses using effect size values Winsorized to the upper/lower fence values yielded no substantial changes to the findings (again, see Section 3.4; therefore, all main analyses proceeded using the original, non- Winsorized effect sizes. We adjusted the standard errors of the effect size estimates used in the meta-analysis for the nesting of participants within clusters (e.g., schools) for those studies (number of included studies k = 20) using designs in which clusters were assigned to conditions. In these cases, we multiplied the standard error of the effect size by the square root of the design effect (Higgins, Deeks, and Altman 2008). When cluster-assigned trials did not report the intra-class correlation (ICC), or the ICC was not available in the IPD, we assumed ICC values of .01 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 21 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING (ever had sex outcomes), .003 (ever pregnant), and .00 (all other outcomes). We estimated these assumed ICC values as the conditional ICC estimates using the IPD from the 15 studies with cluster designs in the IPD sample. These assumed ICC values are similar to those reported in prior reviews of ICCs in group design studies of adolescent sexual health programs (Glassman, Potter, Baumler, and Coyle 2015). 1.5. Moderator Definitions and Coding The study’s key moderators, corresponding to the study’s first four research questions, related to program design, program implementation, participant characteristics, and study methods. This section defines moderators in each of these categories and specifies how they were coded. TABLE 1.5.1: MODERATORS RELATED TO RQ1, PROGRAM DESIGN Moderator Category Typology Coding Program Focus Abstinence Five (exclusive) dummy variables indicating primary Sexual health program focus Youth development HIV/AIDS prevention Reproductive health services Program Condom demonstration 11 dummy variables indicating whether the program Components Service learning included each of the 11 components Role plays Games Reflective exercises Mentoring/tutoring Individualized counseling Direct provision of reproductive or other health services Parent activities Community outreach Positive role model Group Size Individual Five (exclusive) dummy variables indicating standard Small group (<10) with provider format of delivery Large group or whole classroom with provider Online Other strategies Group Composition Same-gender delivery One dummy variable vs. Mixed-gender delivery Program Length Frequency of contact: One ordinal variable indicating frequency of contact Daily 3-4 times per week 1-2 times per week Less than weekly One day only Hours of contact time One continuous variable indicating intended length/intensity in number of hours Weeks from first to last contact One continuous variable indicating the number of weeks from first to last contact Level of Prior Tier 1 (evidence-based) One dummy variable Evidence vs. Tier 2 (new and innovative) Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 22 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING TABLE 1.5.2: MODERATORS RELATED TO RQ2, PROGRAM IMPLEMENTATION Moderator Typology Coding Category Program Setting Classrooms Four (exclusive) dummy variables indicating the Health clinics primary program setting Community centers Other settings Program Delivery Medical professionals Six (exclusive) dummy variables indicating the Personnel Health educators type of staff who typically delivered the Classroom teachers intervention Peer educators Other providers Mixed (no predominant provider type) Implementation Average facilitator-reported fidelity One continuous measure of average fidelity Characteristics observed across all program periods Average participant attendance rate One continuous measure of average attendance rates across all program periods (from OAH performance measures database) Participant retention rate One continuous measure of retention rates across all program periods (defined as the average proportion of participants attending 75% or more of the program sessions) (from OAH performance measures database) TABLE 1.5.3: MODERATORS RELATED TO RQ3, PARTICIPANT CHARACTERISTICS Moderator Typology Coding Category Gender Boys One dummy variable indicating the proportion of boys present in the intervention group Race/Ethnicity White Three (non-exclusive) dummy variables indicating Black the proportion of White, Black, and Hispanic Hispanic participants in the intervention group Age Average age One continuous variable indicating the average age of participants in the intervention group Sexual Risk Control/comparison group sexual activity at One continuous variable indicating the proportion Behavior post-test of participants in the comparison group who reported ever having sex at the first post-test assessment Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 23 CHAPTER 1: ELIGIBILITY, CLEANING, AND CODING TABLE 1.5.4: MODERATORS RELATED TO RQ4, STUDY METHODS Moderator Typology Coding Category Study Design Randomized experiment One dummy variable indicating whether the study vs. used a randomized experimental design Quasi-experiment Overall Attrition Attrition rate One continuous variable indicating the overall attrition rate at the first follow-up Differential Differential attrition rate One continuous variable indicating the differential Attrition attrition rate between the intervention and control/comparison groups at the first follow-up Active Active comparison One dummy variable indicating whether the study Comparison vs. used an active comparison condition Condition Inactive comparison (assessments only) Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 24 CHAPTER 2: ANALYSIS PLAN 2. Analysis Plan This chapter provides details on the study’s analysis plan. Section 2.1 provides methodological details, which were pre-specified prior to data analysis. Section 2.2 discusses deviations from the pre-specified protocol. 2.1. Methodological Specifications This section provides a detailed description of the study’s methodological specifications. 2.1.1 Aggregate Data Meta-Analysis The AD meta-analyses were conducted using a meta-regression framework with robust variance estimates (RVE), which permits the synthesis of statistically dependent effect sizes (Hedges, Tipton, and Johnson 2010; Tanner-Smith and Tipton 2014; Tipton 2013; Tipton 2015). Because studies often reported multiple (dependent) effect size estimates even for confirmatory outcomes (e.g., different operationalization of measures in the same outcome category), the RVE meta-regression model was necessary for synthesizing all available effect sizes without loss of information. The RVE meta-regression is similar in form to traditional meta-regression, which has the structure of Equation (1): (1) where is the ith effect size in the jth study; is the average population effect; is the study-level random effect such that is the between-study variance component; and is the residual for the ith effect size in the jth study. This intercept-only RVE meta-regression model is used for estimating the mean effect size , but can then be extended to examine potential effect size moderators by adding p covariates , as in Equation (2): (2) Consistent with standard meta-analysis models, the RVE meta-regression approach gives more weight to studies whose effect size estimates have greater precision, where precision is primarily driven by study sample size (Borenstein, Hedges, Higgins, and Rothstein 2010). In the RVE meta-regression approach, the weights include a within-study as well as a between-study component to the variance. The within- study component is the average variance across effect sizes within the study, and the between-study component is calculated using a method of moments estimator (Hedges, Tipton, and Johnson 2010). The RVE approach requires an assumed average correlation between effect size estimates within studies ( ) ) which we conservatively assumed to be .80. Sensitivity analyses using different assumed values of this parameter, ranging from .10 to .90, yielded robust findings (see Section 3.4 of this technical supplement; results were robust across assumed values of given the homogeneity in effect sizes in all analyses). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 25 CHAPTER 2: ANALYSIS PLAN To address our research questions, we first estimated unconditional RVE meta-regression models for each of the nine outcome categories (ever had sex, recent sexual activity, recent unprotected sexual activity, number of sexual partners, number of sexual experiences, proportion of sexual experiences that were unprotected, sexually transmitted infections, ever pregnant, recent pregnancy), where we used the intercept ( ) from the unconditional model to estimate the average effect size across studies within each outcome category and overall. We then estimated a series of RVE meta-regression models to address the research questions as to whether program design, program implementation, participant characteristics, and study methods were associated with effect size magnitude. We examined each block of moderators in a separate meta- regression model, given that the small number of included studies precluded our ability to estimate complex multivariable meta-regression models that simultaneously included all moderator variables. Because these meta-regression models often included multiple variables (e.g., 11 dummy variables measuring program component presence/absence), we used an omnibus F-statistic to assess the overall significance of each meta-regression model (Pustejovsky 2015; Tipton and Pustejovsky 2015), followed by an examination of the statistical significance of individual regression coefficients ( ). Although this modeling approach—examining one block of moderators at a time—limited our ability to control for potential confounding between different moderators, the bivariate correlations between all of the examined variables were low to moderate in size, providing some reassurances against the possibility of confounded moderators (see Section 3.5). We also report sensitivity analyses showing results from models examining one moderator variable at a time (without adjusting for other variables within a moderator block) and examining all moderators simultaneously in a single multivariable meta-regression model (see Section 3.6). 2.1.2 Individual Participant Data Meta-Analysis Whereas the standard AD meta-analysis approach can be used to examine whether study-level participant characteristics are associated with larger or smaller program effects (e.g., whether programs with higher proportions of girls are more effective), IPD meta-analysis can be used to examine whether individual- level participant characteristics are associated with program effects (e.g., whether the programs as a whole are more or less effective for girls). IPD meta-analysis can thus provide more detailed information about variability in program effects for clinically relevant subgroups by separating participant-level heterogeneity and study-level heterogeneity, something that is impossible to do in a standard AD meta- analysis that only includes study-level information. Therefore, we used IPD meta-analyses to further examine variability in TPP program effects across the participant characteristics of age, gender, race, and ethnicity. Because some evaluators did not provide IPD (and we did not request them for studies completed after October 31, 2016), the final IPD analysis model included a mixture of IPD and AD. We therefore used a one-stage approach to synthesize findings with a combination of IPD and AD (Fisher, Copas, Tierney, and Parmar 2011; Riley et al. 2008). The one-stage approach uses a multilevel logit model with the structure of Equations (3) and (4) below. 3 In this model, only the IPD trials contribute information to the 3 IPD data were consistently available for only four outcomes (ever had sex, recent sexual activity, recent unprotected sexual activity, and ever pregnant), all of which are binary measures at the participant level. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 26 CHAPTER 2: ANALYSIS PLAN analysis examining the effect of participant-level moderators, but both the IPD and AD trials contribute information to the overall average program effect as well as the between-study variance component (Riley and Steyerberg 2010): (3) (4) where is the outcome (1 = event, 0 = non-event) of participant k in study j; is a dummy variable indicating whether study j provided IPD or AD data (1 = IPD, 0 = AD only); is a dummy variable indicating whether participant k in study j was in the treatment or comparison group (1 = TPP group, 0 = comparison group); and is a participant-level covariate (i.e., gender, race, ethnicity, or age). The parameter estimates the average TPP program effect, estimates the within-study effect of the participant-level covariate, estimates the between-study effect of the participant-level covariate, estimates the interaction between the TPP program effect and within-study participant-level covariate, and estimates the interaction between the TPP program effect and between-study participant-level covariate. The coefficients for , , and are treated as random, to permit variability in program effects and program by covariate interactions across studies (see Section 3.3 for subgroup findings from each study contributing IPD data). 2.1.3 Analysis of Program Attendance and Retention The study’s fifth research question explores the extent to which participant attendance and retention were affected by program characteristics. To address this research question, we used linear regression models to predict the two continuously measured outcomes of participant attendance and retention. Using a parallel approach to the meta-regression model described in Equation (2), we estimated a series of regression models examining blocks of moderators related to program focus, components, group size, group composition, gender specificity, program length, program setting, delivery personnel, implementation fidelity, and participant characteristics. Again, because these regression models often included multiple variables (e.g., several dummy variables measuring program component presence/absence), we used an omnibus F-statistic to assess the overall significance of each regression model, followed by an examination of the statistical significance of individual regression coefficients. 2.2. Deviations from Pre-Specified Analysis Protocol Our final analysis deviated from the original protocol in a few ways. First, our fourth research question (RQ4) originally included an additional study design moderator, missing data handling, intended to capture the analytic methods used by evaluation teams to handle missing data (and whether those were modern methods such as multiple imputation/full information maximum likelihood, or less preferred methods such as listwise/pairwise deletion and/or dummy variable imputation). We dropped this moderator from the final analysis because many studies did not report the methods for handling missing data or reported multiple methods for handling missing data (e.g., dummy variable imputation approaches Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 27 CHAPTER 2: ANALYSIS PLAN combined with pairwise deletion). Furthermore, only one study reported using a modern method (multiple imputation). It appears this was because the technical assistance provided to grantees encouraged them to use techniques such as listwise deletion, pairwise deletion, or dummy variable imputation. Given that we had no directional hypotheses regarding how missing data handling might moderate effect size, and given the imprecision in measurement of this variable, we ultimately elected to drop this variable from the final analysis. Second, the original protocol for RQ4 specified another study design moderator: whether authors conducted an intent-to-treat (ITT) analysis or a treatment-on-the-treated analysis (TOT). 4 We dropped this moderator from the final analysis due to inconsistent reporting and the nature of the technical assistance provided to grantees: Because the TA provider for many of the grantees encouraged all of them to conduct ITT analysis, they may have conducted an ITT analysis but not reported it explicitly. Third, the original protocol for RQ4 specified another study design moderator: potential risk of bias due to random sequence generation. We dropped this moderator from the final analysis because it was perfectly collinear with study design, such that all randomized experiments were deemed at low risk of bias due to random sequence generation, whereas all studies using non-randomized quasi-experimental designs were deemed at high risk of bias due to sequence generation. Fourth, the original protocol did not include the implementation characteristics of fidelity, attendance, and retention as moderators of interest in RQ2. This was an unintentional omission from the protocol, so our final analysis included these three implementation variables as potential moderators of effect sizes. Fifth, our protocol suggested that multivariable meta-regression models might be used in the AD meta- analysis to examine the effect of each moderator variable after adjusting for all other candidate moderators. As noted previously, this procedure was ultimately not feasible given the limited sample size available for fitting such models. As a result, we opted to instead examine each moderator block simultaneously while also assessing bivariate correlations between moderators to assess for potential confounding. Sixth, our protocol stated that we would examine participants’ baseline sexual activity as a moderator in both the AD and IPD meta-analyses. Ultimately, too few studies measured or reported participants’ baseline sexual activity (either in their final evaluation reports or in the IPD data provided) for us to include this as an effect size moderator in our analysis. To address this limitation, we added an additional moderator variable to the analysis, the control group event rate for sexual behavior at post-test, which we included as a crude proxy for the risk level or sexual experience rates of the sample. Finally, our original protocol implied that the meta-analysis would calculate averages across all effect sizes from each study. Prior to our final analysis, an expert panel convened to review the meta-analysis research design recommended that the primary analysis use only the confirmatory outcomes from each study. The expert panel’s concern was that by including all of the outcomes that studies reported—some 4 An intent to treat analysis captures impacts for all sample members, regardless of whether those assigned to the treatment group actually received the program’s services. In other words, it assesses whether the existence of the program led to better outcomes for those offered the chance to participate in it, relative to what they could have obtained without the program. For a voluntary (rather than mandatory) program, the intent to treat estimate is often the most policy relevant. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 28 CHAPTER 2: ANALYSIS PLAN of which might not be very relevant to the programs being evaluated—favorable impacts on key outcomes might be watered down. In theory, looking only at confirmatory outcomes should mitigate this concern to the degree that the study evaluators, after careful consideration, chose confirmatory outcomes that were well aligned with their programs’ logic models and thus amenable to change. To address this recommendation, our Final Report presents results from both types of analysis, but with the analysis of confirmatory outcomes considered primary. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 29 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3. Additional Results and Sensitivity Analyses This chapter provides detailed results for the full sample and for subgroups that supplement those discussed in the report, as well as sensitivity analyses. Sections 3.1–3.3 present detailed results for the AD sample (Section 3.1) and the IPD sample (Sections 3.2 and 3.3) that supplement those presented in the main report. Sections 3.4–3.9 present sensitivity analyses exploring alternate model specifications and assumptions. 3.1. Distribution of Synthesized Effect Sizes and Statistical Findings by Outcome This appendix provides histograms displaying the distribution of effect sizes for each outcome construct reported in Chapter 5 of the report (Overall Effects of the Evaluated Programs). Ever had sex. Figure 3.1.1 shows the distribution of effect sizes from the studies that reported a confirmatory impact for the odds of ever having sex. These 22 studies reported n = 26 effect sizes indexing program effects on lifetime sexual activity, so the histogram includes multiple effect sizes from each study (when available). All effect sizes were coded such that log odds ratios (LOR) greater than zero indicate a beneficial program effect. FIGURE 3.1.1: DISTRIBUTION OF PROGRAM EFFECTS FOR ODDS OF EVER HAVING SEX Notes. Figure 3.1.1 shows the distribution of log odds ratios across all 22 studies that reported at least one confirmatory effect size in the outcome category of ever had sex. Some studies reported multiple effect sizes in this category (e.g., different operational definitions or multiple follow-ups), so the distribution includes all available effect sizes from each study. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., never engaged in sexual activity). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 30 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES As shown in Figure 3.1.1 above, effect sizes for this outcome category are narrowly clustered around the mean effect size, which was small and marginally statistically significant (LOR = 0.07, 95% CI [−0.01, 0.14]; k = 22, n = 26). This indicates that, on average, these TPP programs had slightly more beneficial effects on lifetime sexual activity than did the comparison conditions. Moreover, these null program effects were remarkably homogeneous across studies (τ2 = 0.00, I2 = 0%). Recent sexual activity. Figure 3.1.2 shows the distribution of effect sizes from the 17 studies that measured participants’ recent sexual activity after receipt of the TPP programs. Effect sizes are clustered around zero, suggesting that there were no differences between the TPP and comparison conditions (LOR = −0.05, 95% CI [−0.18, 0.08]; k = 17, n = 26). The average percentage of participants who reported no recent sexual activity was 60 percent in the TPP conditions and 60 percent in the comparison conditions. These (null) program effects were also homogeneous across studies (τ2 = 0.05, I2 = 59.87%). FIGURE 3.1.2: DISTRIBUTION OF PROGRAM EFFECTS FOR ODDS OF RECENT SEXUAL ACTIVITY Notes. Figure 3.1.2 shows the distribution of log odds ratios across all 17 studies that reported at least one confirmatory effect size in the outcome category of recent sexual activity. Several studies reported multiple effect sizes in this outcome category (e.g., different operational definitions or multiple follow-ups), so the distribution includes all available effect sizes from each study. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., no recent sexual activity). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 31 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent unprotected sexual activity. The distribution of effect sizes for this outcome category is shown in Figure 3.1.3. Similar to the results for recent sexual activity, the mean effect size for the odds of having recent unprotected sex was not statistically significant (LOR = 0.05, 95% CI [−0.04, 0.15]; k = 32, n = 48). Although this mean effect size was positive in direction (indicating beneficial effects for TPP participants), it was small and statistically non-significant—whereas 83 percent of TPP participants reported no recent unprotected sexual activity, 82 percent of comparison participants reported no recent unprotected sex either. Again, these null findings were homogeneous across studies (τ2 = 0.00, I2 = 0%). FIGURE 3.1.3: DISTRIBUTION OF PROGRAM EFFECTS FOR ODDS OF RECENT UNPROTECTED SEXUAL ACTIVITY Notes. Figure 3.1.3 shows the distribution of recent unprotected sexual activity across all 32 studies that reported at least one confirmatory outcome in the category. Some studies reported multiple effect sizes in this category (e.g., different operational definitions or multiple follow- ups), so the distribution includes all available effect sizes from each study. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., no recent unprotected sexual activity). Proportion of sexual experiences that were unprotected. Only one study reported an effect size for this outcome category, which was not statistically significant (LOR = −0.29, 95% CI [−0.85, 0.27]). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 32 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Ever pregnant. Figure 3.1.4 shows the distribution of effect sizes for this outcome category. The mean effect size for the odds of any lifetime pregnancy was not statistically significant (LOR = 0.19, 95% CI [−0.68, 1.06]; k = 4, n = 4). This indicates that, on average, these TPP programs did not have more or less beneficial effects on lifetime pregnancy than did the comparison conditions. These null program effects were relatively homogeneous (τ2 = 0.13, I2 = 68.73%). FIGURE 3.1.4: DISTRIBUTION OF PROGRAM EFFECTS FOR ODDS OF ANY PREGNANCY Notes. Figure 3.1.4 shows the distribution of log odds ratios across all 4 studies that reported at least one confirmatory effect size in the outcome category of ever pregnant. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., never pregnant). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 33 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent pregnancy. Figure 3.1.5 shows the distribution of effect sizes for this outcome category. The mean effect size was positive in direction (i.e., favorable) and statistically significant (LOR = 0.26, 95% CI [0.00, 0.52]; k = 12, n = 12). Among studies reporting recent pregnancy as a confirmatory outcome, 87 percent of TPP participants reported no recent pregnancies, 84 percent of comparison participants reported no recent pregnancies. These program effects were homogeneous across studies (τ2 = 0.08, I2 = 54.77%). FIGURE 3.1.5: DISTRIBUTION OF PROGRAM EFFECTS FOR ODDS OF RECENT PREGNANCY Notes. Figure 3.1.5 shows the distribution of log odds ratios across all 12 studies that reported at least one confirmatory effect size in the outcome category of recent pregnancy. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., not recently pregnant). Number of sexual partners. Only two studies reported a confirmatory effect size in this outcome category. The mean effect size was not statistically significant (Hedges’ g = 0.08, 95% CI [−1.27, 1.44], k = 2, n = 2, τ2 = 0.00, I2 = 20.76%). This indicates that, on average, TPP programs did not lead to fewer (or more) sexual partners relative to the comparison conditions. 3.2. IPD Analysis Detailed Results Using IPD, we were able to examine impacts for subgroups of participants defined by gender, race/ethnicity, and age. IPD were consistently available for four confirmatory outcomes: ever had sex, recent sexual activity, recent unprotected sexual activity, and ever pregnant (i.e., pregnancy for girls, causing pregnancy for boys). 5 The TPP program effects on each of these four outcomes for each 5 We also ran these analyses using all available outcomes for each study; results were similar. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 34 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES participant subgroup are displayed in Figures 3.2.1 through 3.2.4. Each of these figures draws on data from only the studies available prior to October 31, 2016, that provided IPD. In each of the figures, the average treatment effect size for each subgroup of participants is expressed as a log odds ratio, where a positive number indicates an effect favoring the treatment group and a negative number indicates an effect favoring the comparison group. The figures include 95 percent confidence intervals for each estimate of the treatment effect size. The 95 percent confidence intervals for each of the subgroups in each of the figures include lower confidence limits that are less than zero, indicating a non- trivial probability that the true effect is negative, and upper limits that are above zero, indicating a non- trivial probability that the true effect is positive. We therefore conclude that for these four outcomes, program impacts are not significantly different from zero for any of the participant subgroups examined. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 35 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES FIGURE 3.2.1: EVER HAD SEX: PROGRAM EFFECTS FOR PARTICIPANT SUBGROUPS Notes. Results from 14 studies including 15,585 individual participants. Log odds ratios and 95 percent confidence intervals shown for each subgroup. Given that many studies reported multiple effect sizes, this figure displays the average (synthetic) mean effect size for each subgroup. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., never had sex). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 36 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES FIGURE 3.2.2: RECENT SEXUAL ACTIVITY: PROGRAM EFFECTS FOR PARTICIPANT SUBGROUPS Notes. Results from 13 studies including 11,627 individual participants. Log odds ratios and 95 percent confidence intervals shown for each subgroup. Given that many studies reported multiple effect sizes, this figure displays the average (synthetic) mean effect size for each subgroup. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., no recent sex). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 37 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES FIGURE 3.2.3: RECENT UNPROTECTED SEXUAL ACTIVITY: PROGRAM EFFECTS FOR PARTICIPANT SUBGROUPS Notes. Results from 21 studies including 19,175 individual participants. Log odds ratios and 95 percent confidence intervals shown for each subgroup. Given that many studies reported multiple effect sizes, this figure displays the average (synthetic) mean effect size for each subgroup. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., no recent unprotected sex). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 38 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES FIGURE 3.2.4: EVER PREGNANT: PROGRAM EFFECTS FOR PARTICIPANT SUBGROUPS Notes. Results from 3 studies including 10,111 individual participants. Log odds ratios and 95 percent confidence intervals shown for each subgroup. Given that many studies reported multiple effect sizes, this figure displays the average (synthetic) mean effect size for each subgroup. All effect sizes coded such that log odds ratios greater than zero indicate a beneficial effect of the program (i.e., any pregnancy or parenting). Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 39 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.3. Subgroup Effects from IPD Meta-Analysis This section presents subgroup findings from each study contributing IPD for each of the study’s four IPD outcomes (ever had sex, recent sexual activity, recent unprotected sexual activity, ever pregnant). Table 3.3.1 presents subgroup effects by participant gender, Table 3.3.2 presents subgroup effects by ethnicity, and Table 3.3.3 presents subgroup effects by race. These tables present subgroup effects for all available outcomes for each study. Confirmatory outcomes are indicated using bold text. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 40 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.3.1: SUBGROUP EFFECTS BY PARTICIPANT GENDER Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Abe et al. (2016) Comparison Girls 286 8.74% 91.26% 286 4.55% 95.45% 286 2.80% 97.20% 286 0.35% 99.65% Comparison Boys 244 13.11% 86.89% 247 6.48% 93.52% 247 2.43% 97.57% 247 0.81% 99.19% Intervention Girls 497 8.65% 91.35% 498 5.22% 94.78% 499 2.00% 98.00% 499 1.40% 98.60% Intervention Boys 461 10.20% 89.80% 465 6.02% 93.98% 465 1.51% 98.49% 464 2.37% 97.63% Abt Associates (2016a) [AZ] Comparison Girls 163 9.82% 90.18% 163 6.13% 93.87% 163 4.91% 95.09% 162 0.62% 99.38% Comparison Boys 185 10.81% 89.19% 185 5.95% 94.05% 185 3.24% 96.76% 184 0.00% 100.00% Intervention Girls 265 10.57% 89.43% 265 6.42% 93.58% 265 4.15% 95.85% 265 0.38% 99.62% Intervention Boys 225 17.33% 82.67% 225 8.44% 91.56% 225 7.56% 92.44% 225 0.44% 99.56% Abt Associates (2016a) [CA] Comparison Girls 122 32.79% 67.21% 122 22.13% 77.87% 122 16.39% 83.61% 122 1.64% 98.36% Comparison Boys 80 46.25% 53.75% 80 27.50% 72.50% 80 21.25% 78.75% 80 2.50% 97.50% Intervention Girls 175 40.57% 59.43% 175 26.29% 73.71% 175 21.71% 78.29% 175 4.00% 96.00% Intervention Boys 109 46.79% 53.21% 109 34.86% 65.14% 109 30.28% 69.72% 109 1.83% 98.17% Abt Associates (2016a) [MA] Comparison Girls 139 36.69% 63.31% 139 22.30% 77.70% 139 17.27% 82.73% 139 2.88% 97.12% Comparison Boys 113 51.33% 48.67% 113 37.17% 62.83% 113 30.97% 69.03% 113 7.08% 92.92% Intervention Girls 250 46.40% 53.60% 250 34.40% 65.60% 250 28.80% 71.20% 250 6.40% 93.60% Intervention Boys 186 47.31% 52.69% 185 28.11% 71.89% 186 20.97% 79.03% 186 3.76% 96.24% Abt Associates (2016b) [CA] Comparison Girls 197 18.27% 81.73% 196 11.22% 88.78% 197 9.14% 90.86% 197 0.51% 99.49% Comparison Boys 186 22.58% 77.42% 186 13.44% 86.56% 186 11.29% 88.71% 185 2.70% 97.30% Intervention Girls 269 20.07% 79.93% 269 12.27% 87.73% 269 10.78% 89.22% 269 0.74% 99.26% Intervention Boys 234 29.49% 70.51% 233 16.74% 83.26% 233 12.45% 87.55% 233 0.43% 99.57% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 41 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Abt Associates (2016b) [IL & MO] Comparison Girls 174 43.68% 56.32% 173 30.64% 69.36% 173 21.39% 78.61% 174 10.34% 89.66% Comparison Boys 193 68.39% 31.61% 192 51.04% 48.96% 193 37.82% 62.18% 193 14.51% 85.49% Intervention Girls 272 50.74% 49.26% 272 31.99% 68.01% 272 21.32% 78.68% 271 7.75% 92.25% Intervention Boys 297 62.63% 37.37% 296 42.91% 57.09% 296 26.01% 73.99% 295 8.14% 91.86% Abt Associates (2016b) [TX] Comparison Girls 197 41.62% 58.38% 197 29.44% 70.56% 197 27.41% 72.59% 197 7.11% 92.89% Comparison Boys 211 54.50% 45.50% 211 32.70% 67.30% 211 30.33% 69.67% 211 5.21% 94.79% Intervention Girls 215 48.37% 51.63% 215 34.42% 65.58% 215 29.77% 70.23% 215 5.12% 94.88% Intervention Boys 225 57.33% 42.67% 225 37.33% 62.67% 225 30.22% 69.78% 225 6.67% 93.33% Abt Associates (2016c) [FL] Comparison Girls 146 85.62% 14.38% 146 72.60% 27.40% 146 61.64% 38.36% 146 15.75% 84.25% Comparison Boys 0 0 0 0 Intervention Girls 280 89.64% 10.36% 280 72.86% 27.14% 280 62.86% 37.14% 280 25.36% 74.64% Intervention Boys 0 0 0 0 Abt Associates (2016c) [MN] Comparison Girls 656 91.77% 8.23% 656 78.66% 21.34% 656 71.65% 28.35% 652 20.86% 79.14% Comparison Boys 0 0 0 0 Intervention Girls 1274 90.27% 9.73% 1270 76.06% 23.94% 1272 66.67% 33.33% 1272 22.48% 77.52% Intervention Boys 0 0 0 0 Abt Associates (2016c) [TN] Comparison Girls 137 89.78% 10.22% 137 68.61% 31.39% 137 62.77% 37.23% 137 21.90% 78.10% Comparison Boys 0 0 0 0 Intervention Girls 275 89.09% 10.91% 275 72.73% 27.27% 275 61.45% 38.55% 275 22.91% 77.09% Intervention Boys 0 0 0 0 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 42 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Advanced Empirical Solutions (2015) Comparison Girls 307 0.33% 99.67% * 307 0.00% 100.00% Comparison Boys 0 * 0 Intervention Girls 294 0.00% 100.00% * 294 0.00% 100.00% Intervention Boys 0 * * 0 Calise et al. (2015) Comparison Girls 294 10.20% 89.80% 290 5.86% 94.14% 289 3.46% 96.54% 289 0.35% 99.65% Comparison Boys 302 14.57% 85.43% 292 6.51% 93.49% 290 3.79% 96.21% 290 1.38% 98.62% Intervention Girls 213 5.16% 94.84% 210 2.86% 97.14% 210 1.90% 98.10% 211 0.00% 100.00% Intervention Boys 251 12.75% 87.25% 244 6.56% 93.44% 244 4.10% 95.90% 243 1.65% 98.35% Carter et al. (2015) Comparison Girls 155 1.94% 98.06% * * * Comparison Boys 115 1.74% 98.26% * * * Intervention Girls 113 1.77% 98.23% * * * Intervention Boys 96 3.13% 96.88% * * * Coyle et al. (2015) Comparison Girls 643 16.95% 83.05% * * 646 2.01% 97.99% Comparison Boys 605 26.45% 73.55% * * 607 1.81% 98.19% Intervention Girls 806 14.89% 85.11% * * 807 1.36% 98.64% Intervention Boys 665 24.96% 75.04% * * 666 1.80% 98.20% Coyle et al. (2016) Comparison Girls 458 12.88% 87.12% * * * Comparison Boys 443 30.70% 69.30% * * * Intervention Girls 487 9.24% 90.76% * * * Intervention Boys 452 25.22% 74.78% * * * Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 43 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Crean et al. (2016) Comparison Girls 146 9.59% 90.41% 189 4.23% 95.77% 188 2.13% 97.87% 189 0.53% 99.47% Comparison Boys 158 27.85% 72.15% 220 11.36% 88.64% 219 6.39% 93.61% 227 0.88% 99.12% Intervention Girls 238 8.82% 91.18% 301 2.66% 97.34% 300 2.33% 97.67% 301 0.66% 99.34% Intervention Boys 189 21.16% 78.84% 260 8.46% 91.54% 257 1.56% 98.44% 264 0.76% 99.24% Cunningham et al. (2016) [LN] Comparison Girls 410 36.10% 63.90% 410 23.90% 76.10% 410 17.07% 82.93% 410 2.44% 97.56% Comparison Boys 244 36.07% 63.93% 244 25.41% 74.59% 246 16.26% 83.74% 244 3.28% 96.72% Intervention Girls 476 31.09% 68.91% 476 18.07% 81.93% 476 15.55% 84.45% 476 0.84% 99.16% Intervention Boys 238 39.50% 60.50% 238 26.05% 73.95% 240 16.67% 83.33% 240 4.17% 95.83% Cunningham et al. (2016) [RtR] Comparison Girls 410 36.10% 63.90% 410 23.90% 76.10% 410 17.07% 82.93% 410 2.44% 97.56% Comparison Boys 244 36.07% 63.93% 244 25.41% 74.59% 246 16.26% 83.74% 244 3.28% 96.72% Intervention Girls 482 23.65% 76.35% 482 15.35% 84.65% 482 11.62% 88.38% 482 0.83% 99.17% Intervention Boys 276 42.03% 57.97% 276 23.91% 76.09% 276 16.67% 83.33% 276 2.17% 97.83% Daley et al. (2015) Comparison Girls 999 35.54% 64.46% 975 23.38% 76.62% 873 17.75% 82.25% 993 3.32% 96.68% Comparison Boys 968 39.98% 60.02% 919 22.85% 77.15% 780 13.97% 86.03% 954 2.41% 97.59% Intervention Girls 812 31.90% 68.10% 747 15.93% 84.07% 667 10.04% 89.96% 779 2.82% 97.18% Intervention Boys 799 36.80% 63.20% 744 17.47% 82.53% 632 12.18% 87.82% 802 4.36% 95.64% Dierschke et al. (2015) Comparison Girls 200 56.50% 43.50% 200 39.00% 61.00% 200 28.00% 72.00% 199 5.03% 94.97% Comparison Boys 199 59.30% 40.70% 199 37.19% 62.81% 199 24.62% 75.38% 198 7.58% 92.42% Intervention Girls 214 58.41% 41.59% 214 40.19% 59.81% 214 33.18% 66.82% 214 5.61% 94.39% Intervention Boys 190 59.47% 40.53% 190 34.74% 65.26% 190 22.63% 77.37% 190 4.21% 95.79% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 44 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Eichner et al. (2015) Comparison Girls * 343 79.59% 20.41% 343 63.56% 36.44% * Comparison Boys * 0 0 * Intervention Girls * 342 80.12% 19.88% 342 61.70% 38.30% * Intervention Boys * 0 0 * Francis et al. (2015) Comparison Girls 254 19.69% 80.31% 253 15.02% 84.98% 253 9.49% 90.51% * Comparison Boys 202 18.81% 81.19% 202 11.39% 88.61% 202 6.44% 93.56% * Intervention Girls 408 22.30% 77.70% 406 15.76% 84.24% 405 9.14% 90.86% * Intervention Boys 335 25.67% 74.33% 334 14.67% 85.33% 334 7.49% 92.51% * Herrling (2016) Comparison Girls 67 16.42% 83.58% 67 10.45% 89.55% 63 4.76% 95.24% 67 2.99% 97.01% Comparison Boys 66 39.39% 60.61% 66 22.73% 77.27% 55 18.18% 81.82% 65 0.00% 100.00% Intervention Girls 77 18.18% 81.82% 76 13.16% 86.84% 73 12.33% 87.67% 77 3.90% 96.10% Intervention Boys 57 43.86% 56.14% 57 28.07% 71.93% 48 10.42% 89.58% 56 5.36% 94.64% Kissinger et al. (2015) Comparison Girls * 268 58.21% 41.79% 131 51.91% 48.09% * Comparison Boys * 0 0 * Intervention Girls * 263 56.27% 43.73% 124 48.39% 51.61% * Intervention Boys * 0 0 * Philliber et al. (2016) Comparison Girls 1929 31.47% 68.53% 1923 22.36% 77.64% 1756 15.21% 84.79% 1930 5.75% 94.25% Comparison Boys 1415 30.95% 69.05% 1409 21.36% 78.64% 1287 11.42% 88.58% 1416 3.32% 96.68% Intervention Girls 2053 34.34% 65.66% 2046 25.66% 74.34% 1872 17.31% 82.69% 2053 8.96% 91.04% Intervention Boys 1500 31.53% 68.47% 1495 21.27% 78.73% 1346 11.52% 88.48% 1501 2.80% 97.20% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 45 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Philliber & Philliber (2016) Comparison Girls 228 20.61% 79.39% 228 8.77% 91.23% 201 4.98% 95.02% 225 3.56% 96.44% Comparison Boys 180 36.67% 63.33% 180 24.44% 75.56% 158 13.92% 86.08% 175 2.86% 97.14% Intervention Girls 304 20.39% 79.61% 304 11.84% 88.16% 278 6.12% 93.88% 301 2.99% 97.01% Intervention Boys 221 35.29% 64.71% 221 20.36% 79.64% 188 11.70% 88.30% 214 2.80% 97.20% Piotrowski et al. (2015) Comparison Girls 347 4.03% 95.97% 347 2.02% 97.98% 346 1.16% 98.84% 346 0.29% 99.71% Comparison Boys 324 9.88% 90.12% 324 7.10% 92.90% 322 3.11% 96.89% 323 0.31% 99.69% Intervention Girls 408 1.96% 98.04% 408 0.25% 99.75% 407 0.25% 99.75% 408 0.00% 100.00% Intervention Boys 376 4.26% 95.74% 376 3.19% 96.81% 376 2.39% 97.61% 375 1.07% 98.93% Robinson et al. (2016) Comparison Girls 642 28.04% 71.96% 639 18.00% 82.00% 646 10.06% 89.94% 636 5.35% 94.65% Comparison Boys 403 40.69% 59.31% 407 24.57% 75.43% 408 12.01% 87.99% 396 3.54% 96.46% Intervention Girls 583 29.33% 70.67% 574 16.90% 83.10% 582 10.65% 89.35% 574 4.88% 95.12% Intervention Boys 389 43.44% 56.56% 387 25.84% 74.16% 398 9.05% 90.95% 378 3.17% 96.83% Rotz et al. (2016) Comparison Girls 305 39.02% 60.98% 303 35.64% 64.36% 303 36.96% 63.04% 301 5.32% 94.68% Comparison Boys 230 49.13% 50.87% 227 44.05% 55.95% 227 45.37% 54.63% 220 5.45% 94.55% Intervention Girls 531 37.10% 62.90% 529 33.65% 66.35% 529 34.22% 65.78% 524 2.48% 97.52% Intervention Boys 419 42.00% 58.00% 402 36.57% 63.43% 402 34.83% 65.17% 397 0.25% 99.75% Slater et al. (2015) Comparison Girls 229 82.53% 17.47% 224 64.29% 35.71% 230 53.04% 46.96% 227 25.11% 74.89% Comparison Boys 247 82.19% 17.81% 241 68.88% 31.12% 248 53.63% 46.37% 244 18.85% 81.15% Intervention Girls 235 83.83% 16.17% 230 67.83% 32.17% 236 55.08% 44.92% 232 28.45% 71.55% Intervention Boys 242 90.50% 9.50% 236 70.34% 29.66% 243 50.62% 49.38% 236 17.37% 82.63% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 46 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Gender N Yes No N Yes No N Yes No N Yes No Smith et al. (2015) Comparison Girls * 244 86.48% 13.52% 244 90.98% 9.02% 268 97.39% 2.61% Comparison Boys * 0 0 0 Intervention Girls * 249 83.53% 16.47% 249 93.98% 6.02% 271 97.79% 2.21% Intervention Boys * 0 0 0 Smith et al. (2016) Comparison Girls 153 30.72% 69.28% 141 17.73% 82.27% 131 13.74% 86.26% 153 5.23% 94.77% Comparison Boys 165 36.97% 63.03% 153 22.22% 77.78% 138 10.14% 89.86% 164 3.05% 96.95% Intervention Girls 217 29.49% 70.51% 205 19.02% 80.98% 192 9.90% 90.10% 216 6.02% 93.98% Intervention Boys 210 38.57% 61.43% 195 23.08% 76.92% 175 9.71% 90.29% 209 3.35% 96.65% The Policy & Research Group (2015) Comparison Girls 176 25.00% 75.00% 174 17.24% 82.76% 168 13.10% 86.90% 174 2.30% 97.70% Comparison Boys 169 49.11% 50.89% 156 30.13% 69.87% 146 14.38% 85.62% 164 3.05% 96.95% Intervention Girls 172 29.65% 70.35% 170 18.82% 81.18% 160 10.63% 89.38% 172 3.49% 96.51% Intervention Boys 171 49.71% 50.29% 163 25.77% 74.23% 149 8.05% 91.95% 166 4.82% 95.18% Vyas et al. (2015) Comparison Girls 210 23.81% 76.19% 210 14.76% 85.24% 210 10.48% 89.52% 212 2.36% 97.64% Comparison Boys 138 50.72% 49.28% 138 30.43% 69.57% 136 11.03% 88.97% 136 4.41% 95.59% Intervention Girls 247 26.72% 73.28% 246 17.48% 82.52% 246 11.79% 88.21% 249 4.02% 95.98% Intervention Boys 191 48.69% 51.31% 189 22.75% 77.25% 189 9.52% 90.48% 191 1.57% 98.43% Walker et al. (2016) Comparison Girls 172 1.16% 98.84% 170 0.00% 100.00% 170 0.00% 100.00% 172 0.00% 100.00% Comparison Boys 148 4.05% 95.95% 142 0.00% 100.00% 142 0.00% 100.00% 146 0.00% 100.00% Intervention Girls 206 0.49% 99.51% 206 0.00% 100.00% 206 0.00% 100.00% 206 0.00% 100.00% Intervention Boys 178 3.37% 96.63% 177 2.26% 97.74% 176 1.70% 98.30% 178 2.25% 97.75% AZ = Arizona, CA = California, FL = Florida, IL = Illinois, LN = Love Notes, MA = Massachusetts, MN = Minnesota, MO = Missouri, RTR = Reducing the Risk, TN = Tennessee TX = Texas. Notes. The presence of an asterisk (*) indicates that this outcome was not reported at the first post-test. Bold text indicates that the outcome was selected as confirmatory. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 47 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.3.2: SUBGROUP EFFECTS BY ETHNICITY Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Abt Associates (2016a) [AZ] Comparison Hispanic 254 9.84% 90.16% 254 7.09% 92.91% 254 5.12% 94.88% 253 0.00% 100.00% Non- Comparison 83 13.25% 86.75% 83 3.61% 96.39% 83 1.20% 98.80% 82 1.22% 98.78% Hispanic Intervention Hispanic 363 11.29% 88.71% 363 6.06% 93.94% 363 4.96% 95.04% 363 0.28% 99.72% Non- Intervention 114 19.30% 80.70% 114 10.53% 89.47% 114 7.89% 92.11% 114 0.88% 99.12% Hispanic Abt Associates (2016a) [CA] Comparison Hispanic 105 33.33% 66.67% 105 23.81% 76.19% 105 20.95% 79.05% 105 3.81% 96.19% Non- Comparison 97 43.30% 56.70% 97 24.74% 75.26% 97 15.46% 84.54% 97 0.00% 100.00% Hispanic Intervention Hispanic 147 41.50% 58.50% 147 26.53% 73.47% 147 21.77% 78.23% 147 2.04% 97.96% Non- Intervention 135 44.44% 55.56% 135 33.33% 66.67% 135 28.89% 71.11% 135 4.44% 95.56% Hispanic Abt Associates (2016a) [MA] Comparison Hispanic 199 41.71% 58.29% 199 29.15% 70.85% 199 22.61% 77.39% 199 4.52% 95.48% Non- Comparison 53 49.06% 50.94% 53 28.30% 71.70% 53 26.42% 73.58% 53 5.66% 94.34% Hispanic Intervention Hispanic 353 46.74% 53.26% 352 33.24% 66.76% 353 26.63% 73.37% 353 5.67% 94.33% Non- Intervention 83 46.99% 53.01% 83 25.30% 74.70% 83 20.48% 79.52% 83 3.61% 96.39% Hispanic Abt Associates (2016b) [CA] Comparison Hispanic 267 20.22% 79.78% 266 13.16% 86.84% 267 10.49% 89.51% 266 1.88% 98.12% Non- Comparison 107 18.69% 81.31% 107 8.41% 91.59% 107 8.41% 91.59% 107 0.00% 100.00% Hispanic Intervention Hispanic 331 22.96% 77.04% 331 12.99% 87.01% 331 9.67% 90.33% 330 0.61% 99.39% Non- Intervention 165 27.27% 72.73% 164 16.46% 83.54% 164 14.63% 85.37% 165 0.61% 99.39% Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 48 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Abt Associates (2016b) [IL & MO] Comparison Hispanic 10 50.00% 50.00% 10 50.00% 50.00% 10 50.00% 50.00% 10 10.00% 90.00% Non- Comparison 352 57.10% 42.90% 350 41.14% 58.86% 351 29.91% 70.09% 352 12.78% 87.22% Hispanic Intervention Hispanic 14 35.71% 64.29% 14 21.43% 78.57% 14 14.29% 85.71% 14 7.14% 92.86% Non- Intervention 550 57.45% 42.55% 549 37.89% 62.11% 549 23.86% 76.14% 547 8.04% 91.96% Hispanic Abt Associates (2016b) [TX] Comparison Hispanic 265 49.06% 50.94% 265 32.08% 67.92% 265 28.68% 71.32% 265 8.30% 91.70% Non- Comparison 142 47.18% 52.82% 142 29.58% 70.42% 142 29.58% 70.42% 142 2.11% 97.89% Hispanic Intervention Hispanic 273 52.75% 47.25% 273 38.10% 61.90% 273 30.77% 69.23% 273 6.96% 93.04% Non- Intervention 166 53.01% 46.99% 166 31.93% 68.07% 166 28.31% 71.69% 166 4.22% 95.78% Hispanic Abt Associates (2016c) [FL] Comparison Hispanic 42 95.24% 4.76% 42 76.19% 23.81% 42 66.67% 33.33% 42 26.19% 73.81% Non- Comparison 104 81.73% 18.27% 104 71.15% 28.85% 104 59.62% 40.38% 104 11.54% 88.46% Hispanic Intervention Hispanic 75 92.00% 8.00% 75 72.00% 28.00% 75 57.33% 42.67% 75 26.67% 73.33% Non- Intervention 205 88.78% 11.22% 205 73.17% 26.83% 205 64.88% 35.12% 205 24.88% 75.12% Hispanic Abt Associates (2016c) [MN] Comparison Hispanic 140 90.00% 10.00% 140 71.43% 28.57% 140 65.71% 34.29% 138 24.64% 75.36% Non- Comparison 516 92.25% 7.75% 516 80.62% 19.38% 516 73.26% 26.74% 514 19.84% 80.16% Hispanic Intervention Hispanic 200 86.00% 14.00% 200 66.00% 34.00% 200 57.00% 43.00% 200 19.00% 81.00% Non- Intervention 1074 91.06% 8.94% 1070 77.94% 22.06% 1072 68.47% 31.53% 1072 23.13% 76.87% Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 49 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Abt Associates (2016c) [TN] Comparison Hispanic 10 80.00% 20.00% 10 60.00% 40.00% 10 60.00% 40.00% 10 20.00% 80.00% Non- Comparison 127 90.55% 9.45% 127 69.29% 30.71% 127 62.99% 37.01% 127 22.05% 77.95% Hispanic Intervention Hispanic 23 91.30% 8.70% 23 73.91% 26.09% 23 65.22% 34.78% 23 39.13% 60.87% Non- Intervention 252 88.89% 11.11% 252 72.62% 27.38% 252 61.11% 38.89% 252 21.43% 78.57% Hispanic Advanced Empirical Solutions (2015) Comparison Hispanic 207 0.48% 99.52% * * 207 0.00% 100.00% Non- Comparison 45 0.00% 100.00% * * 45 0.00% 100.00% Hispanic Intervention Hispanic 184 0.00% 100.00% * * 184 0.00% 100.00% Non- Intervention 57 0.00% 100.00% * * 57 0.00% 100.00% Hispanic Calise et al. (2015) Comparison Hispanic 237 19.41% 80.59% 227 9.69% 90.31% 224 5.80% 94.20% 226 1.77% 98.23% Non- Comparison 359 7.80% 92.20% 355 3.94% 96.06% 355 2.25% 97.75% 353 0.28% 99.72% Hispanic Intervention Hispanic 186 9.68% 90.32% 183 7.10% 92.90% 183 5.46% 94.54% 183 2.19% 97.81% Non- Intervention 278 8.99% 91.01% 271 3.32% 96.68% 271 1.48% 98.52% 271 0.00% 100.00% Hispanic Carter et al. (2015) Comparison Hispanic 45 0.00% 100.00% * * * Non- Comparison 225 2.22% 97.78% * * * Hispanic Intervention Hispanic 31 6.45% 93.55% * * * Non- Intervention 178 1.69% 98.31% * * * Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 50 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Coyle et al. (2015) Comparison Hispanic 134 22.39% 77.61% * * 134 2.24% 97.76% Non- Comparison 1114 21.45% 78.55% * * 1119 1.88% 98.12% Hispanic Intervention Hispanic 161 23.60% 76.40% * * 163 1.23% 98.77% Non- Intervention 1310 18.93% 81.07% * * 1310 1.60% 98.40% Hispanic Coyle et al. (2016) Comparison Hispanic 563 16.70% 83.30% * * * Non- Comparison 338 29.88% 70.12% * * * Hispanic Intervention Hispanic 566 15.02% 84.98% * * * Non- Intervention 373 19.84% 80.16% * * * Hispanic Crean et al. (2016) Comparison Hispanic 73 13.70% 86.30% 92 6.52% 93.48% 92 4.35% 95.65% 92 1.09% 98.91% Non- Comparison 231 20.78% 79.22% 317 8.52% 91.48% 315 4.44% 95.56% 324 0.62% 99.38% Hispanic Intervention Hispanic 154 12.99% 87.01% 186 4.30% 95.70% 183 1.64% 98.36% 187 0.53% 99.47% Non- Intervention 273 15.02% 84.98% 375 5.87% 94.13% 374 2.14% 97.86% 378 0.79% 99.21% Hispanic Cunningham et al. (2016) [LN] Comparison Hispanic 14 71.43% 28.57% 14 57.14% 42.86% 14 42.86% 57.14% 14 14.29% 85.71% Non- Comparison 620 35.48% 64.52% 620 23.87% 76.13% 622 16.72% 83.28% 620 2.26% 97.74% Hispanic Intervention Hispanic 36 38.89% 61.11% 36 22.22% 77.78% 36 11.11% 88.89% 36 0.00% 100.00% Non- Intervention 670 33.73% 66.27% 670 20.60% 79.40% 672 16.37% 83.63% 672 2.08% 97.92% Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 51 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Cunningham et al. (2016) [RtR] Comparison Hispanic 14 71.43% 28.57% 14 57.14% 42.86% 14 42.86% 57.14% 14 14.29% 85.71% Non- Comparison 620 35.48% 64.52% 620 23.87% 76.13% 622 16.72% 83.28% 620 2.26% 97.74% Hispanic Intervention Hispanic 22 36.36% 63.64% 22 18.18% 81.82% 22 9.09% 90.91% 22 0.00% 100.00% Non- Intervention 716 29.89% 70.11% 716 18.16% 81.84% 716 13.97% 86.03% 716 1.40% 98.60% Hispanic Daley et al. (2015) Comparison Hispanic 367 37.06% 62.94% 347 19.88% 80.12% 300 16.33% 83.67% 372 3.49% 96.51% Non- Comparison 1628 38.51% 61.49% 1576 24.37% 75.63% 1374 16.59% 83.41% 1605 3.18% 96.82% Hispanic Intervention Hispanic 334 35.03% 64.97% 316 16.14% 83.86% 265 11.70% 88.30% 334 4.49% 95.51% Non- Intervention 1302 34.18% 65.82% 1199 16.93% 83.07% 1055 11.18% 88.82% 1275 3.61% 96.39% Hispanic Dierschke et al. (2015) Comparison Hispanic 221 57.47% 42.53% 221 41.18% 58.82% 221 31.22% 68.78% 220 7.73% 92.27% Non- Comparison 178 58.43% 41.57% 178 34.27% 65.73% 178 20.22% 79.78% 177 4.52% 95.48% Hispanic Intervention Hispanic 218 61.93% 38.07% 218 41.28% 58.72% 218 32.57% 67.43% 218 6.42% 93.58% Non- Intervention 186 55.38% 44.62% 186 33.33% 66.67% 186 23.12% 76.88% 186 3.23% 96.77% Hispanic Eichner et al. (2015) Comparison Hispanic * 19 57.89% 42.11% 19 47.37% 52.63% * Non- Comparison * 320 81.56% 18.44% 320 65.31% 34.69% * Hispanic Intervention Hispanic * 17 76.47% 23.53% 17 58.82% 41.18% * Non- Intervention * 324 80.25% 19.75% 324 62.04% 37.96% * Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 52 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Francis et al. (2015) Comparison Hispanic 91 25.27% 74.73% 90 14.44% 85.56% 90 5.56% 94.44% * Non- Comparison 358 18.16% 81.84% 358 13.41% 86.59% 358 8.66% 91.34% * Hispanic Intervention Hispanic 117 29.06% 70.94% 117 17.09% 82.91% 117 10.26% 89.74% * Non- Intervention 620 22.74% 77.26% 617 14.91% 85.09% 617 8.10% 91.90% * Hispanic Herrling (2016) Comparison Hispanic 5 60.00% 40.00% 5 40.00% 60.00% 4 25.00% 75.00% 5 20.00% 80.00% Non- Comparison 128 26.56% 73.44% 128 15.63% 84.38% 114 10.53% 89.47% 127 0.79% 99.21% Hispanic Intervention Hispanic 7 42.86% 57.14% 7 42.86% 57.14% 7 14.29% 85.71% 7 0.00% 100.00% Non- Intervention 127 28.35% 71.65% 126 18.25% 81.75% 114 11.40% 88.60% 126 4.76% 95.24% Hispanic Kissinger et al. (2015) Comparison Hispanic * 2 50.00% 50.00% 1 100.00% 0.00% * Non- Comparison * 266 58.27% 41.73% 130 51.54% 48.46% * Hispanic Intervention Hispanic * 5 40.00% 60.00% 2 50.00% 50.00% * Non- Intervention * 259 56.37% 43.63% 122 48.36% 51.64% * Hispanic Philliber et al. (2016) Comparison Hispanic 1191 30.98% 69.02% 1186 21.50% 78.50% 1085 13.00% 87.00% 1192 5.54% 94.46% Non- Comparison 2158 31.42% 68.58% 2151 22.13% 77.87% 1961 13.92% 86.08% 2159 4.26% 95.74% Hispanic Intervention Hispanic 1297 34.93% 65.07% 1290 25.27% 74.73% 1170 16.75% 83.25% 1297 7.71% 92.29% Non- Intervention 2258 32.15% 67.85% 2252 22.96% 77.04% 2049 13.81% 86.19% 2259 5.62% 94.38% Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 53 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Philliber & Philliber (2016) Comparison Hispanic 109 15.60% 84.40% 109 8.26% 91.74% 101 2.97% 97.03% 109 3.67% 96.33% Non- Comparison 299 32.11% 67.89% 299 18.39% 81.61% 258 11.24% 88.76% 291 3.09% 96.91% Hispanic Intervention Hispanic 140 21.43% 78.57% 140 11.43% 88.57% 126 7.14% 92.86% 138 5.07% 94.93% Non- Intervention 386 28.50% 71.50% 386 16.84% 83.16% 341 8.80% 91.20% 378 2.12% 97.88% Hispanic Piotrowski et al. (2015) Comparison Hispanic 93 2.15% 97.85% 93 2.15% 97.85% 93 0.00% 100.00% 93 0.00% 100.00% Non- Comparison 578 7.61% 92.39% 578 4.84% 95.16% 575 2.43% 97.57% 576 0.35% 99.65% Hispanic Intervention Hispanic 108 4.63% 95.37% 108 0.93% 99.07% 107 0.93% 99.07% 108 0.00% 100.00% Non- Intervention 676 2.81% 97.19% 676 1.78% 98.22% 676 1.33% 98.67% 675 0.59% 99.41% Hispanic Robinson et al. (2016) Comparison Hispanic 22 22.73% 77.27% 22 18.18% 81.82% 22 4.55% 95.45% 22 0.00% 100.00% Non- Comparison 926 32.40% 67.60% 926 20.52% 79.48% 934 10.39% 89.61% 915 4.70% 95.30% Hispanic Intervention Hispanic 19 26.32% 73.68% 19 15.79% 84.21% 19 10.53% 89.47% 19 0.00% 100.00% Non- Intervention 873 34.71% 65.29% 863 19.70% 80.30% 880 9.77% 90.23% 855 4.44% 95.56% Hispanic Rotz et al. (2016) Comparison Hispanic 125 34.40% 65.60% 124 29.03% 70.97% 124 33.06% 66.94% 122 6.56% 93.44% Non- Comparison 400 45.75% 54.25% 396 41.92% 58.08% 396 42.42% 57.58% 390 4.87% 95.13% Hispanic Intervention Hispanic 252 34.52% 65.48% 245 28.98% 71.02% 245 28.98% 71.02% 242 2.89% 97.11% Non- Intervention 694 40.49% 59.51% 683 36.60% 63.40% 683 36.02% 63.98% 676 1.04% 98.96% Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 54 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Slater et al. (2015) Comparison Hispanic 253 81.03% 18.97% 246 63.82% 36.18% 254 51.18% 48.82% 250 24.80% 75.20% Non- Comparison 223 83.86% 16.14% 219 69.86% 30.14% 224 55.80% 44.20% 221 18.55% 81.45% Hispanic Intervention Hispanic 241 85.48% 14.52% 235 67.66% 32.34% 242 49.59% 50.41% 235 22.55% 77.45% Non- Intervention 236 88.98% 11.02% 231 70.56% 29.44% 237 56.12% 43.88% 233 23.18% 76.82% Hispanic Smith et al. (2015) Comparison Hispanic * 15 86.67% 13.33% 15 100.00% 0.00% 18 94.44% 5.56% Non- Comparison * 226 86.73% 13.27% 226 90.27% 9.73% 246 97.56% 2.44% Hispanic Intervention Hispanic * 17 82.35% 17.65% 17 100.00% 0.00% 17 100.00% 0.00% Non- Intervention * 228 83.33% 16.67% 228 93.42% 6.58% 250 97.60% 2.40% Hispanic Smith et al. (2016) Comparison Hispanic 111 38.74% 61.26% 102 21.57% 78.43% 90 16.67% 83.33% 111 5.41% 94.59% Non- Comparison 202 31.68% 68.32% 188 19.68% 80.32% 175 9.71% 90.29% 201 3.48% 96.52% Hispanic Intervention Hispanic 161 29.19% 70.81% 154 21.43% 78.57% 147 10.88% 89.12% 160 4.38% 95.63% Non- Intervention 263 36.88% 63.12% 243 21.40% 78.60% 219 9.13% 90.87% 262 4.96% 95.04% Hispanic The Policy & Research Group (2015) Comparison Hispanic 7 28.57% 71.43% 6 16.67% 83.33% 6 16.67% 83.33% 6 0.00% 100.00% Non- Comparison 331 37.16% 62.84% 317 23.34% 76.66% 301 13.62% 86.38% 325 2.77% 97.23% Hispanic Intervention Hispanic 11 45.45% 54.55% 11 27.27% 72.73% 9 11.11% 88.89% 11 9.09% 90.91% Non- Intervention 321 38.94% 61.06% 313 22.36% 77.64% 293 9.22% 90.78% 317 4.10% 95.90% Hispanic Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 55 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Ethnicity N Yes No N Yes No N Yes No N Yes No Walker et al. (2016) Comparison Hispanic 191 2.62% 97.38% 186 0.00% 100.00% 186 0.00% 100.00% 190 0.00% 100.00% Non- Comparison 123 1.63% 98.37% 121 0.00% 100.00% 121 0.00% 100.00% 122 0.00% 100.00% Hispanic Intervention Hispanic 248 1.61% 98.39% 247 0.81% 99.19% 246 0.41% 99.59% 248 0.81% 99.19% Non- Intervention 123 2.44% 97.56% 123 1.63% 98.37% 123 1.63% 98.37% 123 1.63% 98.37% Hispanic AZ = Arizona, CA = California, FL = Florida, IL = Illinois, LN = Love Notes, MA = Massachusetts, MN = Minnesota, MO = Missouri, RTR = Reducing the Risk, TN = Tennessee TX = Texas. Notes. The presence of an asterisk (*) indicates that this outcome was not reported at the first post-test. Bold text indicates that the outcome was selected as confirmatory. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 56 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.3.3: SUBGROUP EFFECTS BY RACE Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Abe et al. (2016) Comparison Black 6 33.33% 66.67% 6 33.33% 66.67% 6 16.67% 83.33% 6 0.00% 100.00% Comparison White 31 0.00% 100.00% 31 0.00% 100.00% 31 0.00% 100.00% 31 0.00% 100.00% Comparison Other 459 11.55% 88.45% 460 5.65% 94.35% 460 2.83% 97.17% 460 0.65% 99.35% Intervention Black 18 5.56% 94.44% 18 5.56% 94.44% 18 0.00% 100.00% 18 0.00% 100.00% Intervention White 69 14.49% 85.51% 69 10.14% 89.86% 69 5.80% 94.20% 69 4.35% 95.65% Intervention Other 775 8.52% 91.48% 780 4.74% 95.26% 781 1.41% 98.59% 780 1.41% 98.59% Abt Associates (2016a) [AZ] Comparison Black 27 14.81% 85.19% 27 0.00% 100.00% 27 0.00% 100.00% 27 0.00% 100.00% Comparison White 69 8.70% 91.30% 69 7.25% 92.75% 69 5.80% 94.20% 68 1.47% 98.53% Comparison Other 243 10.70% 89.30% 243 6.58% 93.42% 243 4.12% 95.88% 242 0.00% 100.00% Intervention Black 25 28.00% 72.00% 25 12.00% 88.00% 25 8.00% 92.00% 25 0.00% 100.00% Intervention White 97 15.46% 84.54% 97 8.25% 91.75% 97 6.19% 93.81% 97 0.00% 100.00% Intervention Other 357 11.76% 88.24% 357 6.44% 93.56% 357 5.32% 94.68% 357 0.56% 99.44% Abt Associates (2016a) [CA] Comparison Black 4 50.00% 50.00% 4 25.00% 75.00% 4 0.00% 100.00% 4 0.00% 100.00% Comparison White 97 45.36% 54.64% 97 27.84% 72.16% 97 20.62% 79.38% 97 2.06% 97.94% Comparison Other 101 30.69% 69.31% 101 20.79% 79.21% 101 16.83% 83.17% 101 1.98% 98.02% Intervention Black 2 50.00% 50.00% 2 50.00% 50.00% 2 50.00% 50.00% 2 0.00% 100.00% Intervention White 126 41.27% 58.73% 126 30.95% 69.05% 126 27.78% 72.22% 126 4.76% 95.24% Intervention Other 154 44.16% 55.84% 154 28.57% 71.43% 154 22.73% 77.27% 154 1.95% 98.05% Abt Associates (2016a) [MA] Comparison Black 22 45.45% 54.55% 22 40.91% 59.09% 22 31.82% 68.18% 22 9.09% 90.91% Comparison White 60 45.00% 55.00% 60 28.33% 71.67% 60 25.00% 75.00% 60 6.67% 93.33% Comparison Other 170 42.35% 57.65% 170 27.65% 72.35% 170 21.76% 78.24% 170 3.53% 96.47% Intervention Black 34 38.24% 61.76% 34 17.65% 82.35% 34 8.82% 91.18% 34 2.94% 97.06% Intervention White 79 55.70% 44.30% 79 34.18% 65.82% 79 27.85% 72.15% 79 6.33% 93.67% Intervention Other 323 45.51% 54.49% 322 32.61% 67.39% 323 26.63% 73.37% 323 5.26% 94.74% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 57 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Abt Associates (2016b) [CA] Comparison Black 16 6.25% 93.75% 16 0.00% 100.00% 16 0.00% 100.00% 16 0.00% 100.00% Comparison White 52 19.23% 80.77% 52 11.54% 88.46% 52 11.54% 88.46% 52 1.92% 98.08% Comparison Other 306 20.59% 79.41% 305 12.46% 87.54% 306 10.13% 89.87% 305 1.31% 98.69% Intervention Black 36 33.33% 66.67% 36 22.22% 77.78% 36 19.44% 80.56% 36 0.00% 100.00% Intervention White 84 26.19% 73.81% 83 16.87% 83.13% 83 15.66% 84.34% 84 1.19% 98.81% Intervention Other 377 23.08% 76.92% 377 12.73% 87.27% 377 9.55% 90.45% 376 0.53% 99.47% Abt Associates (2016b) [IL & MO] Comparison Black 335 56.72% 43.28% 333 40.54% 59.46% 334 29.34% 70.66% 335 12.24% 87.76% Comparison White 3 33.33% 66.67% 3 33.33% 66.67% 3 33.33% 66.67% 3 33.33% 66.67% Comparison Other 24 62.50% 37.50% 24 54.17% 45.83% 24 45.83% 54.17% 24 16.67% 83.33% Intervention Black 508 57.28% 42.72% 507 38.07% 61.93% 507 24.06% 75.94% 506 8.10% 91.90% Intervention White 6 50.00% 50.00% 6 16.67% 83.33% 6 16.67% 83.33% 6 0.00% 100.00% Intervention Other 51 52.94% 47.06% 51 33.33% 66.67% 51 19.61% 80.39% 50 8.00% 92.00% Abt Associates (2016b) [TX] Comparison Black 40 37.50% 62.50% 40 20.00% 80.00% 40 20.00% 80.00% 40 5.00% 95.00% Comparison White 138 48.55% 51.45% 138 29.71% 70.29% 138 29.71% 70.29% 138 3.62% 96.38% Comparison Other 228 50.00% 50.00% 228 33.77% 66.23% 228 29.82% 70.18% 228 7.89% 92.11% Intervention Black 44 52.27% 47.73% 44 27.27% 72.73% 44 18.18% 81.82% 44 6.82% 93.18% Intervention White 148 54.73% 45.27% 148 37.84% 62.16% 148 34.46% 65.54% 148 3.38% 96.62% Intervention Other 247 51.82% 48.18% 247 36.03% 63.97% 247 29.15% 70.85% 247 7.29% 92.71% Abt Associates (2016c) [FL] Comparison Black 71 77.46% 22.54% 71 63.38% 36.62% 71 50.70% 49.30% 71 14.08% 85.92% Comparison White 51 96.08% 3.92% 51 86.27% 13.73% 51 78.43% 21.57% 51 11.76% 88.24% Comparison Other 24 87.50% 12.50% 24 70.83% 29.17% 24 58.33% 41.67% 24 29.17% 70.83% Intervention Black 134 86.57% 13.43% 134 68.66% 31.34% 134 60.45% 39.55% 134 31.34% 68.66% Intervention White 105 92.38% 7.62% 105 79.05% 20.95% 105 69.52% 30.48% 105 20.00% 80.00% Intervention Other 41 92.68% 7.32% 41 70.73% 29.27% 41 53.66% 46.34% 41 19.51% 80.49% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 58 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Abt Associates (2016c) [MN] Comparison Black 230 86.96% 13.04% 230 71.30% 28.70% 230 63.48% 36.52% 228 30.70% 69.30% Comparison White 198 96.97% 3.03% 198 86.87% 13.13% 198 82.83% 17.17% 198 12.12% 87.88% Comparison Other 228 92.11% 7.89% 228 78.95% 21.05% 228 70.18% 29.82% 226 18.58% 81.42% Intervention Black 476 85.29% 14.71% 474 70.04% 29.96% 474 56.12% 43.88% 474 26.16% 73.84% Intervention White 356 98.88% 1.12% 356 89.89% 10.11% 356 85.39% 14.61% 356 15.17% 84.83% Intervention Other 442 88.69% 11.31% 440 71.36% 28.64% 442 62.90% 37.10% 442 24.43% 75.57% Abt Associates (2016c) [TN] Comparison Black 35 77.14% 22.86% 35 48.57% 51.43% 35 42.86% 57.14% 35 17.14% 82.86% Comparison White 89 95.51% 4.49% 89 77.53% 22.47% 89 71.91% 28.09% 89 23.60% 76.40% Comparison Other 13 84.62% 15.38% 13 61.54% 38.46% 13 53.85% 46.15% 13 23.08% 76.92% Intervention Black 68 76.47% 23.53% 68 57.35% 42.65% 68 44.12% 55.88% 68 16.18% 83.82% Intervention White 180 93.89% 6.11% 180 79.44% 20.56% 180 67.78% 32.22% 180 23.89% 76.11% Intervention Other 27 88.89% 11.11% 27 66.67% 33.33% 27 62.96% 37.04% 27 33.33% 66.67% Advanced Empirical Solutions (2015) Comparison Black 16 0.00% 100.00% * * 16 0.00% 100.00% Comparison White 7 0.00% 100.00% * * 7 0.00% 100.00% Comparison Other 22 0.00% 100.00% * * 22 0.00% 100.00% Intervention Black 15 0.00% 100.00% * * 15 0.00% 100.00% Intervention White 9 0.00% 100.00% * * 9 0.00% 100.00% Intervention Other 33 0.00% 100.00% * * 33 0.00% 100.00% Calise et al. (2015) Comparison Black 45 20.00% 80.00% 44 9.09% 90.91% 44 6.82% 93.18% 43 2.33% 97.67% Comparison White 180 8.89% 91.11% 175 4.57% 95.43% 175 2.86% 97.14% 176 0.57% 99.43% Comparison Other 323 12.38% 87.62% 316 6.65% 93.35% 313 3.51% 96.49% 313 0.64% 99.36% Intervention Black 33 15.15% 84.85% 31 6.45% 93.55% 31 3.23% 96.77% 32 0.00% 100.00% Intervention White 179 8.38% 91.62% 176 3.41% 96.59% 176 2.27% 97.73% 174 0.00% 100.00% Intervention Other 226 9.73% 90.27% 221 6.33% 93.67% 221 4.07% 95.93% 222 1.80% 98.20% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 59 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Carter et al. (2015) Comparison Black 6 0.00% 100.00% * * * Comparison White 12 0.00% 100.00% * * * Comparison Other 237 2.11% 97.89% * * * Intervention Black 5 0.00% 100.00% * * * Intervention White 8 0.00% 100.00% * * * Intervention Other 186 2.15% 97.85% * * * Coyle et al. (2015) Comparison Black 524 29.20% 70.80% * * 526 2.28% 97.72% Comparison White 517 12.96% 87.04% * * 520 1.15% 98.85% Comparison Other 143 25.17% 74.83% * * 143 3.50% 96.50% Intervention Black 574 26.48% 73.52% * * 574 1.74% 98.26% Intervention White 624 12.34% 87.66% * * 623 1.44% 98.56% Intervention Other 196 21.43% 78.57% * * 197 1.52% 98.48% Coyle et al. (2016) Comparison Black 251 34.66% 65.34% * * * Comparison White 111 9.01% 90.99% * * * Comparison Other 139 18.71% 81.29% * * * Intervention Black 275 21.45% 78.55% * * * Intervention White 95 18.95% 81.05% * * * Intervention Other 152 16.45% 83.55% * * * Crean et al. (2016) Comparison Black 209 22.01% 77.99% 286 10.14% 89.86% 285 5.96% 94.04% 292 1.03% 98.97% Comparison White 39 5.13% 94.87% 46 0.00% 100.00% 46 0.00% 100.00% 46 0.00% 100.00% Comparison Other 56 17.86% 82.14% 77 5.19% 94.81% 76 1.32% 98.68% 78 0.00% 100.00% Intervention Black 263 16.35% 83.65% 360 6.11% 93.89% 358 2.23% 97.77% 362 0.55% 99.45% Intervention White 71 5.63% 94.37% 86 3.49% 96.51% 86 1.16% 98.84% 86 1.16% 98.84% Intervention Other 93 15.05% 84.95% 115 4.35% 95.65% 113 1.77% 98.23% 117 0.85% 99.15% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 60 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Cunningham et al. (2016) [LN] Comparison Black 606 35.64% 64.36% 606 24.42% 75.58% 608 16.78% 83.22% 606 2.31% 97.69% Comparison White 42 42.86% 57.14% 42 23.81% 76.19% 42 23.81% 76.19% 42 4.76% 95.24% Comparison Other 6 66.67% 33.33% 6 66.67% 33.33% 6 0.00% 100.00% 6 0.00% 100.00% Intervention Black 622 32.80% 67.20% 622 20.26% 79.74% 624 15.71% 84.29% 624 1.92% 98.08% Intervention White 58 48.28% 51.72% 58 24.14% 75.86% 58 17.24% 82.76% 58 0.00% 100.00% Intervention Other 10 40.00% 60.00% 10 40.00% 60.00% 10 20.00% 80.00% 10 20.00% 80.00% Cunningham et al. (2016) [RTR] Comparison Black 606 35.64% 64.36% 606 24.42% 75.58% 608 16.78% 83.22% 606 2.31% 97.69% Comparison White 42 42.86% 57.14% 42 23.81% 76.19% 42 23.81% 76.19% 42 4.76% 95.24% Comparison Other 6 66.67% 33.33% 6 66.67% 33.33% 6 0.00% 100.00% 6 0.00% 100.00% Intervention Black 688 30.81% 69.19% 688 19.19% 80.81% 688 13.95% 86.05% 688 1.16% 98.84% Intervention White 50 28.00% 72.00% 50 12.00% 88.00% 50 4.00% 96.00% 50 4.00% 96.00% Intervention Other 8 25.00% 75.00% 8 0.00% 100.00% 8 25.00% 75.00% 8 0.00% 100.00% Daley et al. (2015) Comparison Black 182 48.90% 51.10% 173 20.23% 79.77% 127 14.96% 85.04% 185 4.86% 95.14% Comparison White 1484 36.99% 63.01% 1435 24.39% 75.61% 1274 16.64% 83.36% 1462 2.80% 97.20% Comparison Other 251 38.25% 61.75% 243 22.63% 77.37% 210 17.62% 82.38% 245 4.49% 95.51% Intervention Black 180 42.78% 57.22% 165 16.97% 83.03% 129 9.30% 90.70% 178 5.06% 94.94% Intervention White 1175 31.74% 68.26% 1088 16.91% 83.09% 981 11.52% 88.48% 1149 3.13% 96.87% Intervention Other 195 40.51% 59.49% 189 16.93% 83.07% 147 12.24% 87.76% 198 3.54% 96.46% Dierschke et al. (2015) Comparison Black 93 60.22% 39.78% 93 35.48% 64.52% 93 21.51% 78.49% 93 6.45% 93.55% Comparison White 83 55.42% 44.58% 83 36.14% 63.86% 83 24.10% 75.90% 83 9.64% 90.36% Comparison Other 223 57.85% 42.15% 223 39.91% 60.09% 223 29.15% 70.85% 221 4.98% 95.02% Intervention Black 92 53.26% 46.74% 92 33.70% 66.30% 92 25.00% 75.00% 92 5.43% 94.57% Intervention White 83 59.04% 40.96% 83 36.14% 63.86% 83 24.10% 75.90% 83 4.82% 95.18% Intervention Other 229 61.14% 38.86% 229 39.74% 60.26% 229 31.00% 69.00% 229 4.80% 95.20% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 61 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Eichner et al. (2015) Comparison Black * 99 68.69% 31.31% 99 54.55% 45.45% * Comparison White * 196 86.22% 13.78% 196 68.88% 31.12% * Comparison Other * 38 81.58% 18.42% 38 65.79% 34.21% * Intervention Black * 108 76.85% 23.15% 108 55.56% 44.44% * Intervention White * 194 82.99% 17.01% 194 66.49% 33.51% * Intervention Other * 35 71.43% 28.57% 35 51.43% 48.57% * Francis et al. (2015) Comparison Black 127 17.32% 82.68% 127 13.39% 86.61% 127 6.30% 93.70% * Comparison White 113 13.27% 86.73% 113 10.62% 89.38% 113 7.08% 92.92% * Comparison Other 119 23.53% 76.47% 119 15.97% 84.03% 119 12.61% 87.39% * Intervention Black 193 31.61% 68.39% 193 20.73% 79.27% 193 9.33% 90.67% * Intervention White 204 16.18% 83.82% 204 11.27% 88.73% 204 7.84% 92.16% * Intervention Other 224 20.98% 79.02% 221 13.12% 86.88% 221 7.24% 92.76% * Herrling (2016) Comparison Black 119 26.05% 73.95% 119 15.97% 84.03% 107 10.28% 89.72% 119 0.84% 99.16% Comparison White 2 100.00% 0.00% 2 50.00% 50.00% 1 0.00% 100.00% 2 0.00% 100.00% Comparison Other 12 33.33% 66.67% 12 16.67% 83.33% 10 20.00% 80.00% 11 9.09% 90.91% Intervention Black 117 29.06% 70.94% 116 18.97% 81.03% 105 10.48% 89.52% 116 4.31% 95.69% Intervention White 0 0 0 0 Intervention Other 16 25.00% 75.00% 16 18.75% 81.25% 15 13.33% 86.67% 16 6.25% 93.75% Kissinger et al. (2015) Comparison Black * 258 57.75% 42.25% 125 50.40% 49.60% * Comparison White * 0 0 * Comparison Other * 10 70.00% 30.00% 6 83.33% 16.67% * Intervention Black * 254 55.91% 44.09% 118 47.46% 52.54% * Intervention White * 0 0 * Intervention Other * 10 60.00% 40.00% 6 66.67% 33.33% * Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 62 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Philliber et al. (2016) Comparison Black 228 30.70% 69.30% 229 20.09% 79.91% 204 10.78% 89.22% 229 5.24% 94.76% Comparison White 1257 35.56% 64.44% 1251 25.10% 74.90% 1128 16.49% 83.51% 1257 4.61% 95.39% Comparison Other 1534 27.25% 72.75% 1528 19.24% 80.76% 1416 11.79% 88.21% 1535 4.23% 95.77% Intervention Black 237 33.76% 66.24% 236 21.19% 78.81% 207 11.11% 88.89% 237 5.06% 94.94% Intervention White 1283 33.59% 66.41% 1280 25.47% 74.53% 1176 15.65% 84.35% 1283 6.39% 93.61% Intervention Other 1669 32.00% 68.00% 1660 22.29% 77.71% 1507 14.66% 85.34% 1670 6.59% 93.41% Philliber & Philliber (2016) Comparison Black 216 33.33% 66.67% 216 20.83% 79.17% 189 11.64% 88.36% 209 3.83% 96.17% Comparison White 30 36.67% 63.33% 30 16.67% 83.33% 24 8.33% 91.67% 29 3.45% 96.55% Comparison Other 162 18.52% 81.48% 162 8.64% 91.36% 146 5.48% 94.52% 162 2.47% 97.53% Intervention Black 293 27.65% 72.35% 293 16.38% 83.62% 260 9.23% 90.77% 287 2.44% 97.56% Intervention White 21 33.33% 66.67% 21 19.05% 80.95% 18 5.56% 94.44% 21 0.00% 100.00% Intervention Other 212 24.53% 75.47% 212 13.68% 86.32% 189 7.41% 92.59% 208 3.85% 96.15% Piotrowski et al. (2015) Comparison Black 9 11.11% 88.89% 9 11.11% 88.89% 9 11.11% 88.89% 9 0.00% 100.00% Comparison White 654 6.88% 93.12% 654 4.43% 95.57% 651 2.00% 98.00% 652 0.31% 99.69% Comparison Other 8 0.00% 100.00% 8 0.00% 100.00% 8 0.00% 100.00% 8 0.00% 100.00% Intervention Black 24 8.33% 91.67% 24 4.17% 95.83% 24 4.17% 95.83% 23 0.00% 100.00% Intervention White 755 2.91% 97.09% 755 1.59% 98.41% 754 1.19% 98.81% 755 0.53% 99.47% Intervention Other 5 0.00% 100.00% 5 0.00% 100.00% 5 0.00% 100.00% 5 0.00% 100.00% Robinson et al. (2016) Comparison Black 955 32.98% 67.02% 955 20.21% 79.79% 964 11.10% 88.90% 941 4.46% 95.54% Comparison White 34 32.35% 67.65% 34 29.41% 70.59% 34 5.88% 94.12% 33 3.03% 96.97% Comparison Other 68 33.82% 66.18% 67 20.90% 79.10% 68 8.82% 91.18% 68 7.35% 92.65% Intervention Black 877 35.69% 64.31% 865 21.04% 78.96% 884 9.95% 90.05% 856 4.56% 95.44% Intervention White 29 34.48% 65.52% 29 27.59% 72.41% 29 17.24% 82.76% 29 0.00% 100.00% Intervention Other 73 27.40% 72.60% 73 10.96% 89.04% 73 8.22% 91.78% 73 2.74% 97.26% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 63 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No Rotz et al. (2016) Comparison Black 217 49.77% 50.23% 214 44.86% 55.14% 214 44.86% 55.14% 208 6.73% 93.27% Comparison White 187 40.64% 59.36% 186 37.63% 62.37% 186 39.25% 60.75% 186 3.76% 96.24% Comparison Other 57 47.37% 52.63% 57 42.11% 57.89% 57 43.86% 56.14% 56 10.71% 89.29% Intervention Black 285 43.86% 56.14% 280 37.86% 62.14% 280 38.57% 61.43% 277 1.44% 98.56% Intervention White 343 38.78% 61.22% 338 35.80% 64.20% 338 34.62% 65.38% 335 1.19% 98.81% Intervention Other 174 35.63% 64.37% 170 33.53% 66.47% 170 31.76% 68.24% 168 0.60% 99.40% Slater et al. (2015) Comparison Black 0 0 0 0 Comparison White 90 84.44% 15.56% 88 71.59% 28.41% 90 58.89% 41.11% 88 15.91% 84.09% Comparison Other 70 77.14% 22.86% 68 64.71% 35.29% 71 57.75% 42.25% 68 22.06% 77.94% Intervention Black 0 0 0 0 Intervention White 81 92.59% 7.41% 77 80.52% 19.48% 81 61.73% 38.27% 81 33.33% 66.67% Intervention Other 60 81.67% 18.33% 59 72.88% 27.12% 60 50.00% 50.00% 60 13.33% 86.67% Smith et al. (2015) Comparison Black * 91 83.52% 16.48% 91 96.70% 3.30% 101 95.05% 4.95% Comparison White * 120 89.17% 10.83% 120 89.17% 10.83% 128 99.22% 0.78% Comparison Other * 22 86.36% 13.64% 22 72.73% 27.27% 25 100.00% 0.00% Intervention Black * 95 84.21% 15.79% 95 97.89% 2.11% 100 98.00% 2.00% Intervention White * 111 85.59% 14.41% 111 90.09% 9.91% 126 96.83% 3.17% Intervention Other * 29 72.41% 27.59% 29 93.10% 6.90% 31 100.00% 0.00% Smith et al. (2016) Comparison Black 193 33.68% 66.32% 176 19.89% 80.11% 163 9.82% 90.18% 193 3.11% 96.89% Comparison White 54 29.63% 70.37% 50 18.00% 82.00% 47 12.77% 87.23% 53 9.43% 90.57% Comparison Other 29 31.03% 68.97% 29 17.24% 82.76% 25 12.00% 88.00% 29 3.45% 96.55% Intervention Black 251 37.45% 62.55% 234 22.22% 77.78% 210 8.57% 91.43% 251 4.78% 95.22% Intervention White 54 29.63% 70.37% 48 18.75% 81.25% 47 10.64% 89.36% 54 1.85% 98.15% Intervention Other 37 29.73% 70.27% 35 20.00% 80.00% 33 12.12% 87.88% 36 2.78% 97.22% Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 64 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Recent Unprotected Ever Had Sex Recent Sexual Activity Ever Pregnant Sexual Activity Study Condition Race N Yes No N Yes No N Yes No N Yes No The Policy & Research Group (2015) Comparison Black 320 36.88% 63.13% 305 22.62% 77.38% 290 13.10% 86.90% 314 2.55% 97.45% Comparison White 0 0 0 0 Comparison Other 23 34.78% 65.22% 23 30.43% 69.57% 22 22.73% 77.27% 22 4.55% 95.45% Intervention Black 317 39.43% 60.57% 308 22.73% 77.27% 288 9.72% 90.28% 311 3.22% 96.78% Intervention White 0 0 0 0 Intervention Other 30 46.67% 53.33% 29 20.69% 79.31% 24 8.33% 91.67% 29 13.79% 86.21% Walker et al. (2016) Comparison Black 81 1.23% 98.77% 80 0.00% 100.00% 80 0.00% 100.00% 81 0.00% 100.00% Comparison White 57 1.75% 98.25% 56 0.00% 100.00% 56 0.00% 100.00% 57 0.00% 100.00% Comparison Other 20 5.00% 95.00% 19 0.00% 100.00% 19 0.00% 100.00% 19 0.00% 100.00% Intervention Black 94 1.06% 98.94% 94 1.06% 98.94% 94 1.06% 98.94% 94 1.06% 98.94% Intervention White 60 3.33% 96.67% 60 1.67% 98.33% 60 1.67% 98.33% 60 1.67% 98.33% Intervention Other 29 0.00% 100.00% 29 0.00% 100.00% 29 0.00% 100.00% 29 0.00% 100.00% AZ = Arizona, CA = California, FL = Florida, IL = Illinois, LN = Love Notes, MA = Massachusetts, MN = Minnesota, MO = Missouri, RTR = Reducing the Risk, TN = Tennessee TX = Texas. Notes. The presence of an asterisk (*) indicates that this outcome was not reported at the first post-test. Bold text indicates that the outcome was selected as confirmatory. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 65 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.4. Sensitivity Analyses Examining Robustness of Mean Effect Size Estimates The robust variance estimation (RVE) approach used in our analysis requires an assumed average correlation between effect size estimates within studies ( ), which we conservatively assumed to be .80. This section presents sensitivity analyses using different assumed values of this parameter, ranging from .10 to .90. Findings presented in Table 3.4.1 below (for the analysis of confirmatory outcomes) show that results were robust across assumed values of . Results were also robust to other analysis assumptions: excluding Cox-transformed effect sizes, Winsorizing outliers, and restricting the AD analysis to the 34 studies providing IPD. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 66 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.4.1: SENSITIVITY ANALYSES EXAMINING ROBUSTNESS OF MEAN EFFECT SIZE ESTIMATES FOR BINARY OUTCOMES Recent Recent Sexual Mean LOR [95% CI] Ever Had Sex Unprotected Sexual Ever Pregnant Recent Pregnancy Activity Activity Primary Analysis 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Sensitivity Analyses Excluding Cox−transformed effect sizes 0.07 [−0.00, 0.16] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Winsorizing outliers 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .10 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .20 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .30 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .40 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .50 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .60 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .70 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Assuming ρ = .90 0.07 [−0.01, 0.14] −0.05 [–0.18, 0.08] 0.05 [−0.04, 0.14] 0.19 [−0.68, 1.06] 0.26 [0.00, 0.52] Restricting to studies providing IPD 0.06 [−0.03, 0.15] −0.07 [–0.23, 0.09] 0.03 [−0.05, 0.11] 0.13 [−1.93, 2.18] 0.25 [−0.11, 0.61] Assuming ICC = .08 0.04 [−0.05, 0.13] −0.06 [–0.20, 0.08] 0.07 [−0.00, 0.14] −0.13 [−1.37, 1.11] 0.31 [0.03, 0.58] ρ = assumed average correlation between effect sizes, CI = confidence interval, ICC = intra-class correlation, IPD = individual participant data, LOR = log odds ratio. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 67 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.5. Bivariate Correlations between Moderators Even after pooling across outcomes, our sample sizes were limited for estimating multivariable meta- regression models. Therefore, all meta-regression analyses were estimated such that each type of effect size moderator was examined individually. Although this approach limited our ability to control for potential confounding between moderators, examination of the bivariate correlations between moderators—presented in this section—suggests that few of the moderators were highly correlated. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 68 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.5.1: BIVARIATE CORRELATIONS BETWEEN MODERATORS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 Focus: Sexual health 1.00 2 Focus: Youth development −0.65 1.00 3 Condom demonstration 0.10 −0.38 1.00 4 Service learning −0.36 0.56 −0.24 1.00 5 Role plays 0.18 −0.45 0.43 −0.23 1.00 6 Games −0.40 −0.04 0.20 0.11 0.32 1.00 7 Reflective exercises 0.20 −0.08 0.20 −0.05 0.17 −0.30 1.00 8 Direct provision of health services −0.14 0.21 −0.27 −0.07 −0.44 −0.16 0.08 1.00 9 Parent activities 0.09 −0.12 −0.35 −0.10 0.11 −0.13 0.05 0.06 1.00 10 Positive role model −0.41 0.64 −0.27 0.74 −0.31 0.06 −0.15 −0.08 −0.12 1.00 11 Size: Individualized 0.19 −0.12 0.34 −0.10 0.16 −0.23 0.67 −0.03 −0.17 −0.12 1.00 12 Size: Small groups (<10) −0.22 −0.20 0.20 −0.11 0.23 0.38 −0.24 −0.13 0.00 −0.13 −0.19 1.00 13 Size: Large groups 0.03 0.17 −0.32 0.20 −0.13 −0.11 −0.22 0.16 0.09 0.23 −0.52 −0.56 1.00 14 At least weekly contact −0.16 0.23 −0.28 0.15 −0.16 0.01 −0.42 −0.03 −0.15 0.18 −0.62 −0.07 0.56 1.00 15 Contact hours −0.25 0.42 −0.24 0.05 −0.36 −0.12 −0.02 0.73 −0.08 −0.01 −0.11 −0.10 0.18 0.16 1.00 16 Same-gender group composition 0.19 −0.25 0.44 −0.16 0.19 −0.12 0.33 −0.12 −0.18 −0.19 0.58 −0.15 −0.32 −0.34 −0.17 1.00 17 Setting: Classroom −0.16 0.14 −0.16 0.04 0.12 0.02 −0.13 −0.03 0.14 0.16 −0.41 −0.09 0.42 0.40 −0.05 −0.16 18 Setting: Community 0.05 0.05 −0.08 0.05 −0.26 0.05 −0.27 0.17 0.02 0.01 −0.21 −0.01 0.20 0.23 0.22 −0.29 19 Personnel: Health educators −0.27 −0.02 0.32 0.12 0.31 0.36 0.25 −0.14 −0.07 0.11 0.40 0.20 −0.30 −0.29 −0.17 0.24 20 Personnel: Classroom teachers 0.08 0.03 −0.05 0.05 0.13 0.04 0.20 −0.09 0.21 0.03 −0.13 −0.13 0.24 −0.01 −0.06 −0.20 21 Implementation fidelity −0.09 −0.03 −0.17 0.00 −0.12 0.12 −0.46 0.04 0.15 0.06 −0.50 0.05 0.42 0.43 −0.14 −0.19 22 Mean attendance 0.04 −0.30 0.39 −0.26 0.36 0.20 0.08 −0.22 0.11 −0.28 0.02 0.16 0.05 −0.11 −0.19 0.07 23 Mean retention 0.08 −0.33 0.37 −0.28 0.35 0.21 0.06 −0.22 0.12 −0.27 0.00 0.13 0.05 −0.07 −0.22 0.11 34 Percentage boys −0.15 0.11 −0.27 0.08 −0.05 0.16 −0.47 0.03 0.08 0.07 −0.76 0.10 0.53 0.53 0.11 −0.47 25 Percentage Black 0.15 0.13 −0.13 −0.01 −0.42 −0.39 −0.10 0.30 −0.17 0.07 −0.04 −0.25 0.23 0.23 0.29 0.04 26 Percentage Hispanic −0.28 0.05 −0.02 −0.12 0.16 0.41 −0.18 −0.18 0.09 −0.09 −0.23 0.24 −0.14 0.02 −0.09 −0.23 27 Average age 0.22 −0.25 0.39 −0.25 −0.03 −0.28 0.34 −0.20 −0.35 −0.21 0.58 0.10 −0.52 −0.37 −0.33 0.39 28 Unprotected sex at baseline 0.20 −0.13 0.20 −0.10 −0.19 −0.32 0.41 0.08 −0.21 −0.14 0.61 −0.08 −0.57 −0.56 −0.17 0.62 29 Control group post−test sex rate 0.21 −0.16 0.38 −0.13 0.03 −0.32 0.45 0.04 −0.20 −0.12 0.73 −0.07 −0.52 −0.55 −0.13 0.52 30 Randomized controlled trial −0.03 −0.09 0.30 −0.04 0.12 0.18 −0.04 −0.35 −0.35 0.09 0.13 −0.01 −0.13 0.06 −0.46 0.21 31 Overall attrition 0.00 0.26 −0.35 0.24 −0.07 −0.09 −0.04 0.03 0.45 0.19 −0.20 0.02 −0.07 −0.04 0.15 −0.23 32 Differential attrition 0.07 0.11 −0.32 −0.04 −0.21 −0.09 −0.05 0.36 0.39 −0.10 −0.17 0.06 −0.01 −0.04 0.21 −0.32 33 Active control group −0.08 0.07 −0.29 −0.02 −0.33 0.02 −0.33 0.07 0.05 −0.07 −0.29 −0.08 0.06 0.24 0.20 −0.01 continued Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 69 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.5.1: BIVARIATE CORRELATIONS BETWEEN MODERATORS (CONTINUED) 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 17 Unprotected sex at baseline 1.00 18 Control group post−test sex rate −0.51 1.00 19 Personnel: Health educators −0.27 0.01 1.00 20 Personnel: Classroom teachers 0.30 −0.15 −0.28 1.00 21 Implementation fidelity 0.41 −0.01 −0.17 −0.13 1.00 22 Mean attendance −0.02 0.05 0.13 0.08 0.29 1.00 23 Mean retention 0.00 0.04 0.11 0.10 0.33 0.99 1.00 24 Percentage boys 0.43 0.16 −0.26 0.13 0.41 0.09 0.07 1.00 25 Percentage Black −0.39 0.50 −0.24 −0.23 0.08 −0.21 −0.15 −0.02 1.00 26 Percentage Hispanic 0.29 −0.25 0.00 0.12 0.01 0.05 0.05 0.15 −0.72 1.00 27 Average age −0.49 −0.01 0.25 −0.27 −0.20 0.20 0.23 −0.55 0.12 −0.28 1.00 28 Unprotected sex at baseline −0.51 −0.12 0.04 −0.19 −0.30 0.06 0.11 −0.79 0.13 −0.41 0.78 1.00 29 Control group post-test sex rate −0.46 −0.07 0.30 −0.29 −0.27 0.03 0.07 −0.66 0.19 −0.39 0.85 0.83 1.00 30 Randomized controlled trial −0.03 −0.12 −0.10 0.09 0.07 −0.01 0.02 −0.12 −0.03 0.11 0.19 0.14 0.01 1.00 31 Overall attrition 0.05 0.07 −0.28 0.16 −0.08 −0.21 −0.21 0.04 −0.23 0.19 −0.37 −0.01 −0.23 −0.19 1.00 32 Differential attrition −0.10 0.33 −0.17 0.11 −0.36 −0.21 −0.22 0.07 0.05 0.07 −0.25 −0.19 −0.16 −0.57 0.42 1.00 33 Active control group −0.17 0.25 −0.24 −0.21 0.07 −0.23 −0.22 0.02 0.35 0.04 −0.30 −0.02 −0.33 0.10 0.12 0.11 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 70 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.6. Additional Meta-Regression Model Specifications for Associations between Moderators and Effect Sizes This section presents sensitivity analyses, for the analysis of confirmatory outcomes, showing results from models examining one moderator variable at a time (without adjusting for other variables within a moderator block) and examining all moderators within a block simultaneously in a single multivariable meta-regression model. TABLE 3.6.1: PROGRAM DESIGN MODERATORS OF EFFECTS: UNSTANDARDIZED COEFFICIENTS AND 95% CONFIDENCE INTERVALS FROM META-REGRESSION MODELS Individual Models Full Model b 95% CI b 95% CI Program Type Tier I −0.10 [−0.24, 0.03] −0.12 [−0.32, 0.09] Program Focus Sexual health −0.08 [−0.25, 0.10] Ref. Youth development 0.01 [−0.17, 0.19] 0.14 [−0.09, 0.37] Other 0.24 [−0.42, 0.89] 0.02 [−0.42, 0.45] Program Components Condom demonstration 0.06 [−0.07, 0.19] 0.10 [−0.08, 0.29] Service learning 0.01 [−0.36, 0.37] 0.19 [−0.30, 0.67] Role plays −0.01 [−0.15, 0.14] −0.05 [−0.26, 0.17] Games 0.04 [−0.15, 0.24] 0.04 [−0.22, 0.31] Reflective exercises 0.13 [−0.09, 0.34] 0.07 [−0.17, 0.30] Direct provision of health services 0.44 [−0.60, 1.48] 0.44 [−0.30, 1.18] Parent activities −0.03 [−0.21, 0.14] 0.02 [−0.18, 0.23] Positive role model −0.08 [−0.30, 0.15] −0.17 [−0.55, 0.22] Group Size Individualized 0.26 [−0.01, 0.52] 0.07 [−0.31, 0.45] Small groups (<10) −0.04 [−0.21, 0.12] −0.03 [−0.28, 0.22] Large groups −0.09 [−0.23, 0.05] Ref. Other −0.06 [−0.22, 0.10] −0.07 Program Length At least weekly contact −0.15 [−0.34, 0.04] −0.04 [−0.25, 0.17] Contact hours 0.00 [0.00, 0.00] 0.00 [0.00, 0.00] Group Composition Same gender 0.08 [−0.09, 0.25] −0.04 [−0.34, 0.27] Gender Targeting Girls only 0.16 [−0.05, 0.37] 0.04 [−0.34, 0.43] Full model intercept na 0.09 [−0.21, 0.39] b = unstandardized meta-regression coefficients, CI = confidence interval, na = not applicable, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 71 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.6.2: PROGRAM IMPLEMENTATION MODERATORS OF EFFECTS: UNSTANDARDIZED COEFFICIENTS AND 95% CONFIDENCE INTERVALS FROM META-REGRESSION MODELS Individual Models Full Model b 95% CI b 95% CI Program Setting Classroom −0.15* [−0.27, −0.03] Ref. Community 0.05 [−0.11, 0.22] 0.09 [−0.12, 0.31] Other 0.15 [0.00, 0.30] 0.25 [−0.08, 0.57] Provider Health educators −0.02 [−0.02, 0.12] −0.13 [−0.42, 0.15] Classroom teachers −0.03 [−0.21, 0.14] Ref. Other 0.04 [−0.11, 0.19] −0.02 [−0.31, 0.26] Implementation Implementation fidelity 0.05 [−1.65, 1.76] 0.34 [−1.79, 2.46] Mean attendance 0.56* [0.02, 1.10] 1.81 [−2.34, 5.96] Mean retention 0.40 [−0.06, 0.85] −1.01 [−3.84, 1.81] Full model intercept na −0.97 [−3.20, 1.25] b = unstandardized meta-regression coefficients, CI = confidence interval, na = not applicable, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. * p < .05 TABLE 3.6.3: A PARTICIPANT CHARACTERISTIC MODERATORS OF EFFECTS: UNSTANDARDIZED COEFFICIENTS AND 95% CONFIDENCE INTERVALS FROM META- REGRESSION MODELS Individual Models Full Model b 95% CI b 95% CI Participant Characteristics Percentage boys −0.26 [−0.60, −0.08] −0.28 [−0.84, 0.28] Percentage Black 0.00 [−0.21, 0.22] −0.01 [−0.44, 0.42] Percentage Hispanic −0.07 [−0.34, 0.20] −0.02 [−0.43, 0.39] Average age 0.02 [−0.02, 0.06] −0.03 [−0.14, 0.09] Risk (control event rate) 0.18 [−0.10, 0.46] 0.24 [−0.50, 0.98] Full model intercept na 0.48 [−0.86, 1.81] b = unstandardized meta-regression coefficients, CI = confidence interval, na = not applicable, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 72 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.6.4: STUDY METHOD MODERATORS OF EFFECTS: UNSTANDARDIZED COEFFICIENTS AND 95% CONFIDENCE INTERVALS FROM META-REGRESSION MODELS Individual Models Full Model b 95% CI b 95% CI Study Characteristics Randomized controlled trial 0.11 [−0.06, 0.28] 0.12 [−0.29, 0.52] Overall attrition 0.10 [−0.27, 0.48] 0.16 [−0.26, 0.58] Differential attrition −0.35 [−3.40, 2.70] −0.06 [−3.04, 2.92] Active comparison condition −0.02 [−0.18, 0.14] −0.04 [−0.21, 0.13] Full model intercept na −0.05 [−0.54, 0.43] b = unstandardized meta-regression coefficients, CI = confidence interval, na = not applicable, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. TABLE 3.6.5: REGRESSION MODELS EXAMINING MODERATORS OF PARTICIPANT ATTENDANCE RATES Individual Models Full Model b 95% CI b 95% CI Program Type – Tier I −0.06 [−0.15, 0.03] −0.04 [−0.22, 0.14] Program Focus Sexual health 0.03 [−0.07, 0.12] Ref. Youth development −0.09 [−0.19, 0.01] 0.29 [−0.26, 0.85] Other 0.14 [−0.01, 0.29] 0.26 [−0.26, 0.78] Program Components Condom demonstration 0.13* [0.04, 0.22] 0.10 [−0.11, 0.31] Service learning −0.15* [−0.28, −0.01] 0.03 [−0.28, 0.35] Role plays 0.13* [0.05, 0.22] 0.25 [−0.31, 0.80] Games 0.03 [−0.08, 0.15] −0.10 [−0.41, 0.22] Reflective exercises 0.10 [−0.01, 0.20] 0.20 [−0.16, 0.57] Direct provision of health services −0.05 [−0.20, 0.10] −0.24 [−0.93, 0.46] Parent activities 0.11* [0.00, 0.21] 0.10 [−0.21, 0.42] Positive role model −0.13* [−0.26, −0.01] 0.04 [−0.51, 0.59] Group Size Individualized 0.05 [−0.09, 0.19] Ref. Small groups (<10) 0.04 [−0.08, 0.16] −0.22 [−1.04, 0.60] Large groups 0.01 [−0.09, 0.10] −0.13 [−1.00, 0.75] Other (combined individual/group) −0.18* [−0.34, −0.02] Ref. Program Length At least weekly contact −0.11* [−0.22, −0.01] −0.08 [−0.30, 0.14] Contact hours −0.00 [−0.00, 0.00] 0.00 [−0.00, 0.01] Group Composition Gender composition – same gender 0.02 [−0.08, 0.13] −0.01 [−0.21, 0.20] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 73 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Individual Models Full Model b 95% CI b 95% CI Program Setting Classroom 0.04 [−0.05, 0.13] Ref. Community −0.07 [−0.17, 0.04] 0.14 [−0.31, 0.60] Other 0.00 [−0.10, 0.11] −0.01 [−0.28, 0.25] Program Delivery Personnel Health educators 0.00 [−0.09, 0.09] −0.08 [−0.31, 0.15] Classroom teachers 0.08 [−0.06, 0.22] Ref. Other −0.03 [−0.12, 0.06] 0.06 [−0.21, 0.34] Implementation fidelity 0.64 [−0.14, 1.42] 2.03* [0.03, 4.04] Participant Characteristics Percentage boys 0.07 [−0.14, 0.28] 0.23 [−0.44, 0.90] Percentage Black −0.20* [−0.33, −0.06] 0.09 [−0.36, 0.55] Percentage Hispanic 0.04 [−0.11, 0.19] 0.10 [−0.32, 0.51] Average age 0.01 [−0.02, 0.03] 0.09 [−0.08, 0.25] Risk (control event rate) 0.03 [−0.13, 0.20] −0.38 [−1.20, 0.45] Full model intercept na −2.59 [−7.04, 1.85] b = unstandardized meta-regression coefficients, CI = confidence interval, na = not applicable, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. * p < .05 TABLE 3.6.6: REGRESSION MODELS EXAMINING MODERATORS OF PROGRAM RETENTION RATES Individual Models Full Model b 95% CI b 95% CI Program Type – Tier I -0.09 [-0.20, 0.03] -0.05 [-0.31, 0.21] Program Focus Sexual health 0.05 [-0.06, 0.17] Ref. Youth development -0.13* [-0.26, -0.01] 0.39 [-0.41, 1.19] Other 0.16 [-0.03, 0.34] 0.30 [-0.44, 1.05] Program Components Condom demonstration 0.15* [0.04, 0.27] 0.14 [-0.16, 0.44] Service learning -0.23* [-0.39, -0.06] 0.04 [-0.41, 0.49] Role plays 0.17* [0.07, 0.28] 0.32 [-0.48, 1.12] Games 0.04 [-0.11, 0.18] -0.09 [-0.55, 0.37] Reflective exercises 0.11 [-0.03, 0.24] 0.27 [-0.25, 0.79] Direct provision of health services -0.08 [-0.27, 0.11] -0.25 [-1.24, 0.75] Parent activities 0.14* [0.00, 0.27] 0.15 [-0.30, 0.61] Positive role model -0.18* [-0.34, -0.02] 0.04 [-0.76, 0.84] Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 74 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES Individual Models Full Model b 95% CI b 95% CI Group Size Individualized 0.07 [-0.11, 0.25] Ref. Small groups (<10) 0.02 [-0.13, 0.17] -0.35 [-1.53, 0.83] Large groups -0.00 [-0.12, 0.12] -0.24 [-1.49, 1.02] Other (combined individual/group) -0.17 [-0.41, 0.06] Ref. Program Length At least weekly contact -0.13 [-0.26, 0.00] -0.11 [-0.42, 0.21] Contact hours -0.00 [-0.00, 0.00] 0.00 [-0.01, 0.01] Group Composition Gender composition – same gender 0.06 [-0.07, 0.19] -0.01 [-0.30, 0.28] Program Setting Classroom 0.05 [-0.07, 0.16] Ref. Community -0.07 [-0.21, 0.06] 0.16 [-0.49, 0.81] Other 0.01 [-0.12, 0.14] -0.06 [-0.44, 0.32] Program Delivery Personnel Health educators -0.00 [-0.12, 0.11] -0.10 [-0.43, 0.23] Classroom teachers 0.11 [-0.06, 0.28] Ref. Other -0.04 [-0.16, 0.07] 0.07 [-0.32, 0.47] Implementation fidelity 0.98 [-0.00, 1.95] 2.74 [-0.15, 5.63] Participant Characteristics Percentage boys 0.04 [-0.24, 0.31] 0.39 [-0.58, 1.35] Percentage Black -0.21* [-0.39, -0.03] 0.18 [-0.47, 0.84] Percentage Hispanic 0.04 [-0.16, 0.24] 0.17 [-0.43, 0.77] Average age 0.02 [-0.01, 0.05] 0.12 [-0.12, 0.36] Risk (control event rate) 0.13 [-0.10, 0.35] -0.52 [-1.71, 0.67] Full model intercept na -3.81 [-10.21, 2.60] Notes. b = unstandardized meta-regression coefficients, CI = confidence interval, Ref. = reference category. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 75 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.7. Meta-Analysis Using All Effect Sizes The meta-regression analysis results reported in Chapter 6 of the final report use the 119 effect sizes for confirmatory outcomes from the 52 eligible studies that reported such outcomes. In this section, we report results from identical analyses using all 385 effect sizes from the 53 eligible studies. Exhibits 3.7.1 through 3.7.3 correspond to Exhibits 6-1 through 6-3 in the final report. TABLE 3.7.1: RELATIONSHIPS BETWEEN PROGRAM DESIGN FEATURES AND AVERAGE EFFECT SIZES b 95% CI Level of Prior Evidence (Program Tier) Tier 2 program Ref. Tier 1 program –0.09 [–0.20, 0.02] Intercept 0.09* [0.01, 0.17] F =2.57, p = .12 Program Focus Sexual health Ref. Youth development 0.05 [–0.09, 0.19] Other 0.20 [–0.21, 0.62] Intercept 0.02 [–0.04, 0.08] F = 1.00, p = .41 Program Components Condom demonstrations 0.08 [–0.06, 0.21] Service learning 0.08 [–0.36, 0.53] Role plays –0.07 [–0.25, 0.11] Games 0.15 [–0.05, 0.36] Reflective exercises 0.07 [–0.08, 0.22] Direct provision of health services 0.18 [–0.20, 0.56] Parent activities –0.05 [–0.20, 0.11] Positive role model –0.08 [–0.45, 0.27] Intercept –0.04 [–0.08, 0.16] F = 0.62, p = .74 Group Size Individualized Ref. Small groups (<10) –0.15 [–0.35, 0.06] Large groups –0.12 [–0.31, 0.07] Other (mixed individual/group) –0.10 [–0.35, 0.15] Intercept 0.16 [–0.03, 0.34] F = 0.73, p = .57 Group Composition Mixed-gender delivery Ref. Same-gender delivery 0.04 [–0.09, 0.17] Intercept 0.04 [–0.03, 0.11] F = 0.43, p = .52 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 76 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES b 95% CI Single-Gender Targeting Girls only 0.11 [–0.04, 0.26] Intercept 0.03 [–0.03, 0.09] F = 2.71, p = .13 Program Length (Valid k = 52, n = 384) At least weekly contact –0.03 [–0.18, 0.11] Contact hours 0.00 [–0.00, 0.00] Intercept 0.08 [–0.06, 0.21] F = 0.10, p = .91 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 53 studies and 385 effect sizes unless noted otherwise. * p < .05 TABLE 3.7.2: RELATIONSHIPS BETWEEN PROGRAM IMPLEMENTATION FEATURES AND AVERAGE EFFECT SIZES b 95% CI Program Setting Classroom Ref. Community 0.10 [–0.07, 0.27] Other 0.12 [–0.00, 0.24] Intercept 0.00 [–0.07, 0.08] F = 2.17, p = .14 Program Delivery Personnel Classroom teachers Ref. Health educators –0.04 [–0.20, 0.13] Other 0.03 [–0.16, 0.23 Intercept 0.06 [–0.11, 0.22] F = 0.57, p = .58 Implementation Characteristics (Valid k = 42, n = 320) Fidelity –0.04 [–1.59, 1.51] Mean attendance 0.82 [–2.98, 4.63] Mean retention –0.30 [–2.97, 2.37] Intercept –0.37 [–2.34, 1.61] F = 0.93, p = .46 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 53 studies and 385 effect sizes unless noted otherwise. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 77 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.7.3: RELATIONSHIPS BETWEEN PARTICIPANT CHARACTERISTICS AND AVERAGE EFFECT SIZES b 95% CI Participant Characteristics Percentage boys −0.18 [−0.60,0.23] Percentage Black −0.06 [−0.47,0.35] Percentage Hispanic −0.11 [−0.45,0.23] Average age 0.00 [−0.12,0.11] Risk (control event rate) 0.04 [−0.68,0.75] Intercept 0.22 [−1.12,1.56] F = 0.37, p = .86 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 38 studies and 324 effect sizes. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 78 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.8. Relationships between Study Methods and Analysis Results This section provides additional detail on the relationships between study methods and effect sizes. Section 3.8.1 presents results from a meta-analysis including only randomized experiments. Section 3.8.2 explores the relationship between post-test assessment timing and effect sizes. 3.8.1 Meta-Analysis of Randomized Experiments The meta-analysis sample included both randomized experiments (k = 47) and high-quality quasi- experiments (k = 6). Although the meta-regression analysis found no evidence that effect sizes differed systematically between the two types of study designs in this sample, there is a widespread belief among researchers that randomized experiments are less prone to bias. Table 3.8.1 through Table 3.8.5 present results from a meta-analysis of the confirmatory effects from only the 47 randomized experiments. Results are nearly identical to the results from the full sample. TABLE 3.8.1: OVERALL EFFECTS OF TPP PROGRAMS ON CONFIRMATORY OUTCOMES Effect Size Expressed as # of Log Odds Ratio Outcome Construct # of Effect Log Odds [95% Studies Sizes Ratio or p-Value Confidence Reported Hedges’ g Interval] Ever had sex 19 23 0.08 † 0.05 [–0.00, 0.16] Recent sexual activity 15 24 –0.04 0.61 [–0.19, 0.12] Recent unprotected sexual activity 28 44 0.06 0.23 [–0.04, 0.16] Number of sexual partners 2 2 0.08 0.57 [–1.27, 1.44] Proportion of sexual experiences that were 1 1 –0.29 - [–0.85, 0.27] unprotected Ever pregnant/parent 4 4 0.19 0.47 [–0.68, 1.06] Recent pregnancy/parenting 12 12 0.26† 0.05 [–0.00, 0.52] Average effect for all outcomes 47 110 0.08* 0.03 [0.01, 0.16] * p < .05. † < .10 TABLE 3.8.2: RELATIONSHIPS BETWEEN PROGRAM DESIGN FEATURES AND AVERAGE EFFECT SIZES b 95% CI Level of Prior Evidence (Program Tier) Tier 2 program Ref. Tier 1 program –0.14* [–0.28, −0.00] Intercept 0.15* [0.05, 0.25] F =4.28, p = .05 Program Focus Sexual health Ref. Youth development 0.02 [−0.16, 0.21] Other 0.22 [−0.41, 0.85] Intercept 0.06 [−0.01, 0.13] F = 0.41, p = .68 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 79 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES b 95% CI Program Components Condom demonstrations 0.09 [−0.09, 0.28] Service learning 0.18 [−0.37, 0.73] Role plays −0.07 [−0.30, 0.17] Games 0.09 [−0.15, 0.34] Reflective exercises 0.11 [−0.13, 0.34] Direct provision of health services 0.41 [−0.40, 1.22] Parent activities 0.05 [−0.19, 0.29] Positive role model −0.15 [−0.37, 0.06] Intercept 0.03 [−0.07, 0.12] F = 2.28, p = .18 Group Size Individualized Ref. Small groups (<10) −0.23 [−0.51, 0.06] Large groups −0.24 [−0.51, 0.03] Other (mixed individual/group) −0.27 [−0.56, 0.02] Intercept 0.29* [0.01, 0.57] F = 1.55, p = .29 Group Composition Mixed-gender delivery Ref. Same-gender delivery 0.06 [–0.12, 0.24] Intercept 0.06 [–0.02, 0.15] F = 0.54, p = .47 Single-Gender Targeting Girls only 0.14 [–0.06, 0.35] Intercept 0.05 [–0.03, 0.13] F = 2.27, p = .16 Program Length At least weekly contact −0.19 [−0.39, 0.02] Contact hours 0.00 [−0.01, 0.01] Intercept 0.21* [0.01, 0.41] F = 1.46, p = .36 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 47 studies and 110 effect sizes unless otherwise indicated. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 80 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.8.3: RELFATIONSHIPS BETWEEN PROGRAM IMPLEMENTATION FEATURES AND AVERAGE EFFECT SIZES b 95% CI Program Setting Classroom Ref. Community 0.10 [−0.08, 0.28] Other 0.16 [−0.01, 0.33] Intercept 0.02 [−0.09, 0.12] F = 2, p = .17 Program Delivery Personnel Classroom teachers Ref. Health educators 0.03 [−0.16, 0.23] Other 0.07 [−0.14, 0.27] Intercept 0.04 [−0.12, 0.21] F = 0.24, p = .79 Implementation Characteristics (Valid k = 38, n = 94) Fidelity 0.01 [−1.72, 1.74] Mean attendance 3.56 [−1.42, 8.54] Mean retention −2.13 [−5.53, 1.27] Intercept −1.23 [−3.46, 1.01] F = 1.27, p = .34 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 47 studies and 110 effect sizes unless otherwise indicated. * p < .05 TABLE 3.8.4: RELATIONSHIPS BETWEEN PARTICIPANT CHARACTERISTICS AND AVERAGE EFFECT SIZES b 95% CI Participant Characteristics Percentage boys −0.22 [−0.78, 0.33] Percentage Black −0.01 [−0.43, 0.42] Percentage Hispanic 0.00 [−0.42, 0.42] Average age −0.03 [−0.15, 0.09] Risk (control event rate) 0.32 [−0.44, 1.08] Intercept 0.54 [−0.80, 1.88] F = 0.73, p = .62 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 36 studies and 92 effect sizes. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 81 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.8.5: RELATIONSHIPS BETWEEN STUDY METHODS AND AVERAGE EFFECT SIZES b 95% CI Study Method Overall attrition 0.33 [−0.10, 0.77] Differential attrition 1.71 [−1.55, 4.98] Active control group −0.08 [−0.26, 0.09] Study rated inconclusivea −0.25* [−0.49, −0.01] Intercept 0.02 [−0.14, 0.18] F = 1.62, p = .24 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model. a See Farb and Margolis (2016). Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 43 studies and 104 effect sizes. 3.8.2 Post-Test Assessment Timing To determine whether there was a systematic relationship between effect sizes and post-test assessment timing (e.g., if programs were likely to be more effective in the long-term), we conducted two analyses. First, we coded post-test assessment timing as a series of dummy variables corresponding to different timing intervals. Then we conducted a single meta-regression analysis of this moderator block. The results from this analysis, presented in Table 3.8.6, show no evidence of a relationship between post-test assessment timing and effect sizes. However, effect sizes appeared to be somewhat larger for all post-test assessment timing intervals of less than 12 months (with intervals greater than 12 months serving as the reference category). To explore whether there was a difference between intervals greater than and less than 12 months, we coded assessment timing as a binary variable indicating whether the assessment was conducted more than 12 months after the end of the program. The results, shown in Table 3.8.7, again provide no evidence that assessment timing was significantly related to effect sizes. TABLE 3.8.6: RELATIONSHIP BETWEEN POST-TEST ASSESSMENT TIMING AND AVERAGE EFFECT SIZES (FOR TIMING CODED AS A CATEGORICAL VARIABLE) b 95% CI Post-Test Timing Since Program End 0 < X ≤ 3 months 0.09 [−0.25, 0.43] 3 < X ≤ 6 months 0.16 [−0.09, 0.40] 6 < X ≤ 9 months 0.02 [−0.16, 0.21] 9 < X ≤ 12 months 0.61 [−0.34, 1.56] 12 < x months Ref. Intercept 0.01 [−0.07, 0.09] F = 0.71, p = .56 b = unstandardized meta-regression coefficients, CI = confidence interval, F = omnibus F-statistic for meta-regression model, Ref. = reference category. Notes. All meta-regression models estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 52 studies and 119 effect sizes. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 82 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.8.7: RELATIONSHIP BETWEEN POST-TEST ASSESSMENT TIMING AND AVERAGE EFFECT SIZES (FOR TIMING CODED AS A BINARY VARIABLE) b p-Value 95% CI Post-Test Timing Since Program End 12+ months −0.03 0.67 [−0.15, 0.10] Intercept 0.08 0.09 [−0.01, 0.17] b = unstandardized meta-regression coefficients, CI = confidence interval. Notes. Meta-regression model estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 52 studies and 119 effect sizes. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 83 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES 3.9. Overall Effects of Programs That Did and Did Not Report Effect Sizes for Recent Pregnancy Chapter 5 of the report notes that in the analysis including all effect sizes, there was an average program effect on recent pregnancy but not an overall effect on any of the behavioral outcomes that are thought to be precursors to pregnancy (such as recent sexual activity or unprotected sexual activity). A potential explanation for this apparent paradox lies in that only 19 of the 53 studies reported an effect size in the recent pregnancy category, whereas many more studies contributed effect sizes for other behavioral outcomes. In this section, we present overall effects of TPP programs for the sample of 19 studies that reported recent pregnancy effect sizes (Table 3.9.1) and for the sample of 34 programs that did not report effect sizes for recent pregnancy (Table 3.9.2). A third exhibit (Table 3.9.3) shows the relationship between reporting recent pregnancy effect sizes and average effect sizes for other outcomes. TABLE 3.9.1: OVERALL EFFECTS OF TPP PROGRAMS FOR STUDIES WITH REPORTED RECENT PREGNANCY OUTCOME Effect Size Expressed as Log Odds Ratio # of Outcome Construct # of Effect Log Odds [95% Studies Sizes Ratio or p-Value Confidence Reported Hedges’ g Interval] Ever had sex 2 6 0.20 0.19 [–0.57, 0.97] Recent sexual activity 15 74 0.03 0.54 [–0.07, 0.14] Recent unprotected sexual activity 19 92 0.10* 0.02 [0.02, 0.18] Number of sexual partners 4 9 0.04 0.36 [–0.09, 0.16] Sexually transmitted infections 11 11 0.17 0.47 [–0.35, 0.70] Ever pregnant/parent 3 7 0.28 0.19 [–0.49, 1.04] Recent pregnancy/parenting 19 24 0.24* 0.02 [0.04, 0.45] Average effect for all outcomes 19 243 0.11* 0.03 [0.01, 0.21] * p < .05. † < .10 TABLE 3.9.2: OVERALL EFFECTS OF TPP PROGRAMS FOR STUDIES WITHOUT REPORTED RECENT PREGNANCY OUTCOME Effect Size Expressed as Log Odds Ratio # of Outcome Construct # of Effect Log Odds [95% Studies Sizes Ratio or p-Value Confidence Reported Hedges’ g Interval] Ever had sex 27 50 0.03 0.43 [–0.05, 0.10] Recent sexual activity 12 18 0.00 0.96 [–0.11, 0.12] Recent unprotected sexual activity 22 54 0.00 0.93 [–0.11, 0.11] Number of sexual partners 4 6 0.02 0.57 [–0.11, 0.15] Ever pregnant/parent 5 9 0.05 0.78 [–0.48, 0.57] Average effect for all outcomes 34 142 0.01 0.83 [−0.06, 0.08] * p < .05. † < .10 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 84 CHAPTER 3: ADDITIONAL RESULTS AND SENSITIVITY ANALYSES TABLE 3.9.3: RELATIONSHIPS BETWEEN REPORT OF RECENT PREGNANCY OUTCOME AND AVERAGE EFFECT SIZES Outcome Construct b p-Value 95% CI Ever had sex Recent pregnancy outcome reported 0.17 0.22 [−0.45, 0.79] Intercept 0.03 0.47 [−0.05, 0.10] Recent sexual activity Recent pregnancy outcome reported 0.03 0.70 [−0.12, 0.17] Intercept 0.00 0.93 [−0.11, 0.12] Recent unprotected sexual activity Recent pregnancy outcome reported 0.10 0.13 [−0.03, 0.23] Intercept 0.00 0.95 [−0.11, 0.11] Number of sexual partners Recent pregnancy outcome reported 0.03 0.56 [−0.09, 0.14] Intercept 0.02 0.57 [−0.11, 0.15] Ever pregnant/parent Recent pregnancy outcome reported 0.23 0.34 [−0.45, 0.90] Intercept 0.05 0.78 [−0.48, 0.57] All outcomes Recent pregnancy outcome reported 0.09 0.12 [−0.03, 0.21] Intercept 0.02 0.55 [−0.05, 0.09] b = unstandardized meta-regression coefficients, CI = confidence interval. Notes. Meta-regression model estimated using robust variance estimation to handle statistically dependent effect sizes. The analytic sample size was n = 53 studies and 385 effect sizes. * p < .05 Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 85 REFERENCES References Borenstein, M., L. V. Hedges, J. P. T. Higgins, and H. R. Rothstein. 2010. “A Basic Introduction to Fixed-Effect and Random-Effects Models for Meta-Analysis.” Research Synthesis Methods 1: 97– 111. Farb, A., and A. Margolis. 2016. “The Teen Pregnancy Prevention Program (2010-2015): Synthesis of Impact Findings.” American Journal of Public Health (Supplement): s9–s15. Fisher, D. J., A. J. Copas, J. F. Tierney, and M. K. B. Parmar. 2011. “A Critical Review of Methods for the Assessment of Patient-Level Interactions in Individual Participant Data Meta-Analysis of Randomized Trials, and Guidance for Practitioners.” Journal of Clinical Epidemiology 64: 949–967. Glassman, J. R., S. C. Potter, E. R. Baumler, and K. K. Coyle. 2015. “Estimates of Intraclass Correlation Coefficients from Longitudinal Group-Randomized Trials of Adolescent HIV/STI/Pregnancy Prevention Programs.” Health Education & Behavior 42: 545–553. Hedges, L. V. 1981. “Distribution Theory for Glass’s Estimator of Effect Size and Related Estimators.” Journal of Educational and Behavioral Statistics 6: 107–128. Hedges, L. V., E. Tipton, and M. C. Johnson. 2010. “Robust Variance Estimation in Meta-Regression with Dependent Effect Size Estimates.” Research Synthesis Methods 1: 39–65. Higgins, J. P. T., J. J. Deeks, and D. G. Altman, eds. 2008. “Special Topics in Statistics.” Chap. 16 in Cochrane Handbook for Systematic Reviews of Interventions, edited by J. P. T. Higgins and S. Green, 481–529. Chichester, United Kingdom: John Wiley & Sons. Lipsey, M. W., and D. B. Wilson. 2001. Practical Meta-Analysis. Thousand Oaks, CA: Sage. Pustejovsky, J. E. 2015. clubSandwich: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample Corrections. R package version 0.0.0.9000. https://github.com/jepusto/clubSandwich Riley, R. D., P. C. Lambert, J. A. Staessen, J. Wang, F. Gueyffier, L. Thijs, and F. Boutitie. 2008. “Meta‐ Analysis of Continuous Outcomes Combining Individual Patient Data and Aggregate Data.” Statistics in Medicine 27: 1870–1893. Riley, R. D., and E. W. Steyerberg. 2010. “Meta-Analysis of a Binary Outcome Using Individual Participant Data and Aggregate Data.” Research Synthesis Methods 1: 2–19. Sánchez-Meca, J., F. Marín-Martínez, and S. Chacón-Moscoso. 2003. “Effect-Size Indices for Dichotomized Outcomes in Meta-Analysis.” Psychological Methods 8: 448–467. Tanner-Smith, E. E., and E. Tipton. 2014. “Robust Variance Estimation with Dependent Effect Sizes: Practical Considerations Including a Software Tutorial in Stata and SPSS.” Research Synthesis Methods 5: 13–30. Tipton, E. 2013. “Robust Variance Estimation in Meta-Regression with Binary Dependent Effects.” Research Synthesis Methods 4: 169–187. Tipton, E. 2015. “Small Simple Adjustments for Robust Variance Estimation with Meta-Regression.” Psychological Methods 20: 375–393. Tipton, E., and J. E. Pustejovsky. 2015. “Small-Sample Adjustments for Tests of Moderators and Model Fit Using Robust Variance Estimation in Meta-Regression.” Journal of Educational and Behavioral Statistics 40: 604–634. Office of Population Affairs ▌ Website: www.hhs.gov/opa ▌ Email: OPA@hhs.gov ▌ Twitter: @HHSPopAffairs 86