The Social and Character Development Research Program: Development, Goals, and Opportunities Free

School-based social and character development programs that focus on promoting character (i.e., moral virtues, such as honesty, responsibility, and respect), supporting social competencies, and preventing problem behavior (such as noncompliance, conduct problems, and aggression) are attractive to school administrators because of the prevalence of problem behaviors in schools that teachers perceive as interfering with their ability to teach and students’ ability to engage in learning (Mansfield, Alexander, Farris, & Westat, 1991). Although the rate of serious violence in schools is low (for example, violence resulting in school-associated deaths; Centers for Disease Control and Prevention [CDC], 2008), the National Center for Education Statistics (Dinkes, Cataldi, Kena, & Baum, 2006) reports that in 2004, students 12 to 18 years of age were victims of some 1.4 million nonfatal incidents of violence or theft while at school. Younger elementary school-aged students are not immune to such victimization. Responses to the School Survey on Crime and Safety (Dinkes et al., 2006) indicate that 74% of elementary schools experienced a violent incident in the 2003-2004 school year; at a rate of 28 violent crimes per 1,000 students enrolled. The rate of violence was greater in schools with high minority enrollment, students eligible for free/reduced price lunch, and schools in urban areas. In this same school year, 24% of elementary school principals indicated that bullying took place on a weekly basis.

A substantial body of literature has shown that disruptive classroom behavior, conduct problems, aggression, delinquency, and substance use are associated with poor academic achievement, as well as a lack of school connectedness and involvement (Bennett, Brown, Boyle, Racine, & Offord, 2003; Farrington, 1989; Fors, Crepaz, & Hayes, 1999; McCord, Widom, Bamba, & Crowell, 2000; Najaka, Gottfredson, & Wilson, 2001; O’Donnell, Hawkins, & Abbott, 1995; Trzesniewski, Moffitt, Caspi, Taylor, & Maughan, 2006). On the positive side, social competencies (such as peer relationship skills, cooperation, empathy, decision-making skills, emotion regulation, confidence, and self-assurance) have been linked with higher levels of achievement and school adjustment (Carlson et al., 1999; Malecki & Elliot, 2002; Fleming et al., 2005; Wentzel, 1993; Zins, Weissberg, Wang, & Wahlberg, 2004). Further, one school element that has gained increasing attention in recent years is the creation of a positive school climate. Researchers have theorized that the development of a warm, caring community within a school might reduce student problem behaviors, such as aggression and bullying; significant relationships between climate and problem behaviors have been uncovered (Battistich, Solomon, Watson, & Schaps, 1997; McEvoy & Welker, 2000; Orpinas & Horne, 2006; Williams & Guerra, 2007). However, research that directly tests the influence of changes in school climate on problem behaviors (or vice versa) over time has been limited. Yet the abundance of evidence supporting relations among social competence, problem behavior and achievement, and the development of theories surrounding the potential effects of a positive school climate on student outcomes has motivated the development and evaluation, of school-based programs.

To design effective programs to promote social development and prevent problem behavior, it is critical to understand the factors that place youth at risk for engaging in problem behavior (i.e., risk factors), and the developmental factors that buffer youth and protect them from problem behavior (i.e., protective factors). In middle childhood, children typically have well-developed emotional understanding, regulation, and expression (Denham, von Salisch, Olshof, Kochanoff, & Caverly, 2002). Middle childhood is also a time when children’s beliefs about aggression and conflict-resolution skills are developing (e.g., Samples & Aber, 1998). Because these social and emotional competencies underlie the development of socially-competent family and peer relations, psychological science predicts that deficits in these social skills will likely lead to the emergence of problem behavior.

Research evidence points to a lack of problem-solving skills (Slaby & Guerra, 1988), attention problems and hyperactivity (Farrington, 1989; Rey, Sawyer, & Prior, 2005), beliefs about the acceptability of aggression (Guerra, Huesmann, Tolan, VanAcker, & Eron, 1995), and attribution of aggression to others’ innocuous behavior (Dodge & Coie, 1987) as key intrapersonal factors that increase a child’s risk for behavior problems. In addition, family, peer, and community risk factors are influential in the development of problem behaviors, including poor parental supervision and monitoring (Gorman-Smith, Tolan, Zelli, & Huesmann, 1996), association with aggressive peers (Farrington & Hawkins, 1991; Henry, Tolan, & Gorman-Smith, 2001), community disorganization (Sampson, 2000), and exposure to community violence (Schwartz & Proctor, 2000).

For many years, researchers focused on the factors that placed youth at risk for engaging in problem behaviors. There has been increasing recognition, however, that there are factors that may develop social competence and promote health in young people (Catalano, Hawkins, Berglund, Pollard, & Arthur, 2002; Flay, 2002; Weissberg, Kumpfer, & Seligman, 2003). Many protective factors that are hypothesized to promote social competence are also thought to inoculate children from developing externalizing behavior problems when children face adversity. Although the research literature on protective factors that relate to risk has been slow in developing, evidence suggests that socially-adaptive interpersonal problem-solving (Wentzel, 1991), empathy (Schultz, Izard, & Bear, 2004), and parental and teacher use of positive rewards and encouragement to support appropriate behavior (Gorman-Smith, Tolan, Henry, & Florsheim, 2000; Sugai & Horner, 2002) are related to a lower occurrence of problem behaviors.

Importantly, research has highlighted that social competencies, such as emotional understanding and problem solving, are developmentally sequenced. Developmental psychopathology has shown that accumulated failures in the development of competencies compound over time, and set youth on a trajectory for increasing problem behavior (Tolan & Gorman-Smith, 2002). Thus, it is critical to support youth over the full course of development, from early childhood into adolescence.

Public health provides a useful frame for understanding how social and character development can be promoted, and problem behavior can be prevented. Underlying the public health approach is the principle that violence and related problem behaviors are preventable through defining the problem, identifying the factors that increase or decrease risk for violence, developing and testing prevention strategies, and implementing and disseminating effective prevention strategies (Biglan, Brennan, & Foster, 2004; Hamburg, 1998; Mercy, Rosenberg, Powell, Broome, & Broome, 1993). Public health prevention strategies address factors at multiple levels of the social ecology: at the individual, relationship, community, and societal levels. That is, behavior can be modified directly at the individual level, by influencing the close, interpersonal relationships of people and their families, by changing the settings in which people live, and by making more societal, system-wide changes (Dahlberg & Butchart, 2005).

From an interventionist perspective, schools are an obvious target for promoting social development and preventing problem behaviors: schools offer universal access to children over time and, in turn, allow for prevention efforts to reach a large population of youth. Public health prevention approaches to addressing problem behaviors in schools have gained increasing attention, as highlighted by Farrell, Meyer, Kung, and Sullivan (2001) and others (e.g., Leaf & Keys, 2005; Multisite Violence Prevention Project, 2004), because risk and protective factors associated with problem behaviors are commonly related to school and the school environment (e.g., school engagement, school climate norms about the acceptability of aggression), natural opportunities for intervention are available at the individual, relationship, and community levels within school structure (e.g., students, student-teacher relationships, school climate), and schools are places in which youth spend much of their time during their developmental years (Farrell, Meyer, Kung, & Sullivan, 2001).

Elementary school is thought to be a critical time for prevention —the average age at which students begin their pathways into problem behavior is around age 7 (although behavior problems can begin to manifest at younger ages; Thornberry, Huizinga, & Loeber, 2004), most types of antisocial behavior are already evident by third grade (Loeber & Stouthamer-Loeber, 1998); and problem behavior in childhood is predictive of violence and other antisocial behavior later in adolescence and adulthood (Moffitt, 1993).

School-based prevention programs utilize universal, selected, or indicated prevention approaches, or a combination of those approaches (Farrell, Meyer, Kung, & Sullivan, 2001). Universal school-based approaches aim to influence all youth in a school and prevent problem behaviors before they occur; selected approaches reach students at risk of developing problem behavior, and focus on modifying those factors that place those students at risk; and indicated approaches are designed to prevent the accumulation of risk and escalation of students’ existing problem behaviors.

Social Emotional Learning (SEL) programs have been put forth as one universal prevention platform for schools to utilize to develop the whole child by encouraging goal-setting, emotion identification and regulation, responsible decision-making, perspective-taking, and effective interpersonal skill, within a caring and engaging school climate (Greenberg et al., 2003). Other approaches to promoting social development and reducing problem behavior in schools have included Character Education (the teaching of moral values and creating a climate of caring and moral discipline; Lickona, 1993), and Behavior Management strategies (e.g., positive behavior reinforcement, school-wide discipline strategies, and utilizing antecedent control to minimize disruptive and aggressive behaviors and promote prosocial behaviors in all settings in the school; Kazdin, 2001), among others.

Schools commonly use support services, intervention curricula, and discipline management strategies to promote social and character development and prevent problem behavior (Crosse, Burr, Cantor, Hagen, & Hantman; Gottfredson & Gottfredson, 2001; Jones, Fisher, Greene, Hertz, & Pritzl, 2006; Sugai & Horner, 2002), yet evidence of effectiveness is limited. Rigorous evidence of effectiveness comes from evaluations that utilize experimental (i.e., randomized control trials) or quasi-experimental designs (e.g., designs involving matched comparison groups, propensity-score matching, or time series designs) that include developmentally and culturally appropriate outcome measures. There have been some promising, rigorous evaluations of elementary school-based programs (in particular, programs based on social, cognitive, developmental, and ecological theory), and school-based evaluation research has become more advanced in recent years (Flay & Collins, 2005); however, many evaluations have suffered from a lack of rigorous methodology, design, and analysis (e.g., lack of randomization to condition, small sample sizes and low statistical power, severe attrition, and inappropriate level of analysis).

The findings from randomized trials evaluating universal (directed at all students), elementary school-based SACD programs can best be described as illustrating the potentially positive effects that such programs might have on students, teachers, and schools. Although many guides present exemplary/promising programs, or best practices in school-based social development and violence prevention (e.g., the U.S. Department of Education’s Exemplary and Promising Safe, Disciplined, and Drug-Free Schools Programs; the U.S. Department of Justice’s Blueprints for Violence Prevention; the Centers for Disease Control and Prevention’s Best Practices of Youth Violence Prevention) the results of the studies behind the programs highlighted are often mixed. That is, rigorous studies, like the ones reviewed below, have shown significantly positive effects on some outcomes of interest, but indeterminate or nonstatistically significant effects on other outcomes, and even effects in the opposite direction as expected on occasion (Conduct Problems Prevention Research Group, 1999; Flannery et al., 2003; Grossman et al., 1997).

For example, the Conduct Problems Prevention Research Group (1999) matched and randomly assigned 48 elementary schools to either implement a universal, school-based social-skills curriculum (Promoting Alternative Thinking Strategies; PATHS) with teacher development in behavior management and an indicated intervention for high-risk children, or to continue with standard educational practice. At the end of first grade, for three successive cohorts of children, impacts were tested utilizing a variety of measures including teachers’ perceptions of students’ cognitive concentration, conduct problems, prosocial behaviors, emotion regulation, and peer liking; children’s perceptions of other children’s aggressive, hyperactive-disruptive, and prosocial behavior and likeability; and independent observers’ perceptions of classroom atmosphere. Significant impacts were found for children’s perceptions of other children’s aggressive and hyperactive disruptive behavior (compared to children in control classrooms, children in intervention classrooms rated other children lower on aggression and hyperactive disruptive behavior), and for some measures of classroom atmosphere (intervention classrooms were rated as having a more positive atmosphere than control classrooms in terms of students following rules, expressing feelings appropriately, having interest and enthusiasm, and staying focused on tasks); however, no other statistically significant impacts were detected.

Similarly, Grossman and colleagues (1997) found a few specific effects of the Second Step violence prevention curriculum that targets empathy, impulse control, anger management, and problem solving in a randomized field trial of 12 schools. After 1 year of program implementation, small but statistically significant differences were found for independent observers’ observations of students’ aggressive and prosocial behavior; however, no significant impacts were detected for parent and teacher reports of social competence and aggression.

Flannery and colleagues (2003) found mixed effects of a K–5 school-based program: that is, a negative effect of the intervention was detected among mostly significantly positive effects. PeaceBuilders, a universal program that is implemented by all school staff that focuses on school climate and community in addition to behavior management, was randomly assigned at the school level (four treatment schools vs. four delayed-control schools). The researchers hypothesized positive effects on aggressive behavior, prosocial behavior, peace-building behavior, and social competence. After 1 year of implementation, significant positive impacts were detected for teacher-reported aggressive behavior and social competence and child-reported peace-building behavior, with greater impacts for students with higher aggression scores, lower peace-building scores, and lower prosocial behavior scores at baseline. However, a negative effect was noted for child-reported prosocial behavior—students in the PeaceBuilders schools reported less prosocial behavior than students in the delayed control-group schools. The researchers reported null findings for other outcomes (e.g., child-reported aggressive behavior).

How large are the effects of school-based programs, on average, on student outcomes? The most recent meta-analyses by the CDC Task Force on Community Preventive Services (Hahn et al., 2007) demonstrates that universal, elementary school-based programs on average, contribute to an 18% reduction in aggressive and violent behaviors (see also Wilson & Lipsey, 2007). Meta-analyses have indicated differential effect sizes across groups of children and across groups of programs. Effect sizes tend to be larger for interventions with high-risk populations (groups of individuals that experience factors that put them at greater risk for experiencing violence; Wilson, Gottfredson, & Najaka, 2001), as well as for interventions that target only children exhibiting problem behaviors compared to universal interventions that target all students (Lösel & Beelmann, 2003). Effect sizes of school-based “SACD-like” programs (loosely defined as programs that promote social, emotional, and behavioral competencies, such as emotion regulation, conflict resolution, and problem solving) appear to be larger for social competency outcomes than for behavior problem outcomes (Durlak & Wells, 1997; Lösel & Beelmann, 2003).

Given the methodological limitations of many previous studies conducted and published before 2003, the mixed findings of previous rigorous research, and theoretical rationales for conducting school-based programs, researchers called for a greater number of and higher quality evaluation of universal interventions in the elementary school years to assist in our understanding of whether middle childhood is an optimal time for intervention (Tolan, Guerra, & Kendall, 1995). In particular, there was a call for more rigorous evaluations of school-based programs in which a universal approach to development and prevention is utilized; random assignment to condition is employed; control group practice is documented; reliable, valid, and age-appropriate measures from multiple data sources are used; short and long-term follow-up of students is conducted; statistical power is sufficiently determined; appropriate multilevel analyses are conducted; and mediator and moderator variables are adequately explored (Farrell, Meyer, Kung, & Sullivan, 2001; Greenberg, 2004; Weissberg, Kumpfer, & Seligman, 2003).

In response to the call for greater evaluation, the U.S. Department of Education’s Institute of Education Sciences (IES) and the Centers for Disease Control and Prevention’s Division of Violence Prevention (DVP), National Center for Injury Prevention and Control came together in 2003 to develop a research initiative that would provide scientifically sound information on which educators might base their decisions about universal school-based programs. The Social and Character Development (SACD) Research Program was created to systematically evaluate promising school-based programs through supporting university-school partnerships, and bringing together the expertise from education, public health, psychology, economics, and other disciplines to bear on promoting youths’ socioemotional and educational development and preventing youth problem behavior. The program supports the mission of IES by expanding knowledge about practices that improve academic achievement and supporting decision makers who seek out the best available research and data before adopting programs or practices that will affect significant numbers of students. The program also supports DVP’s mission by applying a multidisciplinary perspective to examining the impact of primary prevention programs on violence-related problem behaviors directed at the universal student population.

In 2003, IES solicited cooperative agreement proposals for rigorous evaluations of comprehensive (addresses multiple outcomes), universal (implemented with all students, not just those at elevated risk), school-wide (every grade) programs for Grades K–5, that had some existing evidence of program success or were commonly implemented in schools, and that targeted the following outcomes of interest: social and emotional competencies, prosocial and problem behavior, school climate, and academic achievement [termed “social and character development programs” (SACD) for the purpose of this study]. Potential study investigators and collaborating schools were offered flexibility in choice of intervention, with the constraint that the program was school-wide and aimed to influence the outcomes of interest. At each “research site,” (with site defined as the locations in which the study was implemented by each cooperative agreement awardee; N = 7) investigators were required to randomly assign schools to experimental or “standard educational practice” conditions, and to assess program fidelity in the experimental schools as well as educational practice in the control schools. Potential study investigators and collaborating schools would participate in two types of data collection. The first type of data collection would be designed and administered by study investigators at each research site who would assess outcomes most relevant and proximal to the programs being tested (a “program-specific” evaluation). The second type of data collection was facilitated by an independent evaluation contractor that would allow for an estimation of impacts on primary student and school outcomes across all research sites (a “multiprogram” evaluation). Cooperative agreement proposals were submitted to IES and subjected to a scientific, peer-review process, and the highest quality research proposals were selected for funding. IES also solicited and reviewed separate contract proposals for an independent multiprogram evaluation that would assess impacts across all programs utilizing consistent data collection across sites.

Through this process, seven cooperative agreements were awarded by IES to research institutions, an independent evaluation contract was awarded to Mathematica Policy Research, Inc. (MPR), and a memorandum of understanding was developed with DVP to assist IES in scientific stewardship of the program. Thus, the “SACD Research Consortium” was formed, which constituted primary collaborators at all cooperating institutions. The SACD Research Consortium would collaboratively evaluate the efficacy of seven different elementary school-based programs, with a primary focus on whether or not these “SACD” programs impact multiple outcomes (e.g., empathy, aggression, prosocial behavior, school climate), and a secondary focus on understanding for whom and under what conditions the programs are effective. Figure 1 illustrates the organization of the SACD Research Consortium and the evaluation strategy. A strength of the SACD Research Consortium is the breadth of disciplines represented by the primary collaborators at cooperating institutions. These disciplines include education, public health, psychology, social work, and economics, among others. The collaborators also represent expertise in child development, educational policy, health, program development, measurement, evaluation, statistics, and implementation science. A full list of collaborators and institutions can be found in the authorship note.

The SACD Research Program is evaluating the effectiveness of the following universal school-based programs: the Academic and Behavioral Competencies (ABC) Model, the Competence Support Program (CSP), Love in a Big World (LBW), Positive Action (PA), Promoting Alternative Thinking Strategies (PATHS), Reading Writing Respect and Resolution (the 4Rs), and Second Step (see Flay et al., this volume, for details on the programs). For simplicity, these programs are considered to fall under the umbrella of “Social and Character Development” Programs, but in reality these programs include a wide array of strategies including, but not limited to, value clarification, social skills training, behavior management, and school climate promotion, and are considered to be “SACD-like.” Each program is an individual program under study; no a priori hypotheses were made as to which program would be more effective than the other. The programs are not representative of all universal school-based interventions, but each program is designed to address each of the student and school outcomes prioritized by IES and DVP; that is, students’ social competence, behaviors and academic achievement, as well as school climate. Character education, social-emotional learning, behavior management, violence prevention programs, and other problem-behavior prevention programs generally tend to focus on social competence, behavior, achievement, and climate, and are of great interest to school practitioners. The fragmented evidence base on the efficacy of school-based programs makes it difficult to compare program effects on similar outcomes, given the wide variation in outcome measures utilized by the field. Although each of the programs’ theories of change differ (as described in Flay et al., this volume), and the geographic site for each program differs, it continues to be useful to test program efficacy utilizing consistent outcome measures across programs to better inform how interventions with varying strategies affect the same competencies, attitudes, and behaviors. This multiprogram evaluation approach is in some ways similar to conducting a meta-analysis, in which different programs with similar-enough strategies are combined under one large analysis to determine if the intervention strategy as a whole can impact outcomes; although, it is important to highlight that this study is a stronger test, given that all the outcome measures are exactly the same across program evaluations. It is hoped that this approach results in a significant contribution to both science and practice.

The SACD Research Consortium examined the seven programs through two levels of evaluation: program-specific evaluations carried out by the cooperative agreement investigators, and a multiprogram evaluation carried out by the independent evaluation contractor. In the program-specific evaluations, investigators examined proximal and distal student, teacher, and school outcomes that are aligned with the specific programs’ theories of change. Data were also collected on fidelity of program implementation (see Bickman et al., this vollume, for examples from some program-specific studies). In the multiprogram evaluation, the SACD Research Consortium examined the effects of the programs on broad social development and behavior outcomes, described later in this article and in Kaminski, Battistich, David-Ferdon, and the SACD Research Consortium (this volume).

The multiprogram evaluation seeks to answer the following research questions: What are the effects of seven different universal elementary school-based programs on school climate and student social emotional competencies, positive behaviors, and problem behaviors, and academic competence over three years of implementation? Do program effects vary for students at high versus low individual, family, and community risk at the beginning of the study? As can be seen in Figure 2, the conceptual model integrates predictions based on previous social development and prevention science literature. It was hypothesized that the SACD programs would promote students’ social-emotional competencies, prosocial behavior, and academic competence1; reduce students’ problem behavior; and promote school climate. Social-emotional competencies and school climate were predicted to affect students’ behavior and achievement, and were hypothesized to potentially mediate the effect of the SACD programs on students’ behavior and achievement. It was also hypothesized that program effects would be moderated by level of student risk, fidelity of program implementation, and level of SACD-like activities implemented in control schools as part of standard educational practice. Additional research questions were posed by individual site investigators in their program-specific studies about the effects of individual programs on proximal outcomes tied to program theory, and about the causes, correlates, and effects of program implementation.

The investigators at each research site recruited 12 to 18 schools for the study and defined eligibility criteria for schools within their site. The grade levels encompassed within the schools varied by research site and by location (e.g., some were K–5, some were K–8). Overall, schools were purposively selected based on their lack of school-wide/institutionalized programs addressing “social and character development.” In the best randomized experimental design, it would be optimal to choose schools based on a complete lack of social and character development programming so that (a) schools assigned as control schools could serve as true no-intervention controls, and (b) intervention schools would include only social and character development activities that are part of the intervention being evaluated. Because most schools implement social and character development activities due to federal funding initiatives and state and local policy, selecting schools without any such activities was not feasible. The next best approach was to choose schools without school-wide/institutionalized programs, like the ones being evaluated in the SACD Research Program. Schools were also selected based on their willingness to participate in random assignment and engage in data collection activities and, if assigned to the treatment condition, their willingness and ability to implement the program in all elementary school grades. Examples of other eligibility criteria employed by individual research sites include relatively low performance on standardized tests, relatively high percentage of students eligible for free/reduced price lunch, low mobility rates, and a sufficient number of students per school to meet sample size requirements (more details may be found in Massetti, Crean, Johnson, et al., this volume). Thus, the schools in the SACD Research Program are not necessarily a representative sample of schools in the research sites or across the United States and, in turn, the degree to which the findings from this evaluation are representative of what would be found if the evaluation was carried out in other schools may be limited.

At each research site, schools were matched into pairs, and then one school from each matched pair was randomly assigned to continue with standard practice, and the other school was randomly assigned to implement a universal elementary school-wide program utilizing character education, social-emotional learning, violence-related problem behavior prevention, and/or behavior management strategies. Table 1 shows the research sites, the programs under evaluation, and the number of schools involved in the evaluation. Investigators randomly assigned schools to treatment and control groups using pairwise matching techniques to reduce the likelihood that the small number of treatmentand controlgroup schools would have markedly different characteristics at baseline. Similar matching techniques were utilized across sites so that any differences in study results across sites could not be attributed to differences in random assignment methodology. Investigators at each research site identified variables hypothesized to correlate with the key outcome measures and used these variables to conduct the pairwise matching procedure. Each investigator utilized different student or school characteristic variables in the matching procedure based on his or her best judgment and what data were available. An algorithm developed by researchers at MPR was used to select the best pairs of schools, utilizing the variables identified by each research site. The algorithm selected the best pairs by minimizing the distance between several measurable characteristics for schools within each pair. A set of candidate pairs was selected such that the overall quality of matches across all the schools remained as high as possible, without creating any serious mismatches for any subset of the individual pairs. In some sites, a large pool of schools were identified matched, and randomly assigned to condition; then the top matches of schools were recruited into the study. In other sites, a smaller number of schools were recruited, and then matched and randomly assigned to condition. A description of specific strategies utilized at some of the research sites is provided by Massetti and her colleagues (this volume).

A consistent set of data were collected across all sites by MPR (and its subcontractor, Decision Information Resources) utilizing a set of common outcome measures. Data were collected in two cohorts of schools over sequential school years. In one cohort of schools (N = 84), assessments focused on third grade students in the 2004-2005 school year, fourth grade students the following year, and fifth grade students in the 2006-2007 school year, over five data collection points. In a second cohort of schools (N = 12), assessments focused on third grade students in the 2005-2006 school year, and fourth grade students in the following year, over three data collection points. This second cohort of schools was added to the research program to enhance statistical power to detect program effects at the individual program level.

Students who were in the study schools at the beginning of the study were tracked individually as they progressed through grades within the study schools. New students who entered the schools over the course of the study were recruited into the sample over time. Students who left the target schools over the course of the study were not followed.

The evaluation also recruited teachers and primary caregivers of cohort students to report on students (i.e., for cohort 1 schools, teachers and caregivers of students in the third grade during the 2004-2005 school year, teachers and caregivers of students in the fourth grade during the 2005-2006 school year, etc.), as well all third, fourth, and fifth grade teachers in each year of the study to report on their classrooms and the school. Strategies for consenting students, teachers, and primary caregivers, and challenges faced in the consenting process, are reviewed in Massetti et al. (this volume).

The primary outcomes assessed in the multiprogram evaluation included school climate, social and emotional competencies, prosocial and problem behaviors, and academic competence and engagement. Although each program hypothesized a particular theory of change, all interventions intended to influence similar social and behavioral outcomes, with greater impacts anticipated over time. To evaluate each program’s effectiveness, the SACD Research Consortium focused on a relatively small, but common set of student and school outcomes that generally apply across the seven programs under evaluation. Program impact was hypothesized to be moderated by a variety of factors, including child, parent, and community factors, intervention implementation, and experimental and control teachers’ use of “SACD-like” strategies.

The SACD multiprogram evaluation battery consisted of instruments completed by children, parents, teachers, and principals (with other school staff, if needed, such as school counselors involved in social and character development programming who could assist principals in reporting on activities). Teachers reported on the organizational climate of the school, the SACD-like activities implemented in their classroom, and their own professional background. A Principal Interview was utilized with the school principal and/or other knowledgeable school staff to assess implementation of SACD-like activities in the school. A Child Report administered in classrooms assessed students’ engagement in school, school connectedness, perceived safety at school, and victimization at school; and beliefs about the acceptability of aggression, perceived ability to interact socially with other peers, empathy for others, altruistic behavior, and aggressive behavior. A self-administered Teacher Report on Students assessed, for each student in the study, the involvement of the caregiver in the student’s school life, and each student’s responsibility, social competence, altruistic behavior, aggression, conduct problems, attention deficit hyperactivity disordertype disruptive classroom behavior, and academic competence. A self-administered Primary Caregiver Report assessed caregiver’s perceptions of the child’s responsibility, social competence, altruistic behavior, aggression, and conduct problems. Caregivers also provided information about their involvement with their student’s teacher, their socioeconomic background, family composition, parenting practices, home atmosphere, and risk and protective factors in the community. Kaminski and her colleagues (this volume) provide more detailed information on the assessment battery, and how the measurement model was tested.

To develop a set of core outcome measures with good psychometric characteristics and to determine if a smaller number of scales could represent them, a series of exploratory and confirmatory factor analyses using the fall 2004, spring 2005, and spring 2006 survey data were conducted, as described in detail by Kaminski and her colleagues (this volume). Twenty core student and school climate outcomes were identified through factor analyses and structural equation modeling procedures. Mathematica Policy Research, Inc. (MPR) is employing two analytical approaches to evaluate the impact of the seven programs on the core outcomes. First, to determine if there were statistically significant impacts at each point in time, a series of cross-sectional analyses utilizing multilevel modeling techniques will compare students in control and treatment schools at the end of third grade (spring 2005), fourth grade (spring 2006), and fifth grade (spring 2007). Second, to examine changes in trajectories of student outcomes at multiple points over time (rather than at a point in time), growth curve analyses will examine outcomes across all five time points between fall 2004 and spring 2007. All analyses will control for key covariates; and differences between subgroups, including gender, level of risk near the beginning of the study, whether the student was in the study in the fall of third grade, and fidelity of program implementation, will be examined.

Multiple methods will be used for reporting findings. The findings from the multiprogram evaluation discovered through analyses conducted by the evaluation contractor, MPR, will be published by the SACD Research Consortium through a report released by the Institute of Education Sciences. Additional analyses conducted by SACD Research Consortium members on data generated through the multiprogram evaluation will be published in the empirical literature, as the data for the multiprogram evaluation will be available on a restricted-use basis from IES to eligible researchers. Findings from the program-specific evaluations will also be published in the empirical literature by individual SACD Research Consortium authors.

The SACD Research Consortium hopes to provide valuable scientific information relevant to researchers evaluating school-based SACD and prevention programs and to educators and other practitioners struggling to make choices about adopting evidence-based practices. The unique approach of testing the efficacy of school-based strategies through a rigorous multiprogram evaluation model that assesses impact on the same student, teacher, and school outcomes should allow for a greater understanding of how various universal prevention programs may affect a broad range of social and academic outcomes, and for different populations of students. Findings from the multiprogram and program-specific evaluation studies should speak to the types of outcomes teachers and administrators can expect to see from universal school-based programs that differ in their delivery strategies. Because of the longitudinal nature of the evaluation, findings will also speak to how social and competencies and problem behaviors develop over time in middle childhood, and how school-based intervention strategies may interrupt such developmental trajectories to improve youth outcomes over multiple years of program implementation.

The lessons learned from this program as described in other artilces in this volume are vast, including lessons on the differences between universal prevention strategies, and on how to recruit schools for participation in evaluation studies, assign schools to study conditions, consent study participants, test measurement models for understanding program effects, and assess and encourage quality program implementation. Specifically, the multiprogram evaluation highlights differences among universal prevention strategies in their theories of change and mechanisms for delivery, and supports the vision that drawing across the education, prevention, and health promotion research foundations can facilitate our understanding of the positive development of children and youth. The measurement tools developed for the multiprogram evaluation might provide practitioners with ways to assess the prevalence of and changes in student problem behavior, social competencies, and school climate. These tools assess a variety of conceptually similar programs and, in turn, provide the advantage of saving evaluation resources as programmatic changes may occur over time. The careful attention of research site investigators to program implementation should enhance researchers’ and practitioners’ understanding of where implementation failures may occur, the factors that predict implementation, strategies for measuring implementation, and methods for improving implementation over time. Finally, the rigorous design of the evaluation should provide an opportunity for learning about the challenges and characteristics of intervention research in school settings, including issues of design, pragmatics, and conceptualization. The knowledge generated from the ongoing efforts of the SACD Research Consortium might be useful to other school programming and evaluation endeavors conducted by other researchers and practitioners in the fields of education, positive youth development, and prevention science.

The findings reported here are based on research conducted as part of the Social and Character Development (SACD) Research Program funded by the Institute of Education Sciences (IES), U.S. Department of Education, under contract ED-01-CO0039/0006 to Mathematica Policy Research (MPR), Princeton, NJ, in collaboration with the Centers for Disease Control and Prevention’s Division of Violence Prevention (DVP), and the recipients of SACD cooperative agreements. The SACD Consortium consists of representatives from IES, DVP, and the national evaluation contractor (MPR), and each cooperative agreement site participating in the evaluation. Research institutions in the SACD program (and principal researchers) include: IES Amy Silverman, Edward Metz, Elizabeth Albro, Caroline Ebanks; DVP Tamara M. Haegerich (previously, IES), Corinne David-Ferdon, Le’Roy Reese (Moorehouse School of Medicine; previously DVP); MPR Karen Needels, John A. Burghardt, Heather Koball, Laura M. Kalb, Peter Z. Schochet, Victor Battistich (University of Missouri—St. Louis); Children's Institute Deborah B. Johnson, Hugh F. Crean; New York University J. Lawrence Aber, Stephanie M. Jones (Harvard University), Joshua L. Brown (Fordham University); University at Buffalo, The State University of New York William Pelham, Greta M. Massetti (CDC), Daniel A. Waschbusch; Oregon State University Brian R. Flay, Carol G. Allred (Positive Action), David L. DuBois (University of Illinois at Chicago), Michael L. Berbaum (University of Illinois at Chicago), Peter Ji (University of Illinois at Chicago), Vanessa Brechling, (University of Illinois at Chicago); University of Maryland Gary D. Gottfredson, Elise T. Pas, Allison Nebbergall; University of North Carolina at Chapel Hill Mark W. Fraser, Thomas W. Farmer (Penn State University), Maeda J. Galinsky, Kimberly Dadisman; and Vanderbilt University Leonard Bickman, Catherine Smith.

Notice of potential conflict of interest: The research described herein was done using the program and the training and technical support of Positive Action, Inc. Dr. Flay’s spouse holds a significant financial interest in Positive Action, Inc.

Portions of this article may appear in the expanded SACD multiprogram evaluation impact report authored by the SACD Research Consortium to be released by the Institute of Education Sciences. The authors wish to thank the Consortium reviewers who commented on earlier drafts of this article. The findings and conclusions in this report are those of the authors and do not represent the official position of the Institute of Education Sciences, Centers for Disease Control and Prevention, Mathematica Policy Research, Inc., or every Consortium member, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. Correspondence concerning this article should be addressed to thaegerich@cdc.gov