Effects of a School Based Program to Improve Adaptive School Behavior and Social Competencies Among Elementary School Youth: The Living Skills Program

The development of positive character through character education is a primary concern in educational settings (cf. Lickona, Schaps, & Lewis, 2007). Character education in a school social setting establishes the foundation upon which favorable academic, social and moral decisions will be made throughout the lifespan, and the very term “character education” highlights the fact that character is developed or taught through social training and interpersonal experience (cf. Cheung & Lee, 2009). The social aspects of character education are fundamental to self-esteem and teach young people about friendship, prosocial behavior, positive decision making, interpersonal morality, social responsibility, cooperation and reciprocity (Cheung & Lee, 2009; Miller, Kraus, & Veltkamp, 2005).

The elementary school years of a child provide an important context for the development of the social skills and positive character children need to successfully engage with their peers and thrive in an academic setting. Without the development of adequate social skills in the early school years, youth become increasingly likely to develop a myriad of co-occurring and causally related problems.

Poor social skills exhibited during the early school years predict poor peer relationships, including peer rejection and isolation (Pope, Bierman, & Mumma, 1991; Shields, Ryan, & Cicchetti, 2001; Wood, Emmerson, & Cowan, 2004). Poor social skills have been found to be correlated with or even predict school behavioral problems (Domagala-Zysk, 2006; Pope et al., 1991), antisocial behaviors (Arthur & Kuperminc, 1991), academic failure (Epstein, Kinder, & Bursuck, 1989; Hawkins, Farrington, & Catalano, 1998), and psychological disorders (Pedersen, Vitaro, Barker, & Borge, 2007). A lack of social skills is associated with peer and teacher descriptions of rejected youth as aggressive, disruptive and uncooperative (Coie & Kupersmidt, 1983) as well as lonely, depressed, and anxious (Bierman & Schwartz, 1986; Pedersen et al., 2007). Moreover, poor social skills developed as a child predict future interpersonal problems, including adult and marital relationship problems (Critchfield & Benjamin, 2008). Exacerbating the problem is the fact that social skills development reaches a critical developmental stage between ages 6 and 11 (Grades 1-6) or the period known as middle childhood (Erikson, 1950). It is at this age that social development shifts from the family to school, the community and peers. A review of the literature on social development indicates that these elementary school years are paramount to successful development over the lifespan; successful navigation through the psychosocial developmental period forms the basis of future positive peer interaction, acceptance versus rejection or isolation and avoidance of antisocial behaviors (Bagwell, Newcomb, & Bukowski, 1998; Parker, Rubin, Price, & DeRosier, 1995).

Adaptive social skills and positive peer relationships are, conversely, related to resiliency factors and positive behavioral outcomes. Research indicates that social skills training improves interpersonal communication and peer relationships, as well as increases school bonding (Hawkins, Smith, & Catalano, 2004). Research by Catalano, Haggerty, Oesterle, Fleming, and Hawkins (2004) indicates close peer relationships are reciprocally related to school bonding, and both school bonding and adaptive social skills have been shown to reduce peer rejection, delay the initiation of substance use and decrease the likelihood of substance use, delinquent behavior and anti-social behaviors (including aggressiveness, noncompliance, and disruptiveness). They have also been shown to increase academic motivation, academic performance, self-esteem, self-efficacy, and the quality of peer relationships (cf. Catalano et al., 2004; Maddox & Prinz, 2003).

The link between poor social skills and a multitude of associated negative outcomes, as well as positive outcomes associated with improved social skills, highlights the need for social skills training and intervention for youth exhibiting social skill deficits at a young age. Of particular interest to the Living Skills program described in this evaluation is the development of social skills to improve peer relationships, school bonding and problem school behavior (including antisocial behavior) in order to decrease the incidence of negatively associated outcomes while simultaneously bolstering positively associated outcomes. The Living Skills school social competency training curriculum addresses youth skill sets within three primary, research based goals. Goals include: (1) reduce the incidence of antisocial peer interaction and behavior, (2) improve interpersonal competencies in interactions with peers and subsequent peer acceptance, and (3) improve school bonding conducive to positive social and behavioral norms and better learning environments.

Goal one: Reduce Antisocial Peer Interaction And Behavior. Skills training programs aimed at altering behavioral problems and anti-social relationship patterns in the school environment have shown considerable promise (Lochman, 1992; Prinz, Blechman, & Dumas, 1994). Social skills training programs have been shown to reduce antisocial behaviors including aggression (Bierman & Schwartz, 1986) and violence (Fraser, Day, Galinsky, Hodges, & Smokowski, 2004). The Seattle Social Development Project (SSDP; Catalano et al., 2004;,Hawkins et al., 2004) emphasizes child social skills development through cooperative interaction and learning as well as alternative problem solving beginning in fifth grade. Program evaluation results presented in Catalano et al. (2004) indicate that those who received social skills training were less likely to commit criminal offenses or engage in violent behavior.

Goal Two: Improve Interpersonal Competencies and Peer Acceptance. The use of social skills training to improve social competence and peer acceptance is well established (cf. Smokowski, Fraser, Day, Galinsky, & Bacallao, 2004). Social skills training has been demonstrated to increase positive peer responses and peer acceptance, as well as to improve behavioral problems, when combined with the opportunity to rehearse the targeted skills (cf. Bierman, 1986; Bierman, Miller, & Stabb, 1987; Ladd & Mize, 1983). For example, Bierman et al. (1987) utilized a social skills training program with boys Grades 1-3. Results of their study indicated that boys provided with social skills instruction were, at a follow-up test, initiating and receiving more positive interaction with peers relative to a group not receiving the skills training. They were also less likely to be rated as “disliked” by peers relative to pretest scores.

Ladd (1981) trained 36 sixth grade children in social skills shown to be effective in improving peer acceptance through improved social competency. Skills included: asking questions about peers, leading peers, and offering support to peers. Results of the study indicated significantly greater use of the first two skills by children in the skills training group relative to a comparison group; moreover, children who participated in skills training showed significant improvement on measures of peer acceptance at both posttest and a follow-up test.

Goal Three: Improve School Bonding And Teacher Preferred Behaviors.

Research evidence suggests that school bonding, or the connectedness a student has to his or her school, its personnel and its ideals (Maddox & Prinz, 2003), is an important determinant of successful development across both social and academic domains. The SSDP research (introduced above; Catalano et al., 2004; Hawkins et al., 2004) also illustrates the importance of school bonding in increasing school social competency and associated outcomes. As part of the social skills training, the program emphasizes bonding to the school through social activities. Results presented in Catalano et al. (2004) indicated that both school and peer bonding were related to several positive attitudinal and behavioral outcomes. The SSDP reduced rates of drinking and smoking and contributed to postponement of both alcohol and tobacco use. Increased school bonding was negatively related to other substance use and future substance dependency. Those bonded to school also displayed higher academic performance. Bryant, Schulenberg, Bachman, O’Malley, and Johnston (2000) also provides support for poor academic performance and higher incidence of anti-social behavior as correlates of poor school bonding, and Wade and Brannigan (1998) found poor school bonding to be a significant predictor of risky behaviors.

Another study using the SSDP’s model, conducted by Williams, Ayers, Abbott, Hawkins and Catalano (1999), suggests that the impact of school bonding on problem behavior is mediated by social skills. Eight hundred eight students participated in the SSDP program from Seattle area schools. Analyses examined both Caucasian and African American fifth graders, and findings of the path analysis indicated that, regardless of ethnicity, poor academics and poor social skills partially mediated the relationship between school bonding and both antisocial behaviors and substance abuse. In other words, higher levels of school bonding were causally associated with greater academic performance and better social skills, which, in turn, were causally related to less participation in antisocial behaviors and less substance abuse. These findings illustrate the importance of both school bonding and the development of social skills in reducing problem behavior.

The Living Skills program is driven by the perspective that youth can improve their social skills through a group process oriented program that teaches and reinforces prosocial behaviors learned through positive social interaction. In accord with program goals established and developed with an appreciation for the youth intervention literature (above), the Living Skills program is a group based program implemented in schools that focuses on improving interpersonal interaction. The Living Skills program addresses goals established above by implementing three theoretically derived program objectives for successful social skills development (objectives are reciprocal, not mutually exclusive): (1) provide safe group environments with norms that foster positive interaction with peers, (2) solidify group norms, skills and desired behaviors by providing positive models and rewards for normative prosocial behaviors, and (3) teach social and life skills conducive to prosocial behavior, positive peer interaction, and school bonding. Details about how program methodology meets these research based objectives are described in the methods section.

Specific hypotheses about the Living Skills program are derived from the goals and objectives of the program. There are three primary hypotheses: (1) program participation will improve social competencies and skills for social interaction, (2) program participation will increase positive peer social interaction while subsequently reducing negative interaction, and (3) program participation will improve teacher preferred behaviors related to classroom expectations and conducive to a favorable learning environment.

Participants for the Living Skills program are selected through a referral process. Teachers, counselors, and community youth group leaders identify youth, grades two through five, who are perceived as at-risk on one or more of the factors addressed by the Living Skills program, including rejection and isolation, aggressive and disruptive behaviors, attention problems, high impulsivity, poor school bonding and poor academic performance.

From an environmental perspective, individuals selected for participation in Living Skills typically come from low-income homes, impoverished living conditions, single-parent homes, homes with high family conflict, neighborhoods with high levels of mobility, high availability of drugs, and/or the presence of gangs, and from families with a history of drug abuse. The two most common reasons for referral to the Living Skills program, however, are peer rejection and disruptive social/class room behavior.

The Living Skills program is designed to be equally effective for students Grades 2 through 5 and for both males and females. This age group was specifically targeted because previous research, reviewed above, emphasizes the importance of social skills development no later than the elementary school years. The program’s evaluation was based in Salt Lake City, Utah; hence, most of the program participants are Caucasian or Hispanic; however, the program was not designed with specific ethnicities in mind. The program was designed as a generalizable intervention strategy implemented within the school setting (however, its applicability beyond the Hispanic and Caucasian populations has not yet been determined).

Assignment to either a comparison or program group is based on a blocked random assignment strategy to ensure that cell sizes for each group are relatively similar. Living Skills program staff collects referral forms until approximately 14 youth (Grades 2 through 5) from a school are available for participation. From this blocked group of participants, a coin flip decides the assignment to condition for each participant until half of the participants are assigned to either the comparison or the program group. Youth participants assigned to the comparison group are offered program services later in the school year.

Because the nature of the program dictates a need for groups that can potentially form cohesive bonds, a few exceptions to the group assignment strategy are necessitated each year. For example, sometimes siblings are separated so that new social (nonfamilial) bonds can more easily develop, or sometimes youth are removed from a specific program group because the group has too many youth with one specific risk factor, such as hyperactivity, which threatens group cohesion. This reassignment represents movement within program groups, or from one program group to another.

Occasionally reassignment occurred not to a different program group, but from the comparison group to the program group. In a small number of cases in which siblings were participating in the study together, and in which one sibling was randomly assigned to the program group while the other was assigned to the comparison group (approximately 14 instances over 5 years), the comparison group sibling was reassigned to the program group (but to a different program group). Due to the family involvement components of some program sessions (described below), all comparison group participants with a program group sibling would have naturally been exposed to at least some elements of the program; accordingly, they would not provide a true comparison group participant. This occasional reassignment is responsible for the slightly discrepant sample sizes indicated in the results section.

Removal and reassignment of both types (from program group to program group and from comparison group to program group) occurred concurrently with assignment to condition; a child was never removed from a group once the group sessions began or once he or she was informed of his or her condition. While these instances of nonrandom assignment were rare, and not ideal, a mixed factorial design was implemented (more below) to help attenuate potentially negative effects due to nonequivalent groups.

The Living Skills program is implemented with youth in groups of six to eight (with an average of seven). Groups are composed of second and third graders only or fourth and fifth graders only. The program consists of 10-12 biweekly classes. Students are generally required to complete at least 80% of the classes; however, exceptions are made in select cases for children who are sick for extended periods or out of school for valid reasons. No exceptions are made for students with less than 50% attendance. Classes are 45 minutes to 1-hour long, and take place during the school day. The Living Skills program is designed to be conducted both during school and in an after school setting. However, no after school sessions occurred during this evaluation and, hence, no after school data are included in the analyses.

Youth are removed from their normal classroom environment to participate in the Living Skills program. Professionally trained Prevention Specialists serve as facilitators and direct the group’s structured activities and games. Lessons are taught using reading materials and stories and through a variety of interactive strategies, such as participation in group activities and role-playing, creating art projects and through cooperative games that reinforce the curriculum. Individual attention, small group interaction and positive reinforcement support the learning of the group (more below).

The first implementation objective of the Living Skills program rests on the importance of a highly structured atmosphere that supports a safe learning environment where youth participants can overcome maladaptive social interaction patterns and a history of negative social experiences with peers. The children in the Living Skills program have typically experienced rejection from peers and some level of isolation as a result of behavioral problems and poor social skills. A cohesive and supportive group structure is viewed as a key mechanism for promoting growth among each individual participant.

The goal of the Living Skills program facilitators is to create a milieu that promotes opportunities for a “corrective emotional experience” (Yalom, 1995) to help youth participants explore new and more positive ways of interacting with others. The corrective emotional experience provides youth the opportunity to revisit past, maladaptive patterns of behavior and to interact in a more favorable and safe environment in order to learn and practice adaptive behaviors. According to Yalom, to be corrective, the new group environment must have the following qualities: (1) the group must be perceived as safe in order for emotions to be expressed without fear of reprisal, and (2) the group reaction must be supportive and without punishment, ridicule or rejection. Only under these circumstances can one replace maladaptive reactions with more favorable reactions to emotional experiences. To this end, Living Skills facilitators are trained to be positive, honest and nonjudgmental while also maintaining an open dialogue conducive to processing the corrective experiences of group members and facilitating the development of positive group norms.¹

The Living Skills group structure emphasizes the establishment and reinforcement (discussed below) of a strong set of pro-social group norms that promote a safe sharing environment (cf. Tuckman, 1965; Yalom, 1995). Normative rules for each Living Skills group are generated in an interactive manner during the first session of every group, with both youth participants and Living Skills facilitators providing input to norm establishment. While youth are encouraged to participate in development of group norms, they are always carefully guided in their choices, and “key” norms, facilitating program goals and objectives, are always part of the final set of norms. Key normative rules focus on general group appropriate behaviors such as being respectful and responsible, as well as peer and classroom appropriate behaviors such as raising hands before speaking, not interrupting others, listening when others speak, keeping hands and feet to themselves, and so forth. By including youth in the development of the group’s normative rules, there is increased “buy in” from participants.

When the Living Skills group begins, participants, initially, are tentative about the purpose of the group and their place within the group. The norms for interactions provide a reliable and safe environment for participants (cf. Tuckman, 1965). Once established, the environment allows youth to safely interact with other group members while learning new behavior patterns and reactions to social-emotional situations to which, in the past, they may have responded in maladaptive ways. This occurs because the norm structure of the program ensures that others’ reactions and behaviors remain orderly, nonjudgmental and constructive.

In the Living Skills program, norms are further established by the second implementation objective of the program: indirect and direct reinforcement of norms, skills and desired behaviors. The Living Skills program utilizes direct positive reinforcement in the form of verbal (i.e., praise) and tangible rewards (e.g., candy), as well as opportunities for observational learning through modeling and vicarious reinforcement (see Bandura, 1977). These processes help create group environments fostering improved social competency and school bonding.

Children are particularly sensitive to direct rewards as guides for both pro- and antisocial behavior. Allen, Hart, Buell, Harris, and Wolf (1964) have shown reinforcement strategies to effectively increase positive peer interaction, and Forehand and McMahon (1981) used reinforcement to reduce child noncompliance. In the Living Skills program, direct verbal praise is emphasized and praise is specific so children know what the praise is acknowledging. For example, rather than saying “Good job,” a facilitator might say “You said cars were hard for you to draw, but look: you did it!” or “That was a nice thing you did sharing your paper.” Though verbal praise is preferred, children in the Living Skills program also receive tangible rewards for desirable behaviors. Typically, these rewards involve small pieces of candy.

Punishments are typically avoided in Living Skills unless a child’s actions present an immediate threat to others’ wellbeing. Research has shown punishment to be less effective than other reinforcement strategies (cf. Powers, 2001). When children act inappropriately, operant extinction is typically used in the Living Skills program. Operant extinction involves the removal of reinforcement rather than punishment (Powers, 2001). Rather than saying “Don’t do that” to a child, facilitators are taught to ignore undesirable behaviors (to the extent possible), thereby removing the attention and reinforcement the child sought. Even initially uncooperative children eventually begin to seek rewards by adopting the appropriate group behaviors, and, in turn, they are reinforced directly.

Children have also been shown to be particularly sensitive to models of behavior, and modeling has been shown to decrease problem behaviors in children, such as social withdrawal, social anxiety, hyperactivity and disruptive behaviors (Melamed & Siegel, 1975; O’Connor, 1969; Ollendick, 1986), and increase adaptive responses, such as increased socialization (O’Connor, 1969; see also Ollendick & King, 1998). Whether a child acts in prosocial ways or antisocial ways is largely determined by the models in his or her environment. When models of negative behavior are present in the environment, children have been shown to exhibit increases in aggression (Silvern & Williamson, 1987), and both peer and parental modeling of alcohol use and substance abuse predict positive attitudes toward use (Capuzzi & Lecoq, 1983) and early initiation of use by children (Braucht, Brakarsh, Follingstad, & Berry, 1973; Catalano & Hawkins, 1996).

In Living Skills, individual sessions provide the opportunity for children to model positive behaviors through activities and skills practice (discussed below). Vicarious learning occurs when one child observes another receive rewards or praise for positive behaviors. Because opportunities for only positive modeling are provided by the Living Skills content, eventually the modeled behavior becomes part of the group’s norms.

In the Living Skills program, appropriate behaviors matching group norms are modeled by children in the group as well as by the facilitator. For example, having received reinforcement for listening or sharing in the past, children actively practice these skills in the group sessions. The facilitator might observe and comment on a child’s modeling of cooperative behavior accordingly: “I like the way you are listening,” or “I like the way you are sharing.” A disruptive, socially isolated or uncooperative child, therefore, sees both the appropriate behavior being displayed and the fact that positive consequences follow appropriate behavior. Perhaps most importantly, children also learn that the group is both a safe and rewarding place for participation, free of ridicule or negative consequences.

Both direct and indirect reinforcement based learning strategies have been shown to guide the formation of our attitudes, and lead to internalization of prosocial attitudes that guide subsequent behaviors (see Petty & Cacioppo, 1986). Thus, both direct and indirect reinforcement guide behavior toward established group norms when rewards for following norms and models of norms are present. The eventual consequence of both reinforcement strategies in the Living Skills program is a group with well-established norms for behavior, which also provides the opportunity to safely rehearse desirable behaviors and interaction strategies (cf. Yalom, 1995).

The third implementation objective of the Living Skills program is learning and rehearsal of new social skills. As discussed previously, social skills training has been utilized with great success in past research (see Supporting Research section above). Each of the 10 Living Skills sessions deals with either forming a safe and cohesive group (discussed above) or teaching skills necessary for successful interpersonal interaction; the program uses activities and or games to provide opportunities for practice. For example, the first group activity emphasizes “put ups” in place of “put downs.” Children are asked to state “one thing” they like about each group member and the facilitator. The activity emphasizes the positive emotional experience associated with complimenting others and being complimented instead of the negative emotions associated with “put downs.”

The Living Skills sessions deal with topics including group formation and norm building, learning appropriate behaviors through reinforcement and modeling, understanding personal differences, self-concept, cooperation, understanding feelings, expressing feelings, anger management and stress management. The skills taught as part of the Living Skills program have been demonstrated in past research to increase social acceptance during the grade school years (cf. Bierman et al., 1987; Hartup, 1983). All lessons are reinforced with activities and opportunities to interact as well as practice new skills. The activities and skills build upon one another as the course progresses. Youth are encouraged to implement the newly developed skills and to share strategies with peers, siblings and parents.

Measurement of the program’s efficacy in meeting its goals and objectives was accomplished using two instruments designed to measure psychosocial adjustment and problem behavioral tendencies within the school context: (1) the Walker-McConnell Scale (WMS) of Social Competence and School Adjustment (Walker & McConnell, 1995a, 1995b, and (2) the IOWA Conners Teacher Rating Scale (IOWA CTRS; Loney & Milich, 1982). Both scales are designed to be completed by participants’ classroom teachers rather than by youth or the Living Skills facilitators. Facilitators do not see the pre- or posttest ratings, but, out of necessity, teachers are not blind to condition (i.e., they inevitably know which students are removed from their classrooms for Living Skills participation, and they are an active part of the referral process). Despite this possible limitation, teacher completed instruments were deemed the best choice for evaluation of student change. Clearly, facilitators would be highly likely to show bias toward program improvement, as child improvement is equally a marker of a child’s success in the program and the facilitator’s success at his or her job. Evaluations by peers and self-report evaluations were also a concern due to possible validity issues with self-report data from populations of this age range (particularly in making judgments about how others perceive them). Peer evaluations were a particular concern due to anonymity issues and legal issues surrounding whether schools could allow peer evaluations. Because teachers interact with and observe participants’ behavior on a daily basis, they are well qualified to assess the classroom and peer interaction behaviors of interest. Additionally, they are the group that is least invested in the success of the Living Skills program; they have no performance evaluation concerns of their own (with respect to the program), and their evaluations have relatively few validity issues compared to evaluations by peers or the youth themselves.

The WMS was included in order to measure program efficacy related to adaptive behavior and interpersonal social competence goals. It was designed as a tool to measure the effectiveness of social skills intervention programs in elementary school age youth. The WMS is a 43-item scale containing three subscales. Ratings on the WMS are provided by classroom teachers based on their observations of youth in school contexts.

Subscale one, the teacher preferred social behavior subscale, contains 16-items assessing social behavior highly valued by teachers (e.g., “shows sympathy for others,” “controls temper,” etc.), and measures characteristics which promote sensitivity, cooperation and self-control. Subscale two, the peer preferred social behavior subscale, contains 17-items that assess social behavior that is highly valued by peers (e.g., “plays or talks with peers for extended periods of time,” “compromises when the situation calls for it,” etc.). This subscale measures characteristics governing positive and cooperative peer interaction. Subscale three, the (positive) school adjustment behavior subscale, contains 10-items that assess behavioral skills important to teachers and conducive to positive learning experiences in the classroom (e.g., “attends to assigned tasks,” “has good work habits,” etc.).

Items on the WMS are rated by teachers on a 5-point scale ranging from (1) “never occurs” to (5) “frequently occurs.” The WMS total score and subscale scores are scored in a manner such that higher scores indicate higher functioning and greater competency, while lower scores indicate lower functioning and fewer competencies. Subscale and total scores for the WMS are calculated as sums; thus, the range of values for the total score is from 43 to 215. For the teacher preferred social behavior subscale, the range of values is from 16 to 80 (normative α = .96); the range for the peer preferred social behavior subscale is 17 to 85 (normative α = .95), and 10 to 50 (normative α = .96) for the school adjustment behavior subscale. The instrument’s normative validation data (Walker & McConnell, 1995b) included a sample of 1,812 male (51%) and female (49%) students from kindergarten to Grade 6 from four U.S. “census zones” (West, North Central, Northeast, and South) comprised of 15 states. As with the sample in the current research (below), the validation sample was predominately Caucasian (80%). Evidence from the validation study indicated differentiation of norms by grade and gender was not necessary. Further validation information and normative values can be found in Walker and McConnell (1995b) and in Table 1. In the current research, the WMS was completed by teachers both before the program began (pretest) and after the program period ended (posttest).

The IOWA CTRS was included in the evaluation to measure program efficacy with respect to reducing inappropriate classroom behavior that precludes learning. The scale consists of 10 items designed to assess symptoms of hyperactivity, attention deficits and aggression. Classroom teachers are asked to rate the frequency with which youth exhibit the ten different behavioral tendencies. Ratings are made on a 4-point scale ranging from (0) “not at all” to (3) “very much.”

The IOWA CTRS is composed of two subscales. The first subscale (normative α = .87) assesses behaviors related to inattention and overactivity (“IO;” e.g., “fidgets,” “hums, makes other odd noises,” “excitable, impulsive,” “inattentive, easily distracted,” and “fails to finish things he [sic] starts”). The second subscale (normative α = .83) assesses oppositional and defiant behavioral tendencies (“OD;” e.g., “quarrelsome,” “acts smart,” “temper outbursts,” “defiant,” and “uncooperative”). The IOWA CTRS total score and subscale scores are scored in a manner such that higher scores indicate greater frequency of attention deficit and hyperactive behaviors; hence, lower score are preferable. Possible scores for both subscales of the IOWA CTRS range from 0 to 15, and the total score ranges from 0 to 30. The instrument’s normative validation data (Pelham, Milich, Murphy, & Murphy, 1989) included a sample of 293 boys and 315 girls in two Florida Schools. As with the sample in the current research, the validation sample was predominately Caucasian (70%). Data were collected on youth from kindergarten to Grade 5. Further validation information for the scale can be found in Loney and Milich (1982), and normative outcomes by grade and sex (discussed more below) can be found in Pelham, Milich, Murphy, and Murphy (1989) and in Table 2. In the current research, the IOWA CTRS was completed by teachers both before the program began (pretest) and after the program period ended (posttest).

The Living Skills program evaluation included data from 645 youth in 5 cohorts from 2002-2007. Across the 5 cohorts of students, data were collected from 336 program participants (52.1%; 10 facilitators directed all classes over the time span) and 309 comparison group participants (47.9%) from 14 schools in Salt Lake County, UT. Of the 645 participants, 61.2% were male, 35.5% were female, and 3.3% did not have a sex specified for them. Participants’ grade levels ranged from grade two to six: 24.3% were in grade two, 27.3% were in grade three, 22.8% were in grade four, 22.5% were in grade five, and 0.3% were in grade six (2.8% did not have a grade specified for them). The majority of participants were Caucasian (59.1%). Hispanics represented the second largest group (21.1%), followed by African American (4.0%), other American (not specified; 1.6%), Native American, (1.4%), Pacific Islander (1.2%) and Asian American (1.2%). The ethnicity item was not completed in 10.4% of cases.

One hundred twenty youth participants from the program and comparison groups had pretests, but no posttests completed by teachers. This outcome could have occurred for several reasons, including children moving (the target population is known to be high in mobility) and, though rare, children not completing the program by choice. An initial concern, however, regarding attrition was that some youth might have dropped from the evaluation because they were assigned to the nontreatment, comparison group and were not receiving immediate services for behavioral problems. To test this concern, a test of proportions was conducted examining whether the attrition rates differed between the program and comparison groups as a function of the total population of each. Results revealed that participants not completing the study were equally likely to be program group (14.4% of all program group participants) or comparison group (16.8% of all comparison group participants) participants, p = .41.

Additional concerns about attrition related differences and group equivalency were examined using univariate analyses of variance (ANOVAs). Two (group: program or comparison) by two (attrition: pretest only or pre- and posttest) ANOVAs were conducted using the five pretest (baseline) subscale scores as the dependent variables in order to test for preexisting differences between groups. No significant differences were found on any of the three subscales from the Walker-McConnell or the two subscales from the IOWA CTRS. At pretest, program group participants did not differ from comparison group participants on any subscale, ps > .200, and those with no posttest did not differ from program completers, ps > .350. Also, none of the interactions was significant between group and attrition at pretest, ps > .080.

To ensure the program reached targeted, at-risk youth, pretest scores on the WMS and IOWA CTRS were examined to determine the preexisting risk levels (measured by pretest) of youth selected for the Living Skills sample (in both the program and comparison group). Normative means, standard deviations and groups sample sizes provided in Walker and McConnell (1995b) were compared to data from the sample involved in the Living Skills evaluation using independent t-tests. The tests revealed the Living Skills sample to be significantly more at-risk than the normative population on all three WMS subscales (see Table 1). For the IOWA CTRS, normative data provided by Pelham et al. (1989) also included normative means, standard deviations and sample sizes. The normative values are established by grade and sex. The normative values for grades are established for Grades 2 to 3 and Grades 4 to 5. Comparisons between these normative data and the Living Skills sample data on the IOWA CTRS’ subscales were made using independent t-tests. The tests revealed the Living Skills sample to be significantly more at-risk than the normative population on both IOWA CTRS’ subscales for both grades and both genders (see Table 2).

Analyses of the primary hypotheses were next conducted on the three subscales from the Walker-McConnell and the two subscales from the IOWA CTRS. Significance test outcomes and descriptives (in terms of the interaction effects only) from repeated measures ANOVAs on all subscales and the total scores are presented in Table 3. Significance tests for the overall or total scores are not discussed, however, as the total scores are redundant with their respective subscales. In the discussion of outcomes by subscale, main effects are presented first so that they can be interpreted in terms of the more important interaction effects that follow. Nonsignificant main effects are not presented. Where interaction effects were significant, follow-up examinations of simple main effects are also provided at group level using the Bonferroni adjusted comparisons of p = .025.

On the Walker-McConnell peer preferred social behavior subscale (pretest α = .95, posttest α=.95), a main effect for time was found, Wilks’ Lambda =.88, F(1,595) = 81.99, p = .000, h² = .12, indicating that posttest scores (M = 53.69, SD = 13.13) were more favorable than pretest scores (M = 50.57, SD = 13.33). A significant interaction was also found between time (pre or post) and group (program or comparison), Wilks’ Lambda =.99, F(1,595) = 7.62, p = .006, h² = .01. Examination of the simple main effects indicated comparison participants improved significantly over time, F(1,288) = 19.15, p = .000, h ²= .06, as did program group participants, F(1,307) = 72.26, p = .000, h² = .19. Although both groups improved over time, examination of the simple main effects in conjunction with the significant interaction suggests program group participants improved at a greater rate than comparison group participants (see Table 3), indicative of an enhanced effect due to program participation.

Analysis of the teacher preferred social behavior subscale (pretest α = .95, posttest α = .96) also revealed a significant main effect for time, Wilks’ Lambda = .92, F(1,591) = 50.76, p = .000, h² = .08, indicating that posttest scores on the scale (M = 49.67, SD = 12.86) were more favorable than pretest scores (M = 47.09, SD = 12.42). A significant interaction was also found between time and group, Wilks’ Lambda = .97, F(1,591) = 15.85, p = .000, h² = .03. Examination of the simple main effects indicated comparison participants improved significantly over time, F(1,282) = 19.15, p = .020, h² = .02, as did program group participants, F(1,309) = 57.71, p = .000, h² =.16. Although both groups improved over time, examination of the simple main effects in conjunction with the significant interaction suggests program group participants improved at a greater rate than comparison group participants (see Table 3), indicative of an enhanced effect due to program participation.

Finally, the school adjustment subscale (pretest α = .94, posttest α = .95) revealed a main effect for time, Wilks’ Lambda =.94, F(1,605) = 41.6, p = .000, h² = .06, indicating that posttest scores (M = 30.60, SD = 9.02) were more favorable than pretest scores (M = 29.09, SD = 9.12). A significant interaction was also found between time and group, Wilks’ Lambda = .99, F(1,605) = 6.97, p = .008, h² = .01. Examination of the simple main effects indicated comparison participants improved significantly over time, F(1,290) =8.09, p = .005, h² = .03, as did program group participants, F(1,315) = 37.45, p = .000, h² = .11. Although both groups improved over time, examination of the simple main effects in conjunction with the significant interaction suggests program group participants improved at a greater rate than comparison group participants (see Table 3), indicative of an enhanced effect due to program participation.

Analysis of the IO subscale (pretest α = .85, posttest α = .87) of the IOWA CTRS revealed only a significant interaction between time and group, Wilks’ Lambda =.97, F(1,620) = 17.47, p = .000, h² = .03. Unlike the WMS subscales, a more theoretically ideal interaction emerged with the IOWA CTRS’ subscales. Examination of the simple main effects indicated comparison participants’ scores became non-significantly less favorable over time, F(1,293) = 2.97, p = .086, h² = .01, while program group participants’ scores became significantly more favorable, F(1,337) = 19.78, p = .000, h² = .06. Examination of the simple main effects in conjunction with the significant interaction suggests program group participants improved significantly over time while comparison group participants did not (see Table 3).

Analysis of the OD subscale (pretest α =.91, posttest α =.92) also revealed only a significant interaction between time and group, Wilks’ Lambda = .96, F(1,620) = 25.27, p = .000, h² = .04. Examination of the simple main effects indicated comparison participants’ scores became significantly less favorable over time, F(1,294) = 9.72, p = .002, h² = .03, while program group participants’ scores became more favorable, F(1,326) = 16.18, p = .000, h² = .05. Examination of the simple main effects in conjunction with the significant interaction suggests program group participants improved significantly over time while comparison group participants’ scores became significantly worse (see Table 3).

The Living Skills program was designed and implemented to be equally effective with males and females, youth of both Caucasian and Hispanic ethnicity, and youth in Grades 2 to 5. Therefore, differences as a function of sex, ethnicity, or grade were not expected. Differences as a function of the year youth were involved in the program (over the 5 year period) were also not expected. To test for equal program effectiveness across these groups, a series of repeated measures ANOVAs were conducted on the three subscales from the Walker-McConnell and the two subscales from the IOWA CTRS.

Sex and group were analyzed as between participants variables along with time as a within participants variable on all five subscales. Results revealed no three-way interactions on any of the three subscales from the Walker-McConnell or the two subscales from the IOWA CTRS, ps > .40. Accordingly, outcomes by sex were statistically equivalent and the program’s effectiveness did not differ statistically by sex.

The Living Skills program is not specifically designed to apply only to Caucasian and Hispanic youth, but the program participants are predominately from these ethnicities as a function of the ethnic composition of Salt Lake County. Accordingly, program directors wanted to ensure the program is equally effective with these predominant ethnic groups (further research will be conducted to test the effectiveness of the program with other ethnic groups as sufficient sample sizes for other groups become available). For this analysis, only participants who were Caucasian or Hispanic were included. This led to the removal of 61 cases (10.6% of cases reporting ethnicity) of varying ethnic backgrounds. Ethnicity and group were analyzed as between participants variables along with time as a within participants variable on all five subscales.

Results revealed no three-way interactions on any of the three subscales from the Walker-McConnell Scale, and no significant three-way interaction on the IOWA CTRS’ OD subscale, ps > .25. However, the inattention/overactivity subscale from the IOWA CTRS did reveal a significant three-way interaction, Wilks’ Lambda = .99, F(1,494) = 6.16, p = .013, h² = .01. To further examine the interaction, ethnicity was split and the two-way interactions between group and time were examined via repeated measures ANOVA.

Considering only the Hispanic group, a significant two-way interaction was found, Wilks’ Lambda =.87, F(1,129) = 18.79, p = .000, h² = .13. Examination of the simple main effects indicated that outcomes on the subscale for the comparison group became less favorable from pre- (M = 4.87, SD = 3.76) to posttest (M = 5.87, SD = 3.74), F(1,514) = 9.45, p = .003, h² = .16, while outcomes for the program group became more favorable from p (M = 5.62, SD = 4.06) to posttest (M = 4.89, SD = 3.86), F(1,78) = 9.05, p = .004, h² = .10.

Examination of the Caucasian group revealed no two-way interaction between time and group, Wilks’ Lambda = .99, F(1,365) = 1.98, p = .160, h² = .01. pre- (M = 6.97, SD = 3.92) to posttest (M = 6.94, SD = 4.40) change for the comparison group and pre (M = 6.88, SD = 3.85) to posttest (M = 6.44, SD = 3.91) change for the program group were not examined further due to the lack of a significant interaction.

Grade (2 through 5) and group (program or comparison) were analyzed as between participants variables along with time as a within participants variable on all five subscales. Results revealed no three-way interactions on any of the three subscales from the Walker-McConnell or the two subscales from the IOWA CTRS, ps > .35. Accordingly, outcomes by grade were statistically equivalent, and the program’s effectiveness did not differ statistically by grade.

Because Living Skills program data was collected over a 5-year period, it was important to determine whether outcomes differed by year and, consequently, whether it was appropriate to combine data across years. To test for differences across program years, a Repeated Measures ANOVA was conducted entering year of administration and group as between participant variables, and time as the within participant variable. Results revealed no three-way interactions on any of the three subscales from the Walker-McConnell or the two subscales from the IOWA CTRS, ps > .05. Outcomes across years were statistically equivalent.

Findings from the evaluation’s primary analyses suggest the Living Skills program is an effective intervention to reduce school problem behavior and increase social competencies among at-risk youth Grades 2 to 5. While effect sizes ranged from small to moderate (cf. Cohen, 1988), favorable results were consistently found across measures. Even though effects were small to moderate, results from all indicators of improvement revealed significant interaction effects, and follow-up test of the simple effects indicated the Living Skills program significantly improved peer preferred behavior, teacher preferred behavior, positive school adjustment, and significantly decreased inattention/overactivity and oppositional/defiant behaviors over time relative to a comparison group that did not receive the program.

While favorable outcomes were found across the subscales, the patterns of change differed by scale. On the Walker-McConnell subscales, both groups showed significant improvement overtime, but the improvement was greater in the program group. For the IOWA CTRS’ subscales, outcomes for comparison group students became either significantly less favorable over time (OD subscale) or remained the same (IO subscale), while program group participants improved significantly on both subscales.

The different pattern of outcomes may have occurred due to the differing nature of the domains measured by the two instruments. Outcomes measured by the Walker-McConnell focus on teacher and peer related social skills and behaviors. These outcomes may be inherently more malleable over the course of the school year as students become more familiar and comfortable interacting with the teacher and other students in their classroom and school. Through the social skills curriculum, the Living Skills program might accelerate this process for at-risk youth, thereby explaining why both groups improved, but program participants improved to a greater extent.

Alternatively, the IOWA CTRS’ subscales measure outcomes that are potentially indicative of clinically abnormal behaviors. In particular, the IOWA CTRS has been extensively used as a screening instrument for Attention Deficit Hyperactivity Disorder and Oppositional Defiant Disorder (cf. Waschbusch & Willoughby, 2008). Given the more clinical nature of these issues, individuals who score high on these attributes may be more likely to require intervention to show improvement. To the extent that this theory is accurate, program group participants should be more likely to improve on these outcomes given that they are the only group receiving a targeted intervention. This may explain why Living Skills participants showed significant declines in abnormal behavior while untreated comparison group participants’ abnormal behaviors became worse or remained the same.

An analysis of factors moderating the generalizability of the Living Skills program demonstrated Living Skills to be effective with both males and females, and with youth grades two to five. In the case of four of five scales, the Living Skills program was equally effective among Caucasian and Hispanic youth. On one subscale, the IOWA CTRS’ OD subscale, the Living Skills program did not produce improvement for Caucasian youth over time relative to the Caucasian comparison group. Hispanic program group youth, however, showed significant improvement over time relative to the Hispanic comparison group. Overall, the Caucasian population was more at-risk than the Hispanic youth at pretest.

It is interesting, though not surprising, that posttest means for the program group did not meet or exceed the favorability of the normative averages on any of the subscales from the Walker-McConnell or the IOWA CTRS (normative values are available in Tables 1 and 2 and can be compared to posttest values for the program group in Table 3). In other words, the at-risk students from the study’s population were still more at-risk than the normative average by posttest or following the Living Skills program. Because the group was composed of very high-risk students (students referred to the program due to being high-risk), improvement at or beyond the normative average is likely not a reasonable goal.

An important limitation of the evaluation of the Living Skills program is that Living Skills participants, and their comparison group counterparts, are not precluded from participation in other programs designed to improve the lives of at-risk youth. Program wait-listed youth and their parents may seek other programs and resources while waiting for assignment to the Living Skills program group. Additionally, Living Skills group participants and their comparison group counterparts may be exposed to universal programs taking place in the school or community as a whole.

The ways in which exposure to other programs might affect the Living Skills evaluation depends on how the exposure occurs. To the extent that wait-listed youth were exposed to other interventions or programs, external program participation would likely hinder the detectable efficacy of the Living Skills program, as other programs might produce positive changes in comparison group youth, creating a nonideal comparison group. In light of this fact, the significant findings discussed above are encouraging. Alternatively, it is also possible that universal programs aimed at improving the life skills of youth were also available in schools, thereby impacting both program and comparison group participants. However, these universal programs would be expected to impact both program and comparison group youth equally. The demonstrated impact of the Living Skills program (provided by the interactions in the mixed design) would thus be above and beyond any universal programs in which youth participate.

Although only one subscale’s outcome indicated an ethnicity based effect, the outcome suggests the Living Skills program was particularly effective for Hispanic youth with respect to improving inattention and overactivity problems. However, it would be premature to assume this outcome to be a consistent feature of the Living Skills program as it was not hypothesized a priori. Future research involving the Living Skills program will be conducted to elucidate any potential ethnic differences impacting the program’s generalizability to Hispanic and Caucasian target groups.

The Living Skills program staff has always been open to improving the program and its effectiveness. Future research will incorporate a referral form requiring teachers to rate (on a 5-point Likert scale) youth on several factors associated with risk for social and school related problems (such as isolation, rejection, aggressiveness, defiance, poor attitudes toward school, etc.). This enhancement will benefit the program’s effectiveness in two primary ways: (1) it will allow Living Skills staff to evaluate how the program impacts youth referred to the program for these specific issues, and (2) it will allow program staff to refine the Living Skills program to more specifically address the specific issues of youth with qualitatively different social or behavioral problems.

The teacher-based rating system described in this evaluation will also be addressed in future research. While teachers were arguably the best evaluators of student improvement, the decision to use teacher evaluations was also not without flaw. As explained above, teacher evaluations could be influenced by the fact that they know the condition to which a student was assigned; this may create some demand characteristics.

Peer evaluations were not used because of a number of logistical concerns, including concerns about asking peers to rate specific children on their social skills (and, correspondingly, getting schools to allow such ratings) and concerns about age related biases. However, obtaining peer ratings from a sample of classmates on a trial basis will remain a goal of future research. Such ratings would prove valuable by providing convergent validation of teacher ratings, and may more directly speak to the issue of whether peer acceptance improved as a result of the program. To the extent that both teachers and peers rated at-risk youth as having improved, future evaluations could more assuredly argue the robust nature of the Living Skills program’s efficacy. Future research will also consider asking the at-risk students to rate their experiences with the program and their personal feelings of social growth as a result of the program. While these ratings will likely prove biased, it will be important to know whether at-risk youth enjoyed the program and recognized its benefits to their personal growth.

Another limitation of the current evaluation is the lack of a long-term follow-up evaluation conducted after the posttest. While an examination of the long-term impact of the Living Skills program would certainly be valuable, there are many logistical issues (including financial and participation compliance issues) that precluded the ability of the provider agency to conduct a follow-up survey during this evaluation’s time frame. However, because the Living Skills program has gained acceptance and recognition in the local schools as an efficacious program, long-term follow-up evaluation of student progress has become more viable currently relative to the past. Future evaluation efforts will explore options for collecting follow-up data and address sustainability issues.

This research was supported by funding from Salt Lake County Substance Abuse Services, United Way of Salt Lake, and the Center for Substance Abuse Prevention (CSAP) Service to Science Initiative. Authors would also like to acknowledge the following individuals for their contributions to the Living Skills program and its research during the program evaluation: Janet McCullough, Program Manager; Skye Phillips, Prevention Specialist; and the prevention services staff.