The Impact of Student Success Skills on Students’ Metacognitive Functioning in a Naturalistic School Setting

Brett Zyromski, Melissa Mariani, Boyoung Kim, Sangmin Lee, John Carey

This study evaluated the impact of the Student Success Skills (SSS) classroom curriculum delivered in a naturalistic setting on the metacognitive functioning of 2,725 middle and high school students in Kentucky. SSS was implemented as one intervention to fulfill an Elementary and Secondary School Counseling Grant. Results in students’ self-reports indicated that those who received the intervention demonstrated increased ability to regulate their levels of emotional arousal. No additional significant differences were found. These findings differ from the results of previous outcome studies involving SSS. Implications for implementing SSS in naturalistic school settings and directions for future research are discussed.

Keywords: Student Success Skills, naturalistic, metacognitive functioning, classroom curriculum, emotional arousal

The purpose of this study was to evaluate the impact of the Student Success Skills (SSS) school counseling curriculum (Brigman, Campbell, & Webb, 2004; Brigman & Webb, 2012) delivered in a naturalistic setting on students’ metacognitive functioning. In this case, the authors use the term naturalistic setting to describe a typical school environment, one which lacks the additional supports (e.g., hiring national trainers) that would be present in a more controlled research study. SSS is an evidence-based, school counselor-delivered, social-emotional learning intervention that is designed to support students by teaching them three integral skill sets: (a) cognitive and metacognitive skills (e.g., goal setting, progress monitoring and memory skills); (b) social skills (e.g., interpersonal skills, social problem solving, listening and teamwork skills); and (c) self-regulation skills (e.g., managing attention, motivation and anger). Research has identified this curriculum as important in promoting students’ academic achievement and success in school (Collaborative for Academic, Social, and Emotional Learning, 2015; Webb & Brigman, 2006).
SSS was designed based on reviews of educational psychology research literature that identified critical skills such as information processing, emotional self-management, and positive social skills needed for student success (Bransford, Brown, & Cocking, 1999; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Greenberg et al., 2003; Hattie, Biggs, & Purdie, 1996; Marzano, Pickering, & Pollock, 2001; Masten & Coatsworth, 1998; Wang, Haertel, & Walberg, 1994; Zins, Weissberg, Wang, & Walberg, 2004). The curriculum (Brigman & Webb, 2012) can be delivered in two formats: (1) the SSS classroom program and (2) the SSS small group program (Brigman et al., 2004), both of which are intended for use with students in grades 4 through 12. SSS is a highly structured, manualized program that consists of weekly 45-minute lessons. The classroom format includes five lessons, while the small group program includes eight lessons. Both sets of weekly lessons are intended to be delivered in chronological order over the corresponding number of consecutive weeks. A 45-minute booster session is delivered once a month for 3 months in the spring.

Developers of SSS designed the curriculum to follow a scripted, manualized format; implementers are encouraged to follow the sequencing, format and language provided in order to ensure fidelity of treatment. If practitioners go “off-script” or change the recommended delivery of the lessons, it may result in less favorable outcomes that might not meet the same levels as have been found in past research. That being said, the SSS program comes with detailed manuals that include recommended verbiage, descriptive diagrams and supplemental handouts (Brigman & Webb, 2012). During each lesson, students learn and practice strategies in five distinct areas: (a) goal setting, progress monitoring and success sharing; (b) creating a caring, supportive and encouraging classroom environment;
(c) cognitive and memory skills; (d) calming skills; and (e) building healthy optimism. Specific strategies are taught and practiced each week and are reviewed and reinforced during subsequent lessons. Between lessons, students are encouraged to apply the new strategies that were taught and to work on the academic and personal goals that they set for themselves during the SSS session. Teachers also are expected to cue students to use the strategies during regular classroom lesson time.

 

Research has established the effectiveness of SSS in several quasi-experimental and experimental studies. SSS implementation has resulted in enhanced academic achievement as measured by standardized achievement tests (Brigman & Campbell, 2003; Brigman, Webb, & Campbell, 2007; C. Campbell & Brigman, 2005; Webb, Brigman, & Campbell, 2005) and district math and reading achievement measures (Lemberger, Selig, Bowers, & Rogers, 2015). Two studies have suggested that the effects of SSS on academic achievement are at least partially mediated by changes in students’ metacognitive functioning (Lemberger & Clemens, 2012; Lemberger et al., 2015). Lemberger and Clemens (2012) found that participation in SSS small groups was associated with improvements in students’ executive functions (as measured by the Behavior Rating Inventory of Executive Function [BRIEF]; Gioia, Isquith, Guy, & Kenworthy, 2000) and increased metacognitive activity (as measured by the Junior Metacognitive Awareness Inventory [Jr. MAI]; Sperling, Howard, Miller, & Murphy, 2002). Lemberger et al. (2015) found that participation in the classroom version of SSS was associated with improvement in executive functions (as measured by the BRIEF-SR; Guy, Isquith, & Gioia, 2004).

 

While researchers have established the efficacy of SSS in well-controlled research environments, little is known about its effectiveness when delivered in naturalistic settings. The purpose of the present study was to measure the effectiveness of the SSS curriculum when delivered in a naturalistic setting within regularly functioning schools. In this study, SSS was implemented in five schools in a district in Kentucky as part of a school counseling improvement project funded by an Elementary and Secondary School Counseling Demonstration Grant awarded by the U.S. Department of Education. The five middle and high schools collaborated together through a regional educational cooperative to apply for the grant. Demographic information for each school can be found in the Setting section below. The grant funded all necessary SSS curriculum materials and provided support for school counselors in evaluating the impact of the program. However, funding was not available to hire national trainers. National trainers are not a requirement when purchasing a manualized school intervention and many schools do not possess the funds needed to hire national trainers. Thus, this funded project provided an opportunity to evaluate the effectiveness of SSS in a naturalistic school setting.

 

The primary evaluation question was: When implemented in a naturalistic setting, does SSS impact students’ metacognitive functioning, as determined by (1) knowledge and regulation of cognition as measured by the Junior Metacognitive Awareness Inventory (Jr. MAI; Sperling et al., 2002) and (2) use of skills related to self-direction of learning, support of classmates’ learning, and self-regulation of arousal as measured by the Student Engagement in School Success Skills survey (SESSS; Carey, Brigman, Webb, Villares, & Harrington, 2013)? The secondary question was: Does the magnitude of any changes in metacognitive functioning depend on the degree to which SSS was implemented with fidelity?

Method

 

Setting

Five participating schools were chosen due to their participation in the Elementary and Secondary School Counseling grant. Specific population and demographic data related to all five schools can be found in Table 1.

 

Table 1

 

Brief Description of the Five Schools in the Study

 

School Enrollment& Grades Served Location Ethnicity Rates Gender Rates Socio-Economic Indicators
School 1 1,484 StudentsGrades 9–12 Suburban/Rural African American-0.8%American Indian-0%Asian American-1.2%

European American-95.8%

Hispanic American-0.9%

Two or More Races-1.2%

 

Female-50.2%Male-49.8% Qualifying for Free
Lunch-28%Qualifying for Reduced Lunch-7.6%
School 2 1,154 StudentsGrades 6–8 Suburban/Rural African American-1.1%American Indian-0.1%Asian American-0.7%

European American-94.1%

Hispanic American-2%

Two or More Races-2%

 

Female-49%Male-51% Qualifying for Free Lunch-33.5%Qualifying for Reduced Lunch-7.8%
School 3 325 StudentsGrades 7–12 Urban African American-2.8%American Indian-0%Asian American-0.6%

European American-95.1%

Hispanic American-1.2%

Two or More Races-0.3%

 

Female-54.5%Male-45.5% Qualifying for Free Lunch-68.6%Qualifying for Reduced Lunch-8.6%
School 4 570 StudentsGrades 6–8 Rural African American-1.4%American Indian-0%Asian American-0.2%

European American-97.5%

Hispanic American-0.5%

Two or More Races-0.4%

 

Female-48.4%Male-51.6% Qualifying for Free Lunch-50.5%Qualifying for Reduced Lunch-7.9%
School 5 471 StudentsGrades 6–8 Urban African American-11.5%American Indian-0.4%Asian American-0.4%

European American-69%

Hispanic American-10%

Two or More Races-7.9%

 

Female-45.2%Male-54.8% Qualifying for Free Lunch-62.2%Qualifying for Reduced Lunch-6.4%

Note: The demographic categories are listed as reported by the state reporting system.

 

Participants

A total of 2,725 students participated in the study with roughly equal numbers of male (50.1%) and female (49.9%) students. A relatively large percentage of participants (41.2%) qualified for free or reduced lunch. Less than 1% of participants were classified as Limited English Proficient and 11.2% qualified for special education services. In terms of racial and ethnic diversity, the participants were less diverse than desired. Eighty-five percent of participating students identified as White (non-Hispanic), 4% as Multiracial, 6% as African American, and 4% as Hispanic. All other groups combined accounted for the remaining 1% of participants. Proportionally more sixth-, seventh-, and eighth-grade students participated in the study. The percentages of participants by grade were: 20.5% sixth grade; 20.9% seventh grade; 20% eighth grade; 10.7% ninth grade; 9.9% tenth grade; 9.2% eleventh grade; and 8.7% twelfth grade.

 

Completed Jr. MAI surveys (pretest and posttest) were obtained from 1,565 students (57% of the total). Completed SESSS surveys were obtained from 1,612 students (59% of the total). School counselors were required by the grant to serve as point persons for the delivery and collection of pre- and post-instruments. School counselors also were trained on instrument collection using a structured, scripted protocol. Instruments were administrated in paper format. Although trained in data collection procedures, not all participating school counselors successfully captured both pre- and post-assessments. Issues around successfully collecting paper instruments contributed to the loss of data. For example, 4% of the Jr. MAI surveys were incomplete and 4% of the SESSS surveys were incomplete. Rather than estimating missing data in these instances, it was determined that only data from fully complete instruments would be used in the analyses.

 

Preparation of the School Counselors

An SSS manual was provided to each of the schools the year prior to implementation of the program. School counselors reviewed the SSS materials and met with the grant project manager to discuss the content and instructional processes associated with the SSS classroom guidance interventions. Again, school counselors within the five schools did not receive formal training from the national SSS trainers on how to implement the curriculum. Formal training was a cost not included in the grant. The lack of formal training reflected the more naturalistic approach to the study. More often than not, school counselors that purchase a manualized program do not have the funding to hire national trainers to guide implementation. School counselors review the manual and follow the manualized program during implementation. This approach was reflected in this study. School counselors at each school used the SSS curriculum manuals and did their best to adhere to the recommended lesson sequence and scripts (Brigman & Webb, 2012). Next, the school counselors at each school conducted a pilot SSS small group in the spring semester, prior to the onset of the whole-school SSS implementation. The intent of the pilot implementation was to ensure that the school counselors were thoroughly familiar with the materials and implementation procedures. After conducting the pilot studies, the school counselors provided 3 hours of SSS training to partner teachers prior to implementation of the whole-school SSS intervention. The teachers in all schools also received a copy of the SSS classroom manual (Brigman & Webb, 2012) to review prior to implementing the curriculum with their students. Delivery of the SSS classroom format began the subsequent fall semester (2013) and concluded the following spring (2014).

 

 

Procedures for Delivering SSS and Fidelity Issues

In every school, the school counselors experienced some problems implementing the SSS curriculum with fidelity. Implementing the program with complete fidelity would have reflected the exact scope and sequence scripted within the SSS manual, mainly delivering the program over a 45-minute time period once a week for 5 consecutive weeks. Schools varied from this scope and sequence, resulting in a lack of fidelity of the recommended implementation format for SSS. However, schools adjusted delivery of SSS in a way that reflected their educational priorities, again reflecting a more naturalistic approach than a traditional controlled research study. The primary resistance school counselors encountered related to teachers’ and administrators’ reluctance to lose instructional time as a consequence of in-class implementation of SSS. Each school identified a contextually appropriate approach to addressing this initial resistance to devoting class time to SSS. In two of the schools, teachers (rather than counselors) delivered the SSS curriculum within their own classrooms. In the other three schools, the SSS curriculum was delivered through learning communities (i.e., advisories), which are existing scheduled blocks of time during which teachers facilitate small groups (8–15 students) outside their normal classrooms. For example, a ninth-grade biology teacher might be responsible for leading a group of students across several grades with whom the teacher does not interact outside of this learning community. The manner in which SSS was delivered in each school is detailed below.

 

School One. The SSS curriculum was not delivered with complete fidelity. Instead, the school leadership determined it was more feasible for teachers to deliver the curriculum through learning communities, once a week for 30 minutes for 10 weeks.

 

School Two. The SSS curriculum was delivered with reasonable fidelity once a week for 60 minutes over a 5-week period by teachers. However, instead of being delivered in a traditional classroom format, the school leadership determined it more feasible to deliver the curriculum through learning communities.

 

School Three. SSS was delivered with reasonable fidelity. Five teachers (trained and supervised by the school counselor) delivered the SSS curriculum in the prescribed format detailed in the SSS classroom manual. The five teachers delivered SSS to all students in the school through various courses, including a study skills course (seventh and eighth grades), a social studies course (ninth grade), a biology course (tenth grade), a college readiness course (eleventh grade) and an English course (twelfth grade).

 

School Four. The SSS curriculum was delivered with reasonable fidelity to all grade levels (sixth through eighth) during social studies courses. The social studies teachers were trained and supervised by the school counselor to deliver the program.

 

     School Five. The SSS curriculum was not delivered with complete fidelity. The school leadership determined it more feasible to deliver the curriculum through learning communities two to three times a week for 25 minutes for each session over a 5-week period.

 

Procedures for Collecting SESSS and Jr. MAI Data

Pretest data were collected at the beginning of the 2013 school year and posttest data were collected in late April and early May of the 2013–2014 school year, before the end-of-grade standardized testing ensued. Prior to beginning the project, school counselors completed a Collaborative Institutional Training Initiative to alert them to issues relating to voluntary participation and confidentiality. School counselors were then trained to follow an instrument administration manual (developed for the project) so that they could administer the SESSS and the Jr. MAI in a standardized fashion. They administered the SESSS and the Jr. MAI using standardized, scripted procedures. In order to protect the confidentiality of the students, school counselors changed the student identification numbers for each student by adding a randomly determined number for each school. No other person besides the school counselor knew the number by which the student identification numbers were changed. All data were kept in a locked file cabinet in the primary investigator’s office. Data were entered into a database, which was saved on an encrypted, password-protected hard drive. As an additional safeguard, the data from each school were saved on an external hard drive and transported by hand to the primary investigator’s office.

 

Instruments

 

Junior Metacognitive Awareness Inventory (Jr. MAI)

The present study used the 18-item version of the Junior Metacognitive Awareness Inventory (Jr. MAI; Sperling et al., 2002), a self-report scale that has two subscales that measure students’ knowledge of cognition and regulation of cognition. The Jr. MAI is used to screen learners for potential metacognitive and cognitive strategy interventions. Sperling et al. (2002) developed two versions of the Jr. MAI based on the Metacognitive Awareness Inventory (Schraw & Dennison, 1994). The 12-item version was developed for students in grades 3 through 5, while the 18-item version was developed for older students.

 

Available evidence suggests that the Jr. MAI and its subscales are reliable. Sperling et al. (2002) reported an internal consistency-based reliability estimate of .82 for the overall scale. Sperling, Richmond, Ramsay, and Klapp (2012) reported an internal consistency reliability of .76 for the knowledge of cognition subscale and .80 for the regulation of cognition subscale. Sperling et al. (2002) found that the Jr. MAI total score (for both versions of the instruments) correlated with other direct measures of student metacognition, but not with teachers’ ratings of students’ metacognitive abilities. Relatedly, Sperling et al. (2012) found the student scores on the 18-item version of the Jr. MAI correlated significantly with their scores on the Swanson Metacognitive Questionnaire (SMQ; Swanson, 1990), their science grade point average and their overall grade point average.  Recently, the 12-item version of the Jr. MAI was used to measure the impact of SSS on students’ metacognitive functioning. Lemberger and Clemens (2012) found that SSS delivered in small group format to fourth- and fifth-grade students resulted in measurable increases in Jr. MAI scores.

 

Student Engagement in School Success Skills (SESSS) Survey

The study also employed the Student Engagement in School Success Skills (SESSS) survey. The SESSS (Carey et al., 2013) is a 27-item scale that was developed to measure the extent to which students use strategies that have been shown to be related to enhanced academic achievement (Hattie et al., 1996; Masten & Coatsworth, 1998; Wang et al., 1994). The SESSS has three subscales that measure students’ self-direction of learning, support of classmates’ learning and self-regulation of arousal.

 

Carey et al. (2013) found in an exploratory factor analysis of the SESSS scores of 402 fourth through sixth graders that a four-factor solution provided the best model of scale dimensionality considering both the solution’s clean factor structure and the interpretability of these factors. However, in a confirmatory factor analysis study (Brigman et al., 2014) using SESSS scores from a diverse sample of almost 4,000 fifth-grade students, researchers found that while a four-factor model fit the data well, the scales associated with two subscales correlated so highly (r = .90) as to be indistinguishable. Consequently, the items associated with the two factors were combined and the subsequent three-factor model also proved to better fit the data.

 

Brigman et al. (2014) suggested that the SESSS is best thought of as having three underlying factors corresponding to self-direction of learning, support of classmates’ learning, and self-regulation of arousal. Based on factor loadings, Brigman et al. (2014) created three SESSS subscales. The self-direction of learning subscale (19 items) reflects the students’ intentional use of cognitive and metacognitive strategies to promote their own learning. The support of classmates’ learning subscale (six items) reflects the students’ intentional use of strategies to help classmates learn effectively. Finally, the self-regulation of arousal subscale (three items) reflects students’ intentional use of strategies to control disabling anxiety and cope with stress.

 

Available data indicate that the SESSS is a reliable assessment tool. Carey et al. (2014) reported an overall alpha coefficient of 0.91. Furthermore, Villares et al. (2014) reported that the coefficient alphas for the three SESSS subscales (self-direction of learning, support of classmates’ learning and self-regulation of arousal) were .89, .79 and .68, respectively.

 

Data Analysis

In order to answer the current study’s research questions, the authors conducted separate multivariate analysis of variance (MANOVA) with a repeated measure (pretest-posttest time) for the Jr. MAI and the SESSS. In the Jr. MAI, the two subscales (knowledge of cognition and regulation of cognition) were the dependent variables. For the SESSS MANOVA, the three subscales (self-direction of learning, support of classmates’ learning, self-regulation of arousal) were the dependent variables.

 

After performing the MANOVAs, follow-up repeated measures of analysis of variance (ANOVA) were conducted, where appropriate, to determine the significance of the pretest-posttest changes for individual subscales. Effect sizes (Cohen, 1988) also were calculated to determine the magnitude of pretest-posttest change in subscale associated with the intervention. For significant subscale changes, effect sizes were compared across schools to ascertain whether the level of fidelity of SSS implementation was related to the intervention’s size of effect.

 

Results

 

MANOVA analyses with a repeated measure (pretest-posttest) were performed to determine the differences between Jr. MAI and SESSS subtests across the pretest-posttest time periods in order to answer the primary evaluation question. The primary question was: When implemented in a naturalistic setting, does SSS impact students’ metacognitive functioning, as determined by (1) knowledge and regulation of cognition as measured by the Jr. MAI (Sperling et al., 2002) and (2) use of skills related to self-direction of learning, support of classmates’ learning and self-regulation of arousal as measured by the SESSS (Carey et al., 2013)? The results of these MANOVAs are shown in Table 2. For the Jr. MAI, the repeated measures MANOVA revealed a significant difference (F (1, 1562) = 3267.47, p < .00l) between the two subscales (knowledge of cognition and regulation of cognition). However, no significant difference existed between the pretest and posttest time points. The interaction effect of main effects, subscale and time was not significant.

 

Table 2

 

Repeated Measure MANOVA:  Effects of SSS on Students’ Metacognitive Activity

 

Dependent Measures                          Jr. MAI                        SESSS
Main effects
Subtest

        3267.47***

                       356.24***
Time

.3900

                       19.84***
Interaction effect
Subtest * Time

  1.4400

28.25***

Note. ***p<.001

 

     In contrast, SESSS repeated measures MANOVA revealed both a significant main effect of Subscale (F (2, 1610) = 356.24, p < .00l) and a significant interaction effect of Subscale x Time (F (2, 1610) = 28.25, p < .00l). Figure 1 shows that the self-regulation of arousal subscale (subscale 3) corresponded to a significantly greater mean change across time, compared to the self-direction of learning (subscale 1) and the support of classmates’ learning (subscale 2) subscales.

 

Figure 1. Pre-Post SSS Treatment Changes in Metacognitive Functioning and Success Skill Use: Means for Jr. MAI and SESSS subtests at Pretest (Time 1) and Posttest (Time 2).

 

Jr. MAI                                                                     SESSS

 

 

For Jr. MAI Subtest 1 = Knowledge of Cognition and Subtest 2 = Regulation of Cognition

For SESSS Subtest 1 = Self-Direction of Learning, Subtest 2 = Support of Classmates’ Learning,
and Subtest 3 = Self-Regulation of Arousal

 

Based on these MANOVA results, authors conducted follow-up repeated measures ANOVAs in order to test the significance of pretest-posttest changes for all three SESSS subscales. Only the SESSS self-regulation of arousal subscale indicated a significant change (F (1, 1610) = 46.147, p < .001) over time, reflecting a self-reported increase in students’ abilities to regulate their levels of potentially debilitating arousal after SSS participation. As shown in Table 3, the effect size of the self-regulation of arousal subscale (Cohen’s d = -.18) pretest-posttest change would be classified as small (Cohen, 1988).

 

 

Table 3

 

Effect Sizes of ANOVAs with Repeated Measures (T1 and T2) for Jr. MAI and SESSS

 

Pre

Post

Cohen’s d

Component

M

SD

M

SD

Knowledge of Cognition

3.18

.83

3.16

.85

+.02

Regulation of Cognition

3.96

.60

3.96

.65

+.00

The Self-Direction of Learning

2.32

.65

2.33

.66

-.02

The Support of Classmates’ Learning

2.59

.74

2.63

.74

-.05

The Self-Regulation of Arousal

2.16

.88

2.32

.87

-.18

 

     The secondary research question was: Does the magnitude of any changes in metacognitive functioning depend on the degree to which SSS was implemented with fidelity? Implementation fidelity was not strongly related to SSS effect size. Cohen’s d effect sizes (Cohen, 1988) were computed for each school to assess the impact of SSS on self-regulation of arousal. Schools 2, 3 and 4 (who had reasonable fidelity of implementation) had effect sizes of .20, .17 and .26 respectively. Schools 1 and 5 (who had the greatest deviation from implementation fidelity) reported effect sizes of .30 and .13, respectively. Therefore, the schools in this study showed considerable variability in effect sizes (.13 to .30). School differences across other factors (e.g., experience levels of SSS leaders, grade levels of students) may have contributed to this variability.

 

In summary, although significant findings were not found for pre- and posttests related to the Jr. MAI, significant findings were found for the SESSS subscale self-regulation of arousal (p < .001), indicating that students increased their ability to regulate levels of potentially debilitating arousal after participating in the SSS intervention. Examination of Cohen’s d effect sizes suggested that implementation fidelity, or the amount that schools varied from the scope and sequence laid out in the SSS manuals, did not correlate to level of effect size. This result suggests that the SSS intervention resulted in positive outcomes even when practitioners modified the scope and sequence to fit the needs of their setting.

 

Discussion

 

Evaluation Question 1. Does SSS delivered in a naturalistic setting impact students’ metacognitive functioning? The results of the present study suggest that when implemented in a naturalistic setting, SSS can be expected to result in statistically significant increases in students’ abilities to regulate potentially debilitating emotional arousal. These enhanced abilities might reasonably be expected to result in benefits related to improved academic performance (Durlak et al., 2011) and better school behavior (perhaps helping students increase their self-control related to daily interpersonal conflict and stressful events; Galla & Wood, 2015). The overall effect of SSS on emotional self-regulation, while statistically significant, was comparatively small. The present study failed to find evidence that SSS influenced other aspects of students’ metacognitive functioning, including their knowledge of cognition, regulation of cognition, use of strategies related to the self-direction of learning, or use of strategies to support fellow classmates’ learning.

 

     Evaluation Question 2. Does the magnitude of any changes in metacognitive functioning depend on the degree to which SSS was implemented with fidelity? While the schools in this study showed considerable variability in effect sizes, implementation fidelity was not strongly related to SSS effect size. For example, School 1 scored lowest on implementation fidelity, but demonstrated the greatest effect size (.30). The degree of departure from fidelity was not large enough to detract from SSS’s effect on students’ self-regulation of arousal.

 

Relationship to Previous SSS Findings

The present study failed to replicate the results of previous studies that found significant effects of SSS on students’ metacognition (Lemberger & Clemens, 2012; Lemberger et al., 2015). While the present study as well as Lemberger and Clemens’ study (2012) both used the Jr. MAI to measure changes in students’ metacognition, the two studies differed in terms of SSS delivery format (classroom vs. small group). The failure to replicate Jr. MAI-measured changes after SSS participation may be related to differences in the format (classroom vs. group) for SSS delivery, or to differences in the delivery context (naturalistic vs. controlled).

 

While this study and the Lemberger et al. (2015) study both delivered the SSS classroom format, these studies differed in the instruments they used to measure students’ metacognitive functioning (the SESSS and Jr. MAI vs. the BRIEF). Differences in results may be related to differences in instrumentation or to differences in the delivery context (naturalistic vs. controlled).

 

Limitations

     A limitation of this study is the use of a one-group pre-post design rather than a quasi-experimental design with a control group. Study researchers were constrained by the practical realities of the school environment, such that it was not feasible to implement SSS in such a way as to result in a comparison group. In addition, researchers were unable to locate similar schools that were willing to have students participate in a comparison group. Since it is not always feasible to employ a control group design, a one-group pre-post design may be used to attempt to replicate the findings of stronger research studies and can point to findings that need to be investigated later using stronger evaluation designs (D. T. Campbell & Stanley, 1963; Shadish, Cook & Campbell, 2001).  The present evaluation, in fact, furthers the understanding of the effects of SSS and highlights directions for future research.

 

The failure to collect data in some schools resulted in losses of both the Jr. MAI and the SESSS data. Corresponding complete pretest and posttest surveys were obtained from only 57% and 59% of participating students, respectively. While the lack of strong control over data collection can be thought of as an inherent problem in natural setting evaluations, care should be taken in future studies to strengthen data collection processes, as it is difficult to speculate on the impact of the loss of data in this study. Loss of data in smaller studies could be extremely harmful to the validity of the study. In the present study, there was no reason to believe that the loss of data was related to students’ reactivity to the intervention; however, it is possible that this loss impacted the results, and additional studies are needed to address this issue.

 

Implications for Future Research

It is important to understand the extent and nature of SSS’s impact on students’ metacognitive functioning, whether SSS’s well-established impact on academic performance (Brigman & Campbell, 2003; Brigman, Webb, & Campbell, 2007; C. Campbell & Brigman, 2005; Lemberger et al., 2015; Webb et al., 2005) is mediated by changes in students’ metacognitive functioning or other variables, and whether the impact on academic performance is related to general improvements in all students or an improvement in the functions of specific groups of students. Larger-scale intervention studies are needed to understand relationships between SSS proximal changes in students’ abilities and functioning (e.g., metacognitive functioning, emotional self-regulation, engagement, self-efficacy and motivation) and distal changes in students’ academic performance. It is important to understand which proximal changes in students are mediating their distal improvements in academic performance. It could be that SSS’s effects on academic performance are mediated by one or several of these variables. It also could be that SSS’s effects on academic performance are mediated by relatively broad variables (e.g., self-efficacy) that would be expected to be evident in virtually all students or that are relatively specific (e.g., reductions in debilitating test anxiety) and would be expected to be evident in only some students.

 

Understanding the mediator(s) of SSS’s effects on academic performance would be useful in identifying the most appropriate target groups for this intervention. As such, future studies are needed to explore whether SSS is most appropriate as a Tier 1 intervention for all students or as a Tier 2 intervention for some groups of “at-risk” students. Given that the results of the present study suggested that SSS helped students who struggled with self-regulation of arousal, it is especially important to examine the effectiveness of SSS as a Tier 2 intervention, specifically for students who demonstrate difficulties with emotional self-regulation. Finally, further research is needed to determine the practical significance of SSS on academic performance when it is implemented in less controlled, more naturalistic settings, and to determine how deviations from implementation fidelity and other contextual factors (e.g., expertise of the SSS leaders) correspond to expected social-emotional and academic achievement-related outcomes.

 

Implications for Practice

The results of the present study indicated that SSS, even when implemented in a naturalistic school setting (as opposed to a highly controlled setting), can have a positive impact on students’ abilities to regulate their emotional arousal. The magnitude of the overall impact of SSS on students’ ability to regulate arousal appears to be relatively small. However, readers should note that this effect size was computed based on students in the general population, not students experiencing difficulties with emotional self-regulation. It is likely that SSS would have had a larger estimated effect size if the target group of participants was those who had emotional self-regulation difficulties. However, the SSS curriculum positively impacted student outcomes even when the program was not implemented as designed. Though practitioners are encouraged to follow the manual and schedule as recommended, the results are encouraging in that impacts can still be found even if practitioners modify the design.

 

Conflict of Interest and Funding Disclosure

The authors reported no conflict of interest

for the development of this manuscript.
This project was supported by an Elementary and
Secondary School Counseling Demonstration
Grant project from the Department of Education
no. S215E13422.

 

 

 

References

 

Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Brigman, G., & Campbell, C. (2003). Helping students improve academic achievement and school success behavior. Professional School Counseling, 7(2), 91–98.

Brigman, G., Campbell, C., & Webb, L. (2004). Student Success Skills: Group counseling manual. Boca Raton, FL: Atlantic Education Consultants.

Brigman, G., & Webb, L. (2012). Student success skills: Classroom manual (3rd ed.). Boca Raton, FL: Atlantic Education Consultants.

Brigman, G., Webb, L., & Campbell, C. (2007). Building skills for school success: Improving the academic and social competence of students. Professional School Counseling, 10, 279–288.

Brigman, G., Wells, C., Webb, L., Villares, E., Carey, J. C., & Harrington, K. (2014). Psychometric properties and confirmatory factor analysis of the student engagement in school success skills. Measurement and Evaluation in Counseling and Development. Advance online publication. doi:10.1177/0748175614544545

Campbell, C., & Brigman, G. (2005). Closing the achievement gap: A structured approach to group counseling. Journal for Specialists in Group Work, 30, 67–82.

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Skokie, IL: Rand McNally.

Carey, J., Brigman, G., Webb, L., Villares, E., & Harrington, K. (2013). Development of an instrument to measure student use of academic success skills: An exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 47, 171–180. doi:10.1177/0748175613505622

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

Collaborative for Academic, Social, and Emotional Learning. (2015). CASEL guide: Effective social and emotional learning programs. Middle and high school edition. Chicago, IL: Author. Retrieved from http://secondaryguide.casel.org/casel-secondary-guide.pdf

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. doi:10.1111/j.1467-8624.2010.01564.x

Galla, B. M., & Wood, J. J. (2015). Trait self-control predicts adolescents’ exposure and reactivity to daily stressful events. Journal of Personality, 83, 69–83.

Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2000). Behavior rating inventory of executive function. Odessa, FL: Psychological Assessment Resources.

Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466–474. doi:10.1037/0003-066X.58.6-7.466

Guy, S. C., Isquith, P. K., & Gioia, G. A. (2004). Behavior Rating Inventory of Executive Function-Self-Report Version.  Odessa, FL: Psychological Assessment Resources.

Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66, 99–136. doi:10.3102/00346543066002099

Lemberger, M. E., & Clemens, E. (2012). Connectedness and self-regulation as constructs of the Student Success                         Skills program in inner-city African American elementary school students. Journal of Counseling &                                     Development, 90, 450–458.

Lemberger, M. E., Selig, J. P., Bowers, H., & Rogers, J. E. (2015). Effects of the Student Success Skills program on                         executive functioning skills, feelings of connectedness, and academic achievement in a predominantly                       Hispanic, low-income middle school            district. Journal of Counseling & Development, 93, 25–37.

Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works: Research-based strategies                         for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum                                     Development.

Masten, A. S., & Coatsworth, J. D. (1998). The development of competence in favorable and unfavorable environments: Lessons from research on successful children. American Psychologist, 53, 205–220.

Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460–475.

Shadish, W., Cook, T., & Campbell, D. (2001). Experimental and quasi-experimental designs for generalized causal inference (2nd ed.). Boston, MA: Cengage.

Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy, C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51–79. doi:10.1006/ceps.2001.1091

Sperling, R. A., Richmond, A. S., Ramsay, C. M., & Klapp, M. (2012). The measurement and predictive ability of metacognition in middle school learners. The Journal of Educational Research, 105, 1–7.

Swanson, H. L. (1990). Influence of metacognition and aptitude on problem solving. Journal of Educational Psychology, 82, 306–314.

Villares, E., Colvin, K., Carey, J. C., Webb, L., Brigman, G., & Harrington, K. (2014). Convergent and divergent validity of the Student Engagement in School Success Skills survey. The Professional Counselor, 4, 541–552.

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1994). What helps students learn? Educational Leadership, 51(4), 74–79.

Webb, L., & Brigman, G. (2006). Student success skills: Tools and strategies for improved academic and social outcomes. Professional School Counseling, 10, 112–120.

Webb, L., Brigman, G., & Campbell, C. (2005). Linking school counselors and student success: A replication of the Student Success Skills approach targeting the academic and social competence of students. Professional School Counseling, 8, 407–413.

Zins, J. E., Weissberg, R. P., Wang, M. C., & Walberg, H. J. (2004). Building academic success on social and emotional learning: What does the research say? New York, NY: Teachers College Press.

 

 

 

Brett Zyromski is an Assistant Professor at The Ohio State University. Melissa Mariani is an Assistant Professor at Florida Atlantic University. Boyoung Kim is a Research Professor at Korea University. Sangmin Lee is an Associate Professor at Korea University. John Carey is a Professor at the University of Massachusetts Amherst. This project was supported by an Elementary and Secondary School Counseling Demonstration Grant project from the Department of Education, no. S215E13422. Correspondence can be addressed to Brett Zyromski, Department of Educational Studies, Counselor Education, PAES Building, 305 W. 17th Ave., Columbus, OH, zyromski.1@osu.edu. 

Effect of Participation in Student Success Skills on Prosocial and Bullying Behavior

Melissa Mariani, Linda Webb, Elizabeth Villares, Greg Brigman

This study involved fifth-grade students (N = 336) from one Florida school district and examined prosocial behaviors, bullying behaviors, engagement in school success skills and perceptions of classroom climate between the treatment group who received the school counselor-led Student Success Skills classroom guidance program, and their peer counterparts (comparison group). Statistically significant differences were found (p values ranged from .000–.019), along with partial eta-squared effect sizes ranging from .01 (small) to .26 (quite large) between groups. Evidence supported the Student Success Skills classroom program as a positive intervention for affecting student engagement, perceptions and behavior. 

 

Keywords: bullying, prosocial behaviors, Student Success Skills, classroom climate, school counselor

 

While some forms of youth victimization have steadily declined over the years, bullying occurrences have remained relatively stable (DeVoe et al., 2004; Wang, Iannotti, & Nansel, 2009). Reports have indicated that 30–40% of students admit to regular involvement in bullying behaviors (Bradshaw, O’Brennan, & Sawyer, 2008; Nansel et al., 2001; Spriggs, Iannotti, Nansel, & Haynie, 2007). Additionally, statistics reveal that bullying is much more common among early adolescents than elementary age children (Bradshaw et al., 2008; Olweus, 1993; Ortega & Lera, 2000). In fact, notable increases in the rates of peer aggression occur during the transition years, in both grade 6 (beginning of middle school) and grade 9 (beginning of high school; Olweus, 1993; Ortega & Lera, 2000); therefore, targeting students prior to these peaks would be considered more proactive.

 

Recent approaches to combat the bullying problem have highlighted the importance of increasing students’ social competencies and coping and social interaction skills (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). Greenberg et al. (2003) offered that alternative approaches to managing problem behavior are most beneficial when they simultaneously foster students’ personal and social skills while improving the quality of the school environment. The philosophy behind incorporating these types of programs in schools suggests that in order for students to fully reach their potential, educators must address the whole child (Payton et al., 2008; Saleebey, 2008). Ultimately, building key skills in all children contributes to creating a positive, safe and caring learning environment, one that discourages aggression and violence.

 

The Consequences of Bullying Behaviors

 

Bullying can negatively impact victims and bullies, as well as bystanders. Emotionally, victims of bullying report higher levels of fear and anxiety (Gini & Pozzoli, 2009; Reijntjes, Kamphuis, Prinzie, & Telch, 2010), are more socially withdrawn (Roth, Coles, & Heimberg, 2002), and are more likely to experience depression (Ttofi, Farrington, Lösel, & Loeber, 2011) than their peers. In terms of social consequences, victims suffer from increased levels of peer rejection (Gini & Pozzoli, 2009; Reijntjes et al., 2010). Victimization also has been linked to academic consequences, including increased tardiness, absentee and dropout rates (Beale & Scott, 2001; Nansel et al., 2001); poorer grades; and more academic struggles than their peer counterparts (Boulton, Trueman, & Murray, 2008). Similarly, bullies and bystanders experience distinct consequences that contribute to the struggles they experience in school. For example, bullies also may earn poorer grades and have higher absentee and dropout rates than non-aggressive peers (Bernstein & Watson, 1997), and bystanders have reported increased levels of fear about school safety (Olweus, 1993).

 

The literature further indicates that the actions of those involved in bullying situations, including bystanders, can either enhance or damage a school’s climate (Catalano, Haggerty, Oesterle, Fleming, & Hawkins, 2004; Swearer, Espelage, Vaillancourt, & Hymel, 2010). Carney (2008) concluded that experiencing bullying firsthand, as well as witnessing bullying incidents, can be traumatic for students. It is evident that schools should be concerned about proactively addressing bullying behaviors. If not, significant consequences related to student behavior and academic achievement can abound.

 

Empirical Support for Student Success Skills

 

The Student Success Skills (SSS) classroom program (Brigman & Webb, 2010) is based on extensive research reviews (Daly, Duhon, & Witt, 2002; Greenberg et al., 2003; Hattie, Biggs, & Purdie, 1996; Masten & Coatsworth, 1998; Payton et al., 2008; Wang, Haertel, & Walberg, 1994; Zins, Weissberg, Wang, & Walberg, 2004) that identified three key categories of skills needed in order to grow, perform and achieve: (a) cognitive and meta-cognitive skills such as goal setting, progress monitoring and memory skills; (b) social skills such as interpersonal, social problem solving, listening and teamwork skills; and (c) self-management skills such as managing attention, motivation and anger. Recent evidence supporting the use of these skills, valuing the teaching of both academic and social skills in order to promote student growth and success, also can be found in the literature (Winne & Nesbit, 2010; Yeager &Walton, 2011).

 

SSS is a comprehensive, evidence-based, school counselor-led program that supports development of these key skills in students. This manualized intervention consists of five 45-minute classroom lessons spaced one week apart, beginning in the fall, usually in late August or early September. Three monthly booster sessions are then implemented beginning in January. A total of 20 strategies are introduced and reinforced using a highly engaging “tell-show-do” format known to increase levels of student engagement and motivation. Each SSS lesson follows a structured beginning, middle and end sequence clearly detailed in the SSS manual. (Due to space limitations, readers are encouraged to review the Webb and Brigman [2006] descriptive article on the SSS classroom program).

 

Five outcome studies testing the effectiveness of SSS classroom and small group programs have resulted in positive effects on both student achievement and behavior, as well as perceived improvement in classroom behaviors (Brigman & Campbell, 2003; Brigman, Webb, & Campbell, 2007; Campbell & Brigman, 2005; León, Villares, Brigman, Webb, & Peluso, 2011; Webb, Brigman, & Campbell, 2005). A recent meta-analysis of these five SSS studies revealed an overall effect size of .29 (large), a medium effect size of .17 (equivalent to an additional half of a year of learning in reading) and a large effect size of .41 (equivalent to an additional full year of learning in math; Villares, Frain, Brigman, Webb, & Peluso, 2012).

 

While the SSS program has been shown to positively affect student achievement and behavior in general, comparison studies have not examined the impact of SSS on reducing bullying behavior. Consequently, the current study sought to measure the effects of the SSS classroom program administered by school counselors (Brigman & Webb, 2010) on student prosocial behavior, bullying behavior, engagement in school success skills and perceptions of classroom climate. The SSS intervention was selected because the cognitive, social and self-management skills taught in the program are associated with promoting academic and prosocial behaviors in youth, behaviors that enhance a positive school climate and discourage negative behaviors like bullying.

 

Purpose of the Study

 

The purpose of this study was to determine the effectiveness of the SSS classroom program (Brigman & Webb, 2010) on fifth-grade students’ prosocial behavior, bullying behavior, engagement in school success skills and perceptions of classroom climate. The problem addressed is significant for two reasons. First, a wide range of negative consequences can result from ineffectively dealing with bullying (Bernstein & Watson, 1997; Carney, 2008; Catalano et al., 2004; Deluty, 1985; Gini & Pozzoli, 2009; Olweus, 1993; Reijntjes et al., 2010; Swearer et al., 2010). Second, further research is needed to demonstrate the positive impact that school counselors have in schools. It has been stated that the school counselor’s role in addressing bullying in schools is crucial (Crothers & Levinson, 2004; Hanish & Guerra, 2000; Hazler & Carney, 2000; Hermann & Finn, 2002).

 

Research Questions

The following research questions were addressed: (a) Is there an increase in the prosocial behaviors of fifth-grade students after participating in the SSS classroom program? (b) Is there a decrease in the bullying behaviors of fifth-grade students after participating in the SSS classroom program? (c) Is there an increase in levels of engagement in student success skills (cognitive and learning, social, and self-management) of fifth-grade students after participating in the SSS classroom program? (d) Is there an improvement in classroom climate after fifth-grade students participate in the SSS classroom program?

 

Method

 

Participants and Sampling Procedures

Fifth-grade students (N = 336, 181 females and 155 males) from five public elementary schools in central Florida volunteered to participate in this study. The eligibility criteria included the following: (a) participating schools had to employ a certified school counselor, (b) school counselors had to agree to implement the manualized SSS classroom program (Brigman & Webb, 2010), and (c) in an attempt to create a whole-school culture, the school had to have more than one fifth-grade classroom participating. On average, each school contained 4–6 general education fifth-grade classrooms; 21 of these 22 classrooms in the five participating elementary schools were included in the study. All students in general education fifth-grade classrooms were invited to participate. Blended classrooms (e.g., multiple grade levels in one classroom) were not included so that generalizations among age levels could be made between schools. The volunteer sample (N = 336) mean age was 10 years old. Racial identifications included 7 (2%) Asian, 52 (15%) African American, 221 (66%) Caucasian, 43 (13%) Latino/a, 12 (3.6%) Multiracial and 1 (.4%) American Indian. Thirty-one percent of the sample (n = 104) received free lunch and 7.1% (n = 24) were on reduced-lunch status.

 

The study followed a pre-post quasi-experimental cohort group design (Cook & Campbell, 1979). Random assignment of individual students was not conducive to preserving the nature of a whole-school culture, so schools were assigned to either the treatment or comparison group based on the order in which they volunteered to participate. The first three schools to volunteer were assigned to the treatment group (schools A, B and C) while the last two schools (schools D and E) were assigned to the comparison group.

Procedures

Following approval from the university’s Institutional Review Board, consent for research was obtained from the participating school district, school administrators, parents, teachers and students. In September, five certified school counselors from the participating schools received a 1-day training in the manualized use of the SSS classroom guidance program as well as other study-related procedures including instrument administration and electronic summary report instructions. The SSS program, consisting of five consecutive 45-minute lessons spaced a week apart, was then implemented in all fifth-grade classrooms in the treatment schools beginning in October. Monthly booster lessons followed beginning in January. Only students with parent permission completed the required instruments: the Peer Relations Questionnaire (PRQ), the Student Engagement in School Success Skills (SESSS) survey and the My Class Inventory-Short Form Revised (MCI-SFR). Students were ensured of the anonymity of their reporting by using generic school, classroom and student numbers. For a classroom to remain eligible to participate, a minimum of 80% of the students in the classroom had to return a signed parent consent form.

 

     Treatment group. Schools A, B and C served as the treatment group (n = 209) and participating fifth-grade students in this group received the SSS classroom intervention. These students completed the following pretests in September 2010: the PRQ, MCI-SFR and SESSS. Implementation of the SSS classroom program began in October. Following the completion of the first five SSS lessons, treatment students completed the SESSS instrument (posttest). Booster lessons were delivered in January, February and March, and treatment students were then asked to complete the PRQ, MCI-SFR and SESSS following the final booster lesson (post-posttest).

 

     Comparison group. Schools D and E served as the comparison group (n = 127) and did not receive the SSS intervention during the study. Students in these schools experienced business as usual, including any regularly scheduled school counseling programming. Comparison schools were eligible to receive the SSS curriculum after the study was completed. Participating students in the comparison schools completed the three instruments at the same time intervals (pretest, posttest and post-posttest) as students in the treatment group.

 

Instruments

     Peer Relations Questionnaire – For Children – Short Form. The PRQ (Rigby & Slee, 1993a) was designed to reveal student experiences with bullying at school. The questionnaire takes approximately 5–7 minutes to complete and is comprised of 20 items in which students are asked to circle how often the statements are true for them. The answers range on a 4-point scale from never = 1, once in a while = 2, pretty often = 3, to very often = 4. The PRQ consists of three scales and several filler items: a Bully Scale, a Victim Scale and a Prosocial Scale; students in the present study took all three scales. Scoring is determined by the items contained in each of the scales, with higher scores corresponding to a propensity for bully, victim and/or prosocial behaviors (Rigby & Slee, 1993b). Rigby and Slee (1993b) reported the reliability of the PRQ using the following alpha coefficients: bully scale (.75–.78), victim scale (.78–.86) and prosocial scale (.71–.74), indicating more than adequate internal consistency. Recent evaluation of the PRQ’s psychometric properties by Tabaeian, Amiri, and Molavi (2012) supported it as a highly reliable and valid instrument that should continue to be used in research.

 

     Student Engagement in School Success Skills Survey. The SESSS is a 33-item student self-report of cognitive engagement in SSS program skills and strategies, using language specific to the SSS curriculum, and takes approximately 15 minutes to complete (Carey, Brigman, Webb, Villares, & Harrington, 2013). Students are asked to circle how often they have engaged in a list of behaviors within the last 2 weeks (e.g., “I tried to encourage a classmate who was having a hard time doing something,” “I noticed when another student was having a bad day,” “I listened to music so that I would feel less stressed”). Possible responses include I didn’t do this at all, I did this once, I did this two times or I did this three or more times. The SESSS is intended for use with students in grades 3–12. Though a four-factor model was first revealed in an exploratory factor analysis conducted by Carey et al. (2013), a subsequent confirmatory factor analysis revealed the following three factors: self-direction of learning (which represents the combination of two original factors—management of learning and application of learning strategies), support of classmates’ learning and self-regulation of arousal, which correspond to the three subscales of the SESSS (Brigman et al., 2014). Coefficient alphas for the three SESSS subscales were as follows: self-direction of learning: 0.89, support of classmates’ learning: 0.79 self-regulation of arousal: 0.68, and 0.90 for the SESSS as a whole (Villares et al., 2014), indicating good internal consistency.

 

     My Class Inventory-Short Form-Revised. The MCI-SFR is a 20-item instrument that intends to measure the perceptions of students in grades 4–6 of four areas related to classroom climate (satisfaction, friction, competitiveness and cohesiveness). The instrument takes approximately 10–15 minutes to complete and respondents are asked to select either “yes” (3 points) or “no” (1 point). Omitted or invalidly scored items receive two points. Reports on the psychometric properties for both the MCI-SF and MCI-SFR have indicated strong concurrent validity when comparing long and short versions across each of the scales (.91–.97). Additionally, some degree of internal consistency (largely adequate coefficient alphas) has been reported for class means with Australian children (.58–.81). The MCI-SF yielded more acceptable alpha coefficients for each of the scales (.84–.93) than did the long version, the MCI. Modifications to the revised MCI-SFR produced a better overall instrument, improving factor interpretability and reliability (Fraser, 1982; Sink & Spencer, 2005). Sink and Spencer (2005) reported that interpreting students’ responses from pretest to posttest on the MCI-SFR should be straightforward, with higher scores on the satisfaction and cohesion scales providing positive indicators of a healthy classroom environment, and higher scores on the competitiveness and friction scales suggesting needed improvement in this area.

 

Data Analysis

Individual students were the units of analysis in the study. An alpha level of .05 and one-way analysis of variance (ANOVA) tests were used to analyze differences in prosocial behaviors, bullying behaviors, school engagement skills and perceptions of classroom climate between students who participated in the SSS program (treatment group) and students who did not (comparison group). A post hoc Bonferroni correction was used to lessen the chance of a Type I error. Prior to the analyses, all the variables of interest were examined for accuracy of data entry, missing values, outliers and the normality of distributions. In addition, effect sizes (ES) were calculated to determine the practical significance of the SSS classroom program for the various student outcomes.

 

In this study, a partial eta-squared (ES; hp2) calculation was computed by SPSS (Field, 2009; Howell, 2008; Sink & Mvududu, 2010). The ES addresses the magnitude of the difference between groups or relationships between variables. The following benchmarks were used to determine small, medium, and large or strong ES strengths regarding hp2 calculations: (a) .01 small, (b) .06 medium, and (c) .14 large or strong (Green & Salkind, 2008; Sink & Mvududu, 2010).

 

Results

 

Preliminary ANOVAs were conducted on the students’ PRQ, SESSS and MCI-SFR pretest scores to determine whether statistically significant differences existed among the treatment and comparison groups prior to the implementation of the SSS intervention. No statistically significant differences were found on pretest scores; therefore, no covariates were used in subsequent analyses of students’ PRQ, SESSS and MCI-SFR posttest scores. Table 1 provides a summary of the study’s main findings.

 

Prosocial Behaviors

Research question 1 examined whether fifth-grade students who participated in the SSS classroom program would experience an increase in prosocial behaviors as compared to their peer counterparts who did not receive the intervention. Prosocial behaviors were assessed using the prosocial scale of the PRQ. A total of 188 students from the treatment group (schools A, B and C) and 123 students from the comparison group (schools D and E) were included in this analysis (n = 311). Findings from an ANOVA showed a statistically significant difference between groups, F(1, 308) = 18.708, p = .000 and hp= .06, a medium effect size. Participants in the treatment group (n = 188, M = 12.61, SD = 2.47) reported higher scores for prosocial behaviors at posttest as opposed to participants in the comparison group (n = 123, M = 11.27, SD = 2.81). Results indicated that students in the treatment schools reported engaging in prosocial behaviors more often at posttest than students in the comparison schools, highlighting the practical significance of using this intervention to positively influence student behavior.

 

Table 1

 

Summary Table of P Values, Effect Size Estimates, and Confidence Intervals for All Measures

Measure p value  hp2 ES Strength              CI
PRQ
     Prosocial .000* .06 Medium 95% [11.68, 12.22]
     Bully .017* .02 Small 95% [7.22, 7.69]
SESSS
     Pretest to Posttest .000* .26 Large 95% [2.05, 2.20]
     Pretest to Post-posttest .366 .00 Negligible 95% [2.46, 2.62]
MCI-SFR
     Satisfaction .019* .02 Small 95% [10.36, 10.96]
     Friction .152 .01 Small 95% [9.21, 9.83]
     Competitiveness .831 .00 Negligible 95% [10.79, 11.41]
     Cohesion .414 .00 Negligible 95% [9.18, 9.85]

Note. PRQ = Peer Relations Questionnaire; SESSS = Student Engagement in School Success Skills;
MCI-SFR = My Class Inventory-Short Form-Revised; p = significance at posttest; hp2 = partial eta-squared
effect size; CI = confidence interval;

* p < .05.

 

Bullying Behaviors

The second research question asked whether fifth-grade students who received SSS would experience a decrease in bullying behaviors, assessed by the bully scale of the PRQ, compared to their peers in the comparison group. Results from a one-way ANOVA showed a statistically significant difference between the participants’ (n = 311) posttest scores, F(1, 308) = 5.708, p = .017 and a small effect size, hp2 = .02. These findings confirmed that students in the treatment group evidenced a decrease in mean change scores on the PRQ bully scale after SSS implementation, whereas students in the comparison schools reported an increase. Thus, students in the treatment group who received the SSS classroom intervention reported less bullying behavior at posttest than students in the comparison group.

 

Engagement in School Success Skills

Research question 3 investigated whether participating fifth-grade students who received the SSS classroom program would experience an increase in levels of engagement in student success skills (cognitive and learning, social, self-management) as compared to their peer counterparts. Results from the SESSS instrument were used in this analysis. A total of 115 students in the treatment group (schools A, B and C) and 85 students in the comparison group (schools D and E) were included in the SESSS analysis (n = 200). Table 2 displays the treatment and comparison group means, standard deviations, and change scores for the SESSS by school at the following three data collection periods: pretest (prior to SSS implementation), posttest (immediately following implementation of the five weekly SSS lessons) and post-posttest (at the end of the study).

 

Table 2

 

Treatment and Comparison Group Means, Standard Deviations and Change Scores for the SESSS by School

School n PretestM (SD) PosttestM (SD) Post-posttestM (SD) Pretest-to-posttestM  +/- Posttest-to-post-posttest M  +/- Pretest-to-post-posttest M  +/-
A* 40 2.49 (.61) 2.88 (.63) 2.41 (2.63) +.39 +.47 -.08
B* 38 2.47 (.68) 2.62 (.66) 2.64 (.63) +.15 +.02 +.17
C* 37 2.44 (.58) 2.60 (.60) 2.82 (.64) +.16 +.22 +.38
D 28 2.53 (.53) 2.47 (.57) 2.56 (.65) -.06 +.09 +.03
E 57 2.07 (.77) 1.37 (.12) 2.39 (.48) -.70 +1.02 +.32
TotalT 115 2.47 (.62) 2.50 (.64) 2.62 (.65) +.03 +.12 +.15
Total

C

85 2.22 (.73) 1.73 (.68) 2.45 (.54) -.49 +.72 +.23

 

Note. SESSS = Student Engagement in School Success Skills; n = number; M = mean; SD = standard deviation;

T = treatment group; C = comparison group; * = treatment school; +/- = mean change score.

 

   SESSS posttest score analysis. Findings from an ANOVA on the posttest scores on the SESSS (from the pretest in October to the posttest in December) showed a statistically significant difference between schools, F(1, 197) = 69.295, p = .000 and hp2 = .26, a large effect size. Students in the treatment group (n = 115, M = 2.50, SD = .642) evidenced higher levels of engagement in school success skills from pretest to posttest than their counterparts in the comparison group (n = 85, M = 1.73, SD = .617).

 

SESSS post-posttest score analysis. A second one-way ANOVA showed no statistically significant differences between the treatment and comparison groups scores from pretest (October) to post-posttest (March), F(1, 197) = .820, p = .366 and hp2 = .004, a small effect size.

 

Perceptions of Classroom Climate

Finally, research question 4 investigated whether fifth-grade treatment group students would perceive an improvement in classroom climate as compared to students in the comparison group. Due to attrition, 308 fifth-grade students completed the four scales (satisfaction, cohesion, competitiveness and friction) of the MCI-SFR. Findings from an ANOVA using the MCI-SFR satisfaction scale posttest scores revealed a statistically significant difference between the treatment and comparison groups, F(1, 305) = 5.523, p = .019 and hp2 = .02, a small effect size. In particular, students in the treatment group (n = 187, M = 10.96, SD = 2.86) reported higher scores on the satisfaction scale at posttest than did students in the comparison group (n = 121, M = 10.39, SD = 2.74). The ANOVA tests on the other three scales of the MCI-SFR did not result in statistically significant differences between the treatment and comparison groups.

 

Discussion

 

The findings of this study reflect the connection between prosocial skills and reduced aggression, a finding which has been well documented in previous literature (Endresen & Olweus, 2001; Feshbach, 1997; McMahon & Washburn, 2003). School counselor interventions that focus on teaching prosocial behaviors have been successful in reducing aggressive behaviors such as bullying (Frey, Hirschstein, & Guzzo, 2000); these types of interventions also have been tied to improved academic achievement (Wentzel, 2003; Wentzel & Caldwell, 1997). The American School Counselor Association (ASCA; 2012) recommends that counselors cover academic, personal and social, and career domains as part of a comprehensive school counseling program. Results of this study support the delivery of interventions that incorporate the teaching of cognitive, social and self-management skills as a means to increase prosocial skills, reduce bullying behavior and promote a positive classroom climate. The design of the current study attempted to create a whole-school approach by implementing the SSS classroom program across an entire grade level (grade 5) in the treatment schools. Given that bullying peaks in the transition years, addressing the fifth-grade population was viewed as a proactive approach. SSS implementation resulted in some positive outcomes for those students, indicating that even a modified whole-school approach can be beneficial.

 

Previous SSS studies have documented the intervention’s positive impact on student academic performance as measured by standardized test scores in math and reading (Villares et al., 2012). Professionals in the field of counseling have identified a need to evaluate the link between the SSS program and intermediate variables related to student learning such as engagement in school success skills, prosocial behavior and perceptions of classroom climate (Carey, Dimmitt, Hatch, Lapan, & Whiston, 2008). Findings from the current study indicate that students who received the SSS intervention engaged significantly more in behaviors indicative of school success at posttest. These results are encouraging, since a body of research cites the negative impact that bullying can have on student academic achievement (Beale & Scott, 2001; Boulton et al., 2008; Nansel et al., 2001; Olweus, 1993).

 

The quality of a classroom climate also can impact students’ success. Although improved perceptions of classroom climate were predicted across all areas in the current study, statistically significant differences were only noted on perceptions related to satisfaction. The researchers postulate that treatment students were more likely to tune into questions pertaining to satisfaction, as this is a focus of the SSS program (noticing small improvements, focusing on the positives, and creating a safe, caring, supportive, encouraging classroom). The maintenance of a positive school and classroom climate directly affects whether or not students feel accepted and happy among their peers (Greenberg et al., 2003; Millings, Buck, Montgomery, Spears, & Stallard, 2012; Shochet, Dadds, Ham, & Montague, 2006). The literature indicates that the effectiveness of school counseling interventions can be greatly impacted by the school’s climate (Greenberg et al., 2003). Specifically, factors such as teacher adherence to the curriculum and staff buy-in can affect a program’s success (Biggs, Vernberg, Twemlow, Fonagy, & Dill, 2008; Yoon, 2004). Teachers should be involved in program implementation so that they become invested in its success. The current study addressed this area in that the classroom teachers were collaborators in SSS implementation. The program asks that classroom teachers be present during the counselor-led sessions so that they can cue students to use the skills taught throughout the regular school day. Thus, evidence-based interventions like the SSS program that emphasize school connectedness can be of benefit to students (Millings et al., 2012).

 

Implications for Practice and Future Research

 

The findings of this study support the use of the school counselor-led SSS classroom program as a practical means of impacting students’ prosocial skills, bullying behavior, engagement in school success skills and some perceptions of classroom climate, as indicated by various student self-report measures. Since the bullying literature calls for the use of multiple measures when attempting to link interventions to improvements, we recommend that additional studies track attendance rates, disciplinary referrals, bullying incident reports, and peer and teacher nominations, in addition to student instruments. Future researchers in this area also should gather data from teacher participants and vary the type of measurements specifically tied to prosocial and bullying behaviors (Pellegrini & Bartini, 2000; Van Schoiack-Edstrom, Frey, & Beland, 2002), as well as academic outcomes (Carey et al., 2008; Hall, 2006). This study sought to create a whole-school culture by incorporating the intervention across an entire grade level at each school. Future researchers might consider implementing SSS across several grade levels or throughout the entire school, as students across various grades often come in contact with one another throughout the school day.

 

Limitations

The participants were derived from one suburban school district and randomization procedures were not possible, thereby limiting the sample size and generalizability of the results. Likewise, due to one school dropping out of the study at the onset, the numbers between the treatment and comparison groups were not equivalent. The high level of attrition also was a limitation, specifically regarding the SESSS instrument. Though 336 students were in the original sample, only 200 of these were included in the analysis on the SESSS due to dropping out or not adequately completing the instrument in its totality at all three intervals.

 

The self-report nature of all three of the instruments was an added limitation, particularly with the problem of bullying. Students involved in bullying incidents, whether they were bullies, victims or bystanders, might be hesitant to report or indicate negative behaviors. This reluctance could have resulted in respondent bias and decreased reliability in the results.

 

Finally, the current study used only one component of the SSS curriculum (classroom program). Future studies might involve additional modalities, including individual and small group counseling as well as parent involvement. This study did not examine the impact of the SSS program over time. Follow-up studies are needed to support the long-term effectiveness of school counselor-led interventions that increase prosocial behaviors, reduce bullying behaviors and promote a positive school climate.

 

Conclusion

 

Results of the study provide support that students who receive the SSS classroom intervention led by school counselors (Brigman & Webb, 2010) evidence statistically significant differences in prosocial behaviors, bullying behaviors, engagement in school success skills and perceptions related to satisfaction with their classroom climate, as compared to students who do not receive the program. The findings provide empirical support for the notion that when students are taught skills in key areas (personal and social, self-management, and cognitive and academic) they benefit across social, emotional and behavioral outcomes. The study also suggests that aggressive behaviors such as bullying can be influenced by programs that do not specifically target these behaviors. Finally, this research points to the positive impact school counselors can have on student success, particularly when they deliver interventions that promote social competence among students. Providing school counselors with an evidence-based program that impacts students across several domains is of great value for school counseling practice.

Conflict of Interest and Funding Disclosure

The authors reported no conflict of interest

or funding contributions for the development

of this manuscript.

 

References

 

American School Counselor Association. (2012). The ASCA national model: A framework for school counseling programs (3rd ed.). Alexandria, VA: Author.

Beale, A. V., & Scott, P. C. (2001). “Bullybusters”: Using drama to empower students to take a stand against bullying behavior. Professional School Counseling, 4, 300–305.

Bernstein, J. Y., & Watson, M. W. (1997). Children who are targets of bullying: A victim pattern. Journal of Interpersonal Violence, 12, 483–498. doi:10.1177/088626097012004001

Biggs, B. K., Vernberg, E. M., Twemlow, S. W., Fonagy, P., & Dill, E. J. (2008). Teacher adherence and its relation to teacher attitudes and student outcomes in an elementary school-based violence prevention program. School Psychology Review, 37, 533–549.

Boulton, M. J., Trueman, M., & Murray, L. (2008). Associations between peer victimization, fear of future victimization and disrupted concentration on class work among junior school pupils. British Journal of Educational Psychology, 78, 473–489. doi:10.1348/000709908X320471

Bradshaw, C. P., O’Brennan, L. M., & Sawyer, A. L. (2008). Examining variation in attitudes toward aggressive retaliation and perceptions of safety among bullies, victims, and bully/victims. Professional School Counseling, 12, 10–21. doi:10.5330/PSC.n.2010-12.10

Brigman, G., & Campbell, C. (2003). Helping students improve academic achievement and school success behavior. Professional School Counseling, 7, 91–98.

Brigman, G., & Webb, L. (2010). Student success skills: Classroom manual (3rd ed.). Boca Raton, FL: Atlantic Education Consultants.

Brigman, G. A., Webb, L. D., & Campbell, C. (2007). Building skills for school success: Improving the academic and social competence of students. Professional School Counseling, 10, 279–288.

Brigman, G., Wells, C., Webb, L., Villares, E., Carey, J. C., & Harrington, K. (2014). Psychometric properties and confirmatory factor analysis of the student engagement in school success skills survey. Measurement and Evaluation in Counseling and Development. doi:10.1177/0748175614544545

Campbell, C. A., & Brigman, G. (2005). Closing the achievement gap: A structured approach to group counseling. Journal for Specialists in Group Work, 30, 67–82. doi:10.1080/01933920590908705

Carey, J., Brigman, G., Webb, L., Villares, E., & Harrington, K. (2013). Development of an instrument to measure student use of academic success skills: An exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 47, 171–180. doi:10.1177/0748175613505622

Carey, J. C., Dimmitt, C., Hatch, T. A., Lapan, R. T., & Whiston, S. C. (2008). Report of the national panel for evidence-based school counseling: Outcome research coding protocol and evaluation of student success skills and second step. Professional School Counseling, 11, 197–206.

Carney, J. V. (2008). Perceptions of bullying and associated trauma during adolescence. Professional School Counseling, 11, 179–188.

Catalano, R. F., Haggerty, K. P., Oesterle, S., Fleming, C. B., & Hawkins, J. D. (2004). The importance of bonding to school for healthy development: Findings from the social development research group. Journal of School Health, 74, 252–261. doi:10.1111/j.1746-1561.2004.tb08281.x

Cook, T. D., & Campbell, D. T. (1979). Quasi-Experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.

Crothers, L. M., & Levinson, E. M. (2004). Assessment of bullying: A review of methods and instruments. Journal of Counseling & Development, 82, 496–503. doi:10.1002/j.1556-6678.2004.tb00338.x

Daly, E. J., III, Duhon, G. J., & Witt, J. C. (2002). Proactive approaches for identifying and treating children at risk for academic failure. In K. L. Lane, F. M. Gresham, & T. E. O’Shaughnessy (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 18–32). Boston, MA: Allyn & Bacon.

Deluty, R. H. (1985). Cognitive mediation of aggressive, assertive, and submissive behavior in children. International Journal of Behavioral Development, 8, 355–369. doi:10.1177/016502548500800309

DeVoe, J. F., Peter, K., Kaufman, P., Miller, A., Noonan, M., Snyder, T. D., & Baum, K. (2004). Indicators of school crime and safety: 2004. Retrieved from http://nces.ed.gov/pubs2005/2005002.pdf

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–32. doi:10.1111/j.1467-8624.2010.01564.x

Endresen, I. M., & Olweus, D. (2001). Self-reported empathy in Norwegian adolescents: Sex differences, age trends, and relationship to bullying. In A. C. Bohart & D. J. Stipek (Eds.), Constructive & destructive behavior: Implications for family, school, and society (pp.147–165). Washington, DC: American Psychological Association.

Feshbach, N. D. (1997). Empathy: The formative years—implications for clinical practice. In A. C. Bohart & L. S. Greenberg (Eds.), Empathy reconsidered: New directions in psychotherapy (pp. 33–59). Washington, D.C.: American Psychological Association.

Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage.

Fraser, B. J. (1982). Development of short forms of several classroom environment scales. Journal of Educational Measurement, 19, 221–227.

Frey, K. S., Hirschstein, M. K., & Guzzo, B. A. (2000). Second step: Preventing aggression by promoting social competence. Journal of Emotional and Behavioral Disorders, 8, 102–113. doi:10.1177/106342660000800206

Gini, G., & Pozzoli, T. (2009). Association between bullying and psychosomatic problems: A meta-analysis. Pediatrics, 123, 1059–1065. doi:10.1542/peds.2008-1215

Green, S. B., & Salkind, S. J. (2008). Using SPSS for Windows and Macintosh: Analyzing and understanding data (5th ed.). Upper Saddle River, NJ: Prentice-Hall.

Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466–474. doi:10.1037/0003-066X.58.6-7.466

Hall, K. R. (2006). Using problem-based learning with victims of bullying behavior. Professional School Counseling, 9, 231–237.

Hanish, L. D., & Guerra, N. G. (2000). Children who get victimized at school: What is known? What can be done? Professional School Counseling, 4, 113–119.

Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66, 99–136. doi:10.3102/00346543066002099

Hazler, R. J., & Carney, J. V. (2000). When victims turn aggressors: Factors in the development of deadly school violence. Professional School Counseling, 4, 105–112.

Hermann, M. A., & Finn, A. (2002). An ethical and legal perspective on the role of school counselors in preventing violence in schools. Professional School Counseling, 6, 46–54.

Howell, D. (2008). Best practices in the analysis of variance. In J. Osborne (Ed.), Best practices in quantitative methods (pp. 341–357). Thousand Oaks, CA: Sage.

León, A., Villares, E., Brigman, G., Webb, L., & Peluso, P. (2011). Closing the achievement gap of Latina/Latino students: A school counseling response. Counseling Outcome Research and Evaluation, 2, 73–86. doi:10.1177/2150137811400731

Masten, A. S., & Coatsworth, J. D. (1998). The development of competence in favorable and unfavorable environments: Lessons from research on successful children. American Psychologist, 53, 205–220.

McMahon, S. D., & Washburn, J. J. (2003). Violence prevention: An evaluation of program effects with urban African American students. The Journal of Primary Prevention, 24, 43–62.

Millings, A., Buck, R., Montgomery, A., Spears, M., & Stallard, P. (2012). School connectedness, peer attachment, and self-esteem as predictors of adolescent depression. Journal of Adolescence, 35, 1061–1067. doi:10.1016/j.adolescence.2012.02.015

Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001). Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. Journal of the American Medical Association, 285, 2094–2100. doi:10.1001/jama.285.16.2094

Olweus, D. (1993). Bullying at school. Malden, MA: Blackwell.

Ortega, R., & Lera, M.-J. (2000). The Seville anti-bullying in school project. Aggressive Behavior, 26, 113–123. doi:10.1002/(SICI)1098-2337(2000)26:1<113::AID-AB9>3.0.CO;2-E

Payton, J., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Schellinger, K. B., & Pachan, M. (2008). The positive impact of social and emotional learning for kindergarten to eighth-grade students: Findings from three scientific reviewsExecutive summary. Retrieved from http://www.lpfch.org/sel/PackardES-REV.pdf

Pellegrini, A. D., & Bartini, M. (2000). An empirical comparison of methods of sampling aggression and victimization in school settings. Journal of Educational Psychology, 92, 360–366. doi:10.1037/0022-0663.92.2.360

Reijntjes, A., Kamphuis, J. H., Prinzie, P., & Telch, M. J. (2010). Peer victimization and internalizing problems in children: A meta-analysis of longitudinal studies. Child Abuse & Neglect, 34, 244–252. doi:10.1016/j.chiabu.2009.07.009

Rigby, K., & Slee, P. T. (1993a). Dimensions of interpersonal relation among Australian children and implications for psychological well-being. The Journal of Social Psychology, 133, 33–42. doi:10.1080/00224545.1993.9712116

Rigby, K., & Slee, P. T. (1993b). The Peer Relations Questionnaire (PRQ) [Measurement instrument]. Adelaide, Australia: University of South Australia.

Roth, D. A., Coles, M. E., & Heimberg, R. G. (2002). The relationship between memories for childhood teasing and anxiety and depression in adulthood. Journal of Anxiety Disorders, 16, 149–164. doi: 10.1016/S0887-6185(01)00096-2

Saleebey, D. (2008). Commentary on the strengths perspective and potential applications in school counseling. Professional School Counseling, 12, 68–75.

Shochet, I. M., Dadds, M. R., Ham, D., & Montague, R. (2006). School connectedness is an underemphasized parameter in adolescent mental health: Results of a community prediction study. Journal of Clinical Child and Adolescent Psychology, 35, 170–179. doi:10.1207/s15374424jccp3502_1

Sink, C. A., & Mvududu, N. H. (2010). Statistical power, sampling, and effect sizes: Three keys to research relevancy. Counseling Outcome Research and Evaluation, 1, 1–18. doi:10.1177/2150137810373613

Sink, C. A., & Spencer, L. R. (2005). My Class Inventory–Short Form as an accountability tool for elementary school counselors to measure classroom climate. Professional School Counseling, 9, 37–48.

Spriggs, A. L., Iannotti, R. J., Nansel, T. R., & Haynie, D. L. (2007). Adolescent bullying involvement and perceived family, peer, and school relations: Commonalities and differences across race/ethnicity. Journal of Adolescent Health, 41, 283–293. doi:10.1016/j.jadohealth.2007.04.009

Swearer, S. M., Espelage, D. L., Vaillancourt, T., & Hymel, S. (2010). What can be done about school bullying? Linking research to educational practice. Educational Researcher, 39, 38–47.

Tabaeian, S. R., Amiri, S., & Molavi, H. (2012). Factor analysis, reliability, convergent and discriminate [sic] validity of “The Peer Relationships Questionnaire” (PRQ). Studies in Learning & Instruction (Journal of Social Sciences and Humanities of Shiraz University), 3(2), 61–62.

Ttofi, M. M., Farrington, D. P., Lösel, F., & Loeber, R. (2011). Do the victims of school bullies tend to become depressed later in life? A systematic review and meta-analysis of longitudinal studies. Journal of Aggression, Conflict and Peace Research, 3, 63–73. doi:10.1108/17596591111132873

Van Schoiack-Edstrom, L., Frey, K. S., & Beland, K. (2002). Changing adolescents’ attitudes about relational and physical aggression: An early evaluation of a school-based intervention. School Psychology Review, 31, 201–216.

Villares, E., Colvin, K., Carey, J., Webb, L., Brigman, G., & Harrington, K. (2014). Convergent and divergent validity of the student engagement in school success skills survey. The Professional Counselor, 4, 541–552. doi:10.15241/ev.4.5.541

Villares, E., Frain, M., Brigman, G., Webb, L., & Peluso, P. (2012). The impact of student success skills on standardized test scores: A meta-analysis. Counseling Outcome Research and Evaluation, 3, 3–16. doi:10.1177/2150137811434041

Wang, J., Iannotti, R. J., & Nansel, T. R. (2009). School bullying among adolescents in the United States: Physical, verbal, relational, and cyber. Journal of Adolescent Health, 45, 368–375. doi:10.1016/j.jadohealth.2009.03.021

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1994). What helps students learn? Educational Leadership, 51(4), 74–79.

Webb, L. D., & Brigman, G. A. (2006). Student success skills: Tools and strategies for improved academic and social outcomes. Professional School Counseling, 10, 112–120.

Webb, L. D., Brigman, G. A., & Campbell, C. (2005). Linking school counselors and student success: A replication of the student success skills approach targeting the academic and social competence of students. Professional School Counseling, 8, 407–413.

Wentzel, K. R. (2003). Sociometric status and adjustment in middle school: A longitudinal study. Journal of Early Adolescence, 23, 5–28. doi:10.1177/0272431602239128

Wentzel, K. R., & Caldwell, K. (1997). Friendships, peer acceptance, and group membership: Relations to academic achievement in middle school. Child Development, 68, 1198–1209. doi:10.2307/1132301

Winne, P. H., & Nesbit, J. C. (2010). The psychology of academic achievement. Annual Review of Psychology, 61, 653–678.

Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81, 267–301. doi:10.3102/0034654311405999

Yoon, J. S. (2004). Predicting teacher interventions in bullying situations. Education and Treatment of Children, 27, 37–45.

Zins, J. E., Weissberg, R. P., Wang, M. C., & Walberg, H. J. (Eds.). (2004). Building academic success on social and emotional learning: What does the research say? New York, NY: Teachers College Press.

 

 

Melissa Mariani is an Assistant Professor at Florida Atlantic University. Linda Webb is Research Faculty III at Florida State University. Elizabeth Villares is an Associate Professor at Florida Atlantic University. Greg Brigman, NCC, is a Professor at Florida Atlantic University. Correspondence may be addressed to Melissa Mariani, 777 Glades Road, COE 47, 274, Boca Raton, FL 33431, mmarian5@fau.edu.