Sep 3, 2014 | Article, Volume 1 - Issue 2
Rebecca A. Newgent, Kristin K. Higgins, Stephanie E. Belk, Bonni A. Nickens Behrend, Kelly A. Dunbar
Group counseling has been highlighted as one effective intervention for at-risk students, yet debate remains as to the comparable efficacy of traditional interventions versus thematic interventions. This study compared two psychosocial educational programs, the PEGS and ARK Programs, designed to help elementary school students with social skills development, problem behaviors, bullying, and self-esteem with 15 elementary-aged students. Results revealed no differences between the programs and improvement on many indicators. Implications for school counselors are presented.
Keywords: elementary students, psychosocial education, prevention programs, school counseling, traditional interventions, thematic interventions
Group work can be an effective means of counseling at-risk students. As such, the American School Counselors Association (ASCA) has endorsed group work as an important component of school counseling (ASCA, 2005). Groups can help students learn to solve problems in an efficient and effective manner and is an ideal method for meeting the needs of at-risk populations (Akos & Milsom, 2007). Group counseling allows students to develop connections while at the same time exploring factors that may affect their achievement (Kayler & Sherman, 2009). Groups are used to address such issues as social skills (Bostick & Anderson, 2009), bolstering students’ self-perceptions (Eppler, Olsen, & Hidano 2009), preventing problem behavior (Todd, Campbell, Meyer, & Horner, 2008), increasing academic success (Brigman, Webb, & Campbell, 2007), and reducing school-wide bullying (Allen, 2010). Further, Quinn, Kavale, Mathur, Rutherford, and Forness (1999) conducted a meta-analysis (35 studies) of social skills interventions used with children exhibiting problem or emotional behaviors. Results revealed several important issues. First, it appears that there is a wide range of presenting deficits in children’s social skills. Second, social skills training is widely used as a mechanism to address problem behavior in children; however, it may not be as effective at addressing problem behaviors when used in isolation. The purpose of this study is to compare the effectiveness of two psychosocial intervention programs, Psychosocial Educational Groups for Students (PEGS) and the At-Risk Kids Groups (ARK) and to assess the impact of each program. The PEGS and ARK Programs are designed to help elementary school students in the following areas: social skills development, problem behaviors, bullying, and self-esteem.
Issues Addressed in Groups
According to Berry and O’Conner (2010), social skills are a set of learned behaviors that allow for positive social interactions, such as sharing, helping, initiating and sustaining relationships. Children who enter the academic setting with problem behaviors are often the children who lack the social skills to create and maintain positive social interactions. In a recent study by Bostick and Anderson (2009), 49 third graders with social skill deficits participated in a 10-week social skills program aimed at reducing loneliness and anxiety in an academic setting. Findings revealed significant reductions in loneliness and anxiety as well as significant increases in reading scores.
Groups also can be an effective manner in addressing children with problem behavior. For example, Hudley, Graham, and Taylor (2007) studied the level of children’s aggression in relation to personal responsibility. After a series of 12 lessons related to detecting other’s intentions, externalization, and positive responses, students showed improvement in socially acceptable behavior along with a reduction in overall aggression. For additional examples also see Cotugno (2009), McCurdy, Lannie, and Barnabus (2009), and Todd, Campbell, Meyers, and Horner (2008).
Olweus (1997) defines bullying as purposeful behavior, repeated over time, including an imbalance of power. Several psychoeducational programs have been developed to address the issue of bullying in schools (Horne, Bartolomucci, & Newman-Carlson, 2003; Jenson & Dieterich, 2007; Olweus, 1991). At-risk students are at particular risk for bully-related behaviors, including both roles of victim and perpetrator (Allen, 2010). Therefore, group intervention in the schools may be a beneficial way of directly addressing bullying.
Self-esteem, also known as self-concept, is often defined as the way in which children think about themselves in relation to their attributes and abilities (Kenny & McEachern, 2009). “Part of preventing problem behaviors is increasing the self-esteem of those with problem behaviors” (Newgent, Behrend, Lounsbery, Higgins, & Lo, 2010, p. 82). There are relatively few empirical studies, however, on the effectiveness of groups and self-esteem (e.g., O’Moore & Kirkham, 2001; Whitney & Smith, 1993).
Prior Research
A paucity of research exists on addressing multiple presenting problems in children. A major implication of the Quinn et al. (1999) meta-analysis suggested that psychoeducational groups may need to address more than one problem. A study conducted by Newgent et al. (2010) examined the effectiveness of a 6-week psychosocial educational group for students that addressed social skills, problem behaviors, bullying, and self-esteem. Results showed that students with multiple presenting issues benefited from a group addressing multiple issues. Further, results showed that students with no clinically relevant presenting issues also benefited from the multi-issue group (Newgent et al., 2010). In addition, some research is suggesting that while traditional, process-oriented groups can be effective at addressing problems, thematic groups have become more prevalent, especially when there is a time constraint (Hartzler & Brownson, 2001). While the benefit of thematic groups is a more specific or more common goal it also can lack the breadth of more traditional groups.
This study aimed to compare the effectiveness of two selective intervention programs (traditional vs. thematic) on measures of social skills, problem behaviors, teacher- and self-reports of peer relationships (bullying behaviors and peer victimization), self-esteem (self-worth, ability, self-satisfaction, and self-respect) and perception of self. Further, this study aimed to assess the impact of each of the intervention programs. The following research questions were tested: Is there a differential impact when comparing the PEGS Program to the ARK Program? Do the PEGS and ARK Programs have a positive impact on social skills, problem behaviors, peer relationships, and self-esteem?
Method
Participants
Eleven (n = 11) students were enrolled in the PEGS Program and four (n = 4) students were enrolled in the ARK Program. No attrition of program participants occurred. While participation was open to all students, teachers only recommended male students for both programs. Students in the PEGS Program were in the 4th grade and students in the ARK Program were in the 5th grade. In the PEGS Program five students (45%) identified as Caucasian/White, four (36%) identified as African American, one (9%) identified as Hispanic, and one (9%) identified as other. In the ARK Program one student (25%) identified as Caucasian, one student (25%) identified as African American, and two students (50%) identified as Hispanic. No students were identified as having some type of diagnosed learning, behavioral, or emotional disability.
Selective Intervention Program
One of the underlying tenants of the PEGS and ARK Programs is that students should not be labeled or stigmatized for having problems. Therefore, both programs utilized referrals that were based on underlying characteristics that lead to specific problems, not labels. Thus, students who are impulsive, depressed, dominant, lonely, easily frustrated, anxious, lack empathy, have low self-esteem, have difficulty following rules, are socially withdrawn, view violence in a positive way, have few friends, make negative attributions, have mood swings, instigate conflict, have difficulty handling conflict, or are aggressive or stressed were identified by their teachers and recommended for one of the two programs.
While both the PEGS and the ARK Program cover the same underlying psychosocial educational content, the primary difference is that the PEGS Program consists of traditional psychosocial education units while the ARK Program units are targeted toward peer victimization (i.e., bullying). Both PEGS and ARK provide a series of 6 one-half hour group sessions over the course of 6 weeks for elementary school students in grades 3–5. The six psychosocial education components of each of the intervention programs include: (a) improving social skills, (b) building and increasing self-esteem, (c) developing problem-solving skills, (d) assertiveness training, (e) enhancing stress/coping skills, and (f) prevention of mental health problems/problem behaviors. Lessons for the PEGS Program were adapted from Lively Lessons for Classroom Sessions (Sartori, 2000), More Lively Lessons for Classroom Sessions (Sartori, 2004), and the Missouri Comprehensive Guidance Programs (Frankenbert, Grandelious, Keller, & Schaaf, n.d.). These lessons focused on cooperation, encouraging students to be proud of who they are, breaking the chain of violence, handling anxiety and stress, and tolerance regarding differences. Lessons for the ARK Program were adapted from Bully Busters: A Teacher’s Manual for Helping Bullies, Victims, and Bystanders (Horne, Bartolomucci, & Newman-Carlson, 2003). These lessons focused on increasing awareness of bullying, building personal power, recognizing the bully and the victim, recommendations and interventions for helping victims, and relaxation and coping skills. Puppets were utilized in both programs to help model the lessons being taught. Worksheets related to the lessons also were utilized to help crystallize the concepts.
All recommended students participated in the one of the programs. Given class schedules, the school counselor recommended that students be assigned to one of the programs by grades. Therefore, all recommended 4th grade students were assigned to the PEGS Program and all recommended 5th grade students were assigned to the ARK Program. These assignments were done on a random basis. No control groups were utilized in this study at the school’s request. The same facilitators were used for both programs.
Instruments
Social Skills Improvement System – Teacher Form (SSiS–T). The Social Skills Improvement System – Teacher Form was developed by Gresham and Elliott (SSiS; 2008) and published by NCS Pearson, Inc. The SSiS–T is an 83-item rating scale designed specifically for teachers to use to assess children’s school-related problem behaviors and competencies. Scores reported for each of the three measurement areas are percentiles. Clinical levels for each of the three areas are as follows: social skills (≤ 16), problem behaviors (≥ 84), and academic competence (≤ 16). That is, lower scores on social skills and academic competence and higher scores on problem behaviors are considered clinically problematic. For the purposes of this study, only social skills and problem behaviors were evaluated. Cronbach alphas for social skills were .95 and .92 at pre- and post-test assessment, respectively. Cronbach alphas for problem behaviors were .86 and .89 at pre- and post-test follow-up assessment, respectively.
Social Skills Improvement System – Student Form (SSiS–S). The Social Skills Improvement System – Student Form was developed by Gresham and Elliott (SSiS; 2008) and published by NCS Pearson, Inc. The SSiS-S is a 75-item rating scale designed specifically for students aged 8–12 to use to assess their own school-related problem behaviors and competencies. Scores reported for each of the two measurement areas are percentiles. Clinical levels for each of the two areas are as follows: social skills (≤ 16) and problem behaviors (≥ 84). That is, lower scores on social skills and academic competence and higher scores on problem behaviors are considered clinically problematic. Cronbach alphas for social skills were .80 and .87 at pre- and post-test assessment, respectively. Cronbach alphas for problem behaviors were .74 and .70 at pre- and post-test assessment, respectively.
Peer Relationship Measure – Teacher Report. The Peer Relationship Measure – Teacher Report (Newgent, 2009a) was developed specifically for the PEGS and ARK Programs. It measures teachers’ perceptions about peer victimization. Nine items measure bullying behaviors and 9 items measure victimization. Items are scored as 0 = never, 1 = sometimes, and 2 = a lot, with scores ranging from 0 to 18 for each area. Scores reported for both of the measurement areas are totals. A high score indicates a high level of bullying behaviors and/or victimization; a low score indicates a low level of bully behaviors and/or victimization. Cronbach alphas for bullying behaviors were .89 and .90 at pre- and post-test assessment, respectively. Cronbach alphas for victimization were .78 and .90 at pre- and post-test assessment, respectively.
Peer Relationship Measure – Self Report. The Peer Relationship Measure – Self Report (Newgent, 2009b) was developed specifically for the PEGS and ARK Programs. It measures students’ perceptions about peer victimization. Nine items measure bullying behaviors and nine items measure victimization. Items are scored as 0 = never, 1 = sometimes, and 2 = a lot, with scores ranging from 0 to 18 for each area. Scores reported for both of the measurement areas are totals. A high score indicates a high level of bullying behaviors and/or victimization; a low score indicates a low level of bully behaviors and/or victimization. This measure is a parallel measure to the Peer Relationship Measure – Teacher Report. Cronbach alphas for bullying behaviors were .07 and .87 at pre- and post-test assessment, respectively. Cronbach alphas for victimization were .81 and .82 at pre- and post-test assessment, respectively.
Modified Rosenberg’s Self-Esteem Inventory (a). The Modified Rosenberg’s Self-Esteem Inventory (a) (Zimprich, Perren, & Hornung, 2005) measures an individual’s level of self-esteem (i.e., perception of self-worth, ability, self-satisfaction, and self-respect) and was completed by the students. The 10-item inventory uses a 4-point likert scale (strongly agree, agree somewhat, disagree somewhat, and strongly disagree). Five of the items are reverse coded and the score reported is the total, which ranges from 0–30. A high score indicates a high level of self-esteem; a low score indicates a low level of self-esteem. Cronbach alphas were .60 and .54 at pre- and post-test assessment, respectively.
Modified Rosenberg’s Self-Esteem Inventory (b). The Modified Rosenberg’s Self-Esteem Inventory (b) (Zimprich et al., 2005) measures an individual’s self-esteem (i.e., perception of self) and was completed by the students. The 4-item inventory uses a 5-point likert scale (never, seldom, sometimes, often, always). One of the items is reverse coded and the score reported is the total, which ranges from 4–20. A high score indicates a high level of self-esteem; a low score indicates a low level of self-esteem. Cronbach alphas were .59 and -1.56 at pre- and post-test assessment, respectively.
Procedure
Fifteen students were recommended by their teachers for the PEGS and ARK Programs and 15 parental consents/child assents were received. Recommending teachers completed the SSiS–T (Gresham & Elliott, 2008) and the Peer Relationship Measure – Teacher Report (Newgent, 2009a) at the start of the PEGS and ARK Programs and at two months after the conclusion of the PEGS and ARK Programs. Ideally, we would have liked an additional follow-up assessment; however, there was difficulty in securing a longer period of involvement in the assessments. Students completed the SSiS–S (Gresham & Elliott, 2008), the Peer Relationship Measure – Self Report (Newgent, 2009b), the Modified Rosenberg’s Self-Esteem Inventory (a) (Zimprich et al., 2005), and the Modified Rosenberg’s Self-Esteem Inventory (b) (Zimprich et al., 2005) at the start of the PEGS and ARK Programs and at two months after the conclusion of the PEGS and ARK Programs. All participating students received a certificate of completion at the conclusion of the last session.
Sessions for both programs were co-facilitated by two graduate students in the counseling and educational research programs. Both facilitators passed criminal background checks and had prior professional experience working with children who exhibit problematic behaviors. Supervision was provided to the facilitators by two counseling faculty members overseeing the programs.
Data Analytic Plan
Each student who participated in the PEGS or ARK Programs was assessed at two time points utilizing a variety of measurement instruments (see Instruments section). Pre-test scores on each of the instruments were initially analyzed using independent-samples t-tests to test the underlying assumption that participant scores between the groups were not significantly different. Next, scores were analyzed using independent-samples t-tests to assess for significant differences on the post-test assessment scores between the PEGS Program and the ARK Program participants. Finally, paired-samples t-tests, comparing pre-test to post-test assessment scores within each program, were used to assess the impact of each of the programs. Effect sizes are reported as small (d = .20), medium (d = .50), and large (d = .80) (Cohen, 1992; O’Rourke, Hatcher, & Stepanski, 2005).
Results
Between-Group Analysis
Teacher-reported measures. Results were initially analyzed using independent-samples t-tests comparing the pre-test assessment scores of the PEGS Program to the ARK Program. Statistical comparisons are displayed in Table 1. Analysis of teacher-reported social skills failed to reveal a significant difference between the two groups, t(12) = 0.76; p < .46. The effect size was computed as d = .76, which represents a large effect. Analysis of teacher-reported problem behaviors also failed to reveal a significant difference between the groups, t(10.73) = -0.15; p < .88. The effect size was computed as d = .08, which represents a very small effect. Analysis of teacher-reported bullying behaviors failed to reveal a significant difference between the groups, t(12) = -0.97; p < .35. The effect size was computed as d = .97, which represents a large effect. Analysis of teacher-reported peer victimization also failed to reveal a significant difference between the groups, t(12) = -2.14; p < .054. The effect size was computed as d = 2.13, which represents a very large effect.
Results were then analyzed using independent-samples t-tests comparing the post-test assessment scores of the PEGS Program to the ARK Program. Statistical comparisons are displayed in Table 1. Analysis of teacher-reported social skills failed to reveal a significant difference between the two groups, t(12) = 0.08; p < .94. The effect size was computed as d = .08, which represents a very small effect. Analysis of teacher-reported problem behaviors also failed to reveal a significant difference between the groups, t(12) = -0.28; p < .78. The effect size was computed as d = .28, which represents a small effect. Analysis of teacher-reported bullying behaviors failed to reveal a significant difference between the groups, t(12) = -1.56; p < .14. The effect size was computed as d = 1.56, which represents a very large effect. Analysis of teacher-reported peer victimization revealed a significant difference between the groups, t(11.29) = -3.48; p < .005. The effect size was computed as d = 1.84, which represents a very large effect.
Self-reported measures. Results were initially analyzed using independent-samples t tests comparing the pre-test assessment scores of the PEGS Program to the ARK Program. Mean scores, significance, and effect size are displayed in Table 1. Analysis of self-reported social skills failed to reveal a significant difference between the two groups, t(3.25) = 1.16; p < .32. The effect size was computed as d = 5.88, which represents a very large effect. Analysis of self-reported problem behaviors also failed to reveal a significant difference between the groups, t(13) = 0.56; p < .59. The effect size was computed as d = .56, which represents a medium effect. Analysis of self-reported bullying behaviors failed to reveal a significant difference between the groups, t(13) = 1.25; p < .23. The effect size was computed as d = 1.25, which represents a very large effect. Analysis of self-reported peer victimization also failed to reveal a significant difference between the groups, t(13) = 1.09; p < .30. The effect size was computed as d = 1.09, which represents a very large effect. Analysis of self-reported self-esteem failed to reveal a significant difference between the groups, t(13) = 0.18; p < .86. The effect size was computed as d = .18, which represents a small effect. Analysis of self-reported perception of self also failed to reveal a significant difference between the groups, t(13) = -1.17; p < .26. The effect size was computed as d = 1.17, which represents a very large effect.
Results were then analyzed using independent-samples t-tests comparing the post-test assessment scores of the PEGS Program to the ARK Program. Mean scores, significance, and effect size are displayed in Table 1. Analysis of self-reported social skills failed to reveal a significant difference between the two groups, t(13) = 2.03; p < .06. The effect size was computed as d = 2.03, which represents a very large effect. Analysis of self-reported problem behaviors also failed to reveal a significant difference between the groups, t(13) = 0.56; p < .59. The effect size was computed as d = .56, which represents a medium effect. Analysis of self-reported bullying behaviors failed to reveal a significant difference between the groups, t(13) = -1.31; p < .21. The effect size was computed as d = 3.40, which represents a very large effect. Analysis of self-reported peer victimization also failed to reveal a significant difference between the groups, t(13) = 0.82; p < .43. The effect size was computed as d = .82, which represents a large effect. Analysis of self-reported self-esteem failed to reveal a significant difference between the groups, t(13) = 0.79; p < .44. The effect size was computed as d = .80, which represents a large effect. Analysis of self-reported perception of self also failed to reveal a significant difference between the groups, t(13) = 0.33; p < .75. The effect size was computed as d = .33, which represents a small to medium effect.
Within Group Analysis – PEGS
Teacher-reported measures. Results were analyzed using paired-samples t-tests comparing the pre-test assessment scores of the PEGS Program to the post-test assessment scores of the PEGS Program. Mean scores, significance, and effect size are displayed in Table 2. Analysis of teacher-reported social skills failed to reveal a significant difference between the pre- and post-test assessments, t(10) = 0.38; p < .71. The effect size was computed as d = .11, which represents a very small effect. Analysis of teacher-reported problem behaviors also failed to reveal a significant difference between the pre- and post-test assessments, t(10) = 1.95; p < .08. The effect size was computed as d = .59, which represents a medium effect. Analysis of teacher-reported bullying behaviors failed to reveal a significant difference between the pre- and post-test assessments, t(10) = 0.13; p < .90. The effect size was computed as d = .04, which represents a very small effect. Analysis of teacher-reported peer victimization also failed to reveal a significant difference between the pre- and post-test assessments, t(10) = -0.20; p < .84. The effect size was computed as d = .06, which represents a very small effect.
Self-reported measures. Results were analyzed using paired-samples t tests comparing the pre-test assessment scores of the PEGS Program to the post-test assessment scores of the PEGS Program. Mean scores, significance, and effect size are displayed in Table 2. Analysis of self-reported social skills failed to reveal a significant difference between the pre- and the post-test assessments, t(10) = -1.52; p < .16. The effect size was computed as d = .46, which represents a medium effect. Analysis of self-reported problem behaviors revealed a significant difference between the pre- and post-test assessments, t(10) = 2.81; p < .02. The effect size was computed as d = .85, which represents a large effect. Analysis of self-reported bullying behaviors failed to reveal a significant difference between the pre- and post-test assessments, t(10) = 0.52; p < .62. The effect size was computed as d = .15, which represents a small effect. Analysis of self-reported peer victimization revealed a significant difference between the pre- and post-test assessments, t(10) = 2.95; p < .01. The effect size was computed as d = .89, which represents a large effect. Analysis of self-reported self-esteem failed to reveal a significant difference between the pre- and post-test assessments, t(10) = -0.22; p < .83. The effect size was computed as d = .07, which represents a very small effect. Analysis of self-reported perception of self also failed to reveal a significant difference between the groups, t(10) = 0.76; p < .46. The effect size was computed as d = .23, which represents a small effect.
Within Group Analysis – ARK
Teacher-reported measures. Results were analyzed using paired-samples t-tests comparing the pre-test assessment scores of the ARK Program to the post-test assessment scores of the ARK Program. Mean scores, significance, and effect size are displayed in Table 3. Analysis of teacher-reported social skills revealed a significant difference between the pre- and post-test assessments, t(2) = 6.25; p < .02. The effect size was computed as d = 3.61, which represents a very large effect. Analysis of teacher-reported problem behaviors failed to reveal a significant difference between the pre-and post-test assessments, t(2) = 1.84; p < .21. The effect size was computed as d = 1.06, which represents a very large effect. Analysis of teacher-reported bullying behaviors revealed a significant difference between the pre- and post-test assessments, t(2) = 5.00; p < .04. The effect size was computed as d = 2.88, which represents a very large effect. Analysis of teacher-reported peer victimization failed to reveal a significant difference between the pre- and post-test assessments, t(2) = 0.00; p < 1.00. The effect size was computed as d = 0, which represents no effect.
Self-reported measures. Results were analyzed using paired-samples t-tests comparing the pre-test assessment scores of the ARK Program to the post-test assessment scores of the ARK Program. Mean scores, significance, and effect size are displayed in Table 3. Analysis of self-reported social skills revealed a significant difference between the pre- and post-test assessments, t(3) = -4.14; p < .03. The effect size was computed as d = 2.07, which represents a very large effect. Analysis of self-reported problem behaviors failed to reveal a significant difference between the pre- and post-test assessments, t(3) = 1.31; p < .28. The effect size was computed as d = .65, which represents a medium effect. Analysis of self-reported bullying behaviors revealed a significant difference between the pre- and post-test assessments, t(3) = 5.42; p < .01. The effect size was computed as d = 2.71, which represents a very large effect. Analysis of self-reported peer victimization failed to reveal a significant difference between the pre- and post-test assessments, t(3) = 1.14; p < .34. The effect size was computed as d = .57, which represents a medium effect. Analysis of self-reported self-esteem also failed to reveal a significant difference between the pre- and post-test assessments, t(3) = -0.80; p < .48. The effect size was computed as d = .40, which represents a small to medium effect. Analysis of self-reported perception of self failed to reveal a significant difference between the groups, t(3) = -0.69; p < .54. The effect size was computed as d = .34, which represents a small effect.
Discussion
The purpose of this study was to compare and assess the impact of two selective intervention psychosocial education programs (traditional vs. thematic) on measures of social skills, problem behaviors, teacher- and self-reports of peer relationships (bullying behaviors and peer victimization), self-esteem (self-worth, ability, self-satisfaction, and self-respect) and perception of self in a small number of elementary school students. The PEGS and ARK Programs were designed to be short-term, non-stigmatizing programs that can easily augment current school counselor or school-based counseling services. Two groups of students were assigned to one of two programs. A discussion of the comparison between the programs and impact of each program follows.
Comparison
Findings indicated that there were no significant differences between the pre-test assessment measures when comparing the PEGS and ARK Program participants. That is, the participants in each group were comparable prior to their participation in their respective programs. Therefore, should we find significant differences between the two programs at post-test assessment, we may attribute those differences to the impact of the program. Further, there were no significant differences between the post-test assessment measures when comparing the two programs’ participants, with the exception of teacher-reported peer victimization. In other words, participants in each group were comparable after their participation in the respective programs with the exception of participants in the ARK Program having significantly lower levels of teacher-reported victimization than those in the PEGS Program. Note however that there was no difference between the pre- and the post-test assessment for ARK participants on this measure and only a non-significant increase for PEGS participants.
While these results indicate that the implementation of thematic programming directed at peer harassment does not have a significantly greater positive impact on students than traditional programming directed at students who show more generalized problems, there were some large effects (d ≥ .80) between the groups. This may indicate that there may have been some differences between the participants of the two programs, although not statistically significant.
PEGS
Findings indicated that there was significant improvement on self-reported problem behaviors and peer victimization when comparing the pre- and post-test assessments for the PEGS Program participants. In other words, students reported fewer problem behaviors and a decrease in victimization by their peers. While not significant, improvement also was found for teacher-reported problem behaviors and bully behaviors and self-reported social skills, bully behaviors, and self-esteem.
ARK
Findings indicated that there was significant improvement on teacher-reported bully behavior and on self-reported social skills and bully behaviors. In other words, students reported increased social skills and both teachers and students reported a decrease in victimization. Additionally, meaningful improvements (d ≥ .80) were found on teacher-reported problem behaviors. That is, teachers noted fewer problem behaviors in students after their participation in the ARK Program.
Conclusions
Finding effective programming that can positively impact at-risk students can be difficult. Further complicating the issue is the onslaught of thematic programming targeting specific groups of at-risk students (Hartzler & Brownson, 2001). While targeted programming can be beneficial to a select group of students it may exclude other students who can benefit but may not have the same “label.” This study showed that the more traditional group (i.e., PEGS) was equally effective as the more thematic group (i.e, ARK).
Limitations and Future Directions
There were several limitations to this study. First, we were limited to working within the parameters the school established. That is, we were limited to 4th and 5th grade males only and we were limited to providing services during their respective lunch periods. It may have been beneficial to have both genders in each group as well as a mix of grades. Second, group sizes were small. While larger group sizes would result in a greater ability to detect a statistically significant difference if one exists, larger group sizes also can result in reduced effectiveness. Third, we were unable to have a no-treatment group. The use of a control group may have resulted in a more robust study. Finally, the unequal group sizes may have impacted the comparison of the two groups. We attempted to adjust for this by using the Satterthwaite method when the equality of variances was significantly different (O’Rourke et al., 2005).
Implications
There are several implications for school and school-based counselors. First, it would be important in program management if school and school-based counselors are made aware that traditional psychoeducational groups are similarly effective to thematic psychoeducational groups. With this information they can make more informed decisions about the type of groups they implement as well as the type of intervention programs they offer and purchase. If the results of this study hold true for other groups of students and other schools, school and school-based counselors who choose to utilize more traditional programming would be able to provide these services to a broader range of students, consistent with the ASCA Model (ASCA, 2005), and not limit it to a select group of students targeted for a specific issue. Additionally, the purchase of thematic intervention programs can be costly given that the use is limited to a smaller number of students and several different types of programs are needed to address students with differing issues.
Second, the PEGS Program is an empirically supported program (see Newgent et al., 2010). It is a short-term, inexpensive, and non-intrusive program that can positively impact students with a variety of underlying issues. School and school-based counselors can easily augment their services with the implementation of this program. School administrators may be more supportive of a program that is both cost effective and would not hinder counselors from fulfilling other duties. The ever changing demands on school and school-based counselors will most likely continue. Counselors need effective tools that they can use to help students address problems, increase self-esteem, improve social skills, and decrease peer victimization.
References
Akos, P., & Milsom, A. (2007). Introduction to special issue: Group work in K-12 schools. The Journal for Specialists in Group Work, 32, 5–8. doi:10.1080/01933920600977440
Allen, K. P. (2010). A bullying intervention system: Reducing risk and creating support for aggressive students. Preventing School Failure, 54, 199–209.doi:10.1080/10459880903496289
American School Counselor Association. (2005). The ASCA national model: A framework for school counseling programs. Alexandria, VA: Author.
Berry, D., & O’Conner, E. (2010). Behavioral risk, teacher-child relationships, and social skill development across middle childhood: A child-by-environment analysis of change. Journal of Applied Developmental Psychology, 31, 1–14. doi:10.1016/j.appdev.2009.05.001
Bostick, D., & Anderson, R. (2009). Evaluating a small-group counseling program-A model for program planning and improvement in the elementary setting. Professional School Counseling, 12, 428–433.
Brigman, G. A., Webb, L. D., & Campbell, C. (2007). Building skills for school success: Improving the academic and social competence of students. Professional School Counselor, 10, 279–288.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. doi:10.1037/0033-2909.112.1.155
Cotugno, A. J. (2009). Social competence and social skills training and intervention for children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, 1268–1277. doi:10.1007/s10803-009-0741-4
Eppler, C., Olsen, J. A., & Hidano, L. (2009). Using stories in elementary school counseling: Brief, narrative techniques. Professional School Counseling, 12, 387–391.
Frankenbert, J., Grandelious, M., Keller, K., & Schaaf, P. (n.d.). STAR problem solving model. Missouri Comprehensive Guidance Programs. Retrieved from http://missouricareereducation.org
Gresham, F. M., & Elliott, S. N. (2008). Social Skills Improvement System. Minneapolis, MN: NCS Pearson.
Hartzler, B., & Brownson, C. (2001). The utility of change models in the design and delivery of thematic group interventions: Applications to a self-defeating behaviors group. Group Dynamics: Theory, Research, and Practice, 5(3), 191–199. doi:10.1037/1089-2699.5.3.191
Horne, A. M., Bartolomucci, C. L., & Newman-Carlson, D. (2003). Bully busters: A teacher’s manual for helping bullies, victims, and bystanders. Champaign, IL: McNaughton & Gunn.
Hudley, C., Graham, S., & Taylor, A. (2007). Reducing aggressive behavior and increasing motivation in school: The evolution of an intervention to strengthen school adjustment. Educational Psychologist, 42, 251–260. doi:10.1080/00461520701621095
Jenson, J. M., & Dieterich, W. A. (2007). Effects of a skills-based prevention program on bullying and bully victimization among elementary school children. Prevention Science, 8, 285–296. doi:10.1007/s11121-007-0076-3
Kayler, H., & Sherman, J. (2009). At-risk ninth-grade students: A psychoeducational group approach to increase study skills and grade point averages. Professional School Counseling, 12, 434–439.
Kenny, M. C., & McEachern, A. (2009). Children’s self concept: A multicultural comparison. Professional School Counseling, 12(3), 207–212.
McCurdy, B. L, Lannie, A. L., & Barnabas, E. (2009). Reducing disruptive behavior in an urban school cafeteria: An extension of the Good Behavior Game. Journal of School Psychology, 47, 39–54. doi:10.1016/j.jsp.2008.09.003
Newgent, R. A. (2009a). The Peer Relationship Measure – Teacher Report. Unpublished Assessment, University of Arkansas at Fayetteville.
Newgent, R. A. (2009b). The Peer Relationship Measure – Self Report. Unpublished Assessment, University of Arkansas at Fayetteville.
Newgent, R. A., Behrend, B. A., Lounsbery, K. L., Higgins, K. K., & Lo, W. J. (2010). Psychosocial educational groups for students (PEGS): An evaluation of the treatment effectiveness of a school-based behavioral intervention program. Counseling Outcome Research and Evaluation, 1, 80–84. doi:10.1177/215037810373919
Olweus, D. (1991). Bully/victim problems among school children: Basic facts and effects of a school based intervention program. In D. Pepler & K. Rubin (Eds.), The development and treatment of childhood aggression (pp. 411–448). Hillsdale, NJ: Lawrence Erlbaum. doi:10.01111/j.1467-9507.1995.tb00066.x
Olweus, D. (1997). Bully/victim problems in school: Facts and interventions. European Journal of Psychology of Education, 12, 495–510. Retrieved from http://ispa.pt/ejpe/
O’Moore, M., & Kirkham, C. (2001). Self-esteem and its relationship to bullying behavior. Aggressive Behavior, 27, 269–283. doi:10.1002/ab.1010
O’Rourke, N., Hatcher, L., & Stepanski, E. J. (2005). A step-by-step approach to using SAS for univariate & multivariate statistics (2nd ed.). Cary, NC: SAS Institute Inc.
Quinn, M. M., Kavale, K. A., Mathur, S. R., Rutherford, R. B., Jr., & Forness, S. R. (1999). A meta-analysis of social skills interventions for students with emotional or behavioral disorders. Journal of Emotional & Behavioral Disorders, 7(1), 54.
Sartori, R. S. (2000). Lively lessons for classroom sessions. Warminster, PA: MAR-CO Products.
Sartori, R. S. (2004). More lively lessons for classroom sessions. Warminster, PA: MAR-CO Products.
Todd, A. W., Campbell, A. L., Meyer, G. G., & Horner, R. H. (2008). The effects of a targeted intervention to reduce problem behaviors. Journal of Positive Behavior Interventions, 10, 46–55. doi:10.1177/10983007073113639
Whitney, I., & Smith, P. K. (1993). A survey of the nature and extent of bullying in junior/middle and secondary schools. Education Resources, 35, 3–25.
Zimprich, D., Perren, S., & Hornung, R. (2005). A two-level confirmatory factor analysis of a modified Rosenberg self-esteem scale. Educational and Psychological Measurement, 65, 465–481. doi:10.1177/00131644042724
Rebecca A. Newgent, NCC, is a Licensed Professional Clinical Counselor with Supervision Designation in Ohio, a Licensed Professional Counselor with Supervision Specialization in Arkansas, and Professor and Chairperson of the Department of Counselor Education at Western Illinois University – Quad Cities. Kristin K. Higgins is a Licensed Professional Counselor with Supervision Specialization in Arkansas and an Assistant Professor of School Counseling in the Counselor Education Program at the University of Arkansas. Stephanie E. Belk is a certified school counselor in the state of Arkansas and a doctoral student in the Counselor Education Program at the University of Arkansas. Kelly A. Dunbar, NCC, is a Licensed Professional Counselor in Oklahoma, a doctoral candidate in the Counselor Education Program at the University of Arkansas, and is an Assistant Professor of Counseling and Psychology at Northeastern State University. Bonni A. Nickens Behrend has a Master of Science degree in Education Statistics and Research Methods and is currently a master’s student in the Counselor Education Program and a Senior Graduate Research Assistant at the National Office for Research on Measurement and Evaluation Systems at the University of Arkansas. This research was supported in part by a grant from Mental Health America in Northwest Arkansas. Correspondence can be addressed to Rebecca A. Newgent, Department of Counselor Education, 3561 60th Street, Moline, IL, 61265, ra-newgent@wiu.edu.
Sep 3, 2014 | Article, Volume 1 - Issue 1
Chiharu Hensley
Raising a child with special needs exacerbates the inherent challenges of parenting. Although the needs of children with special needs are addressed frequently in the literature, the needs of the parents of children with special needs are often neglected. In order to offer effective and useful services for parents of children with special needs, this article examines the types and nature of support services used by the parents of children with special needs and the effectiveness of those support services in reducing the parents’ stress levels and/or increasing their coping skills. Seventy-four parents of special needs children were assessed and results revealed that low-cost services, particularly those that resulted in mutual support, were a significant priority among parents. The article concludes with a discussion of clinical implications and needed directions for future research.
Keywords: parenting, children, special needs, support services, counseling
Parenting involves much effort and countless responsibilities. Child rearing can be one of the most challenging tasks with which a person is confronted. Raising a child with special needs intensifies the challenge significantly. However, although the needs of children with special needs are addressed frequently in professional literature and in the media, the needs of parents of children with special needs are addressed far less often. In order to offer effective and useful services for parents of children with special needs, their experiences with common issues and concerns and how their needs can be met must be investigated and understood because such information is essential to enable parents to feel empowered in raising their children with special needs.
Parents of children with special needs often experience high levels of stress from both internal and external factors. For example, a study conducted by Heiman (2002) revealed that 84.4% of the participants who had children with various special needs experienced feelings including “depression, anger, shock, denial, fear, self-blame, guilt, sorrow, grief, confusion, despair, [and/or] hostility” at the time of their children’s first diagnoses. Barnett, Kaplan-Estrin, and Fialka (2003) reported a study of parents of children who were mildly or moderately impaired that showed about half of the parents were still experiencing negative responses to their children’s diagnoses two or more years after the initial diagnosis.
In addition, parents of children with special needs may suffer being stereotyped by others. For example, Goddard, Lehr, and Lapadat (2000) used focus groups to collect individual narratives from parents of children with special needs. They found that, more than the parents’ guilt or the condition of the child, being perceived as a victim of a tragedy and the sole advocate for the child as well as a lack of understanding from others, including professionals, contributes to parental stress. Financial concern is another external factor which contributes to high stress levels in parents raising children with special needs. Looman, O’Conner-Von, Ferski, and Hildenbrand (2009) found that the severity of a child’s special needs increased the odds of financial burden experienced by the family. Clearly, there are a variety of both internal and external stressors, and accompanying emotional reactions, with which parents of children with special needs are confronted. Therefore, providing services to reduce the stress and negative feelings to minimum levels would lead to better quality of life for the parents of children with special needs.
Given the relative lack of attention to the support service needs of parents raising children with special needs, the purpose of this study was to conduct an exploratory investigation of the types of services used by parents of children with special needs and the effectiveness of those services for reducing parents’ stress levels and increasing their coping skills.
Four primary research questions were addressed in this study:
1. What are the types of services used by parents of children with special needs?
2. How effective are services in reducing stress levels of such parents?
3. How effective are services in increasing the coping skills of parents?
4. What are some of the needs of parents which may be met by counseling services?
Method
There were two major parts to this research. The first involved distribution of a survey to parents of children with special needs and the second involved an extensive interview with a representative parent of a child with special needs. In the first part of the study a survey was used to collect data for approximately one year. Potential respondents included parents and/or primary caregivers of preschool or school-age children with special needs who resided in a Midwestern state. No restriction was placed on the potential respondents based on the type or number of special needs their child had. Participants were recruited through contact with organizations for families of children with special needs (e.g., local associations for learning disabilities, pervasive developmental disabilities, and physical disabilities) and snowball sampling with assistance of professionals at local public schools who work with children with special needs and their parents. An online survey, the primary means of data collection, was created using a commercial website (www.surveymonkey.com), and potential respondents were directed to the survey webpage from either the websites of the organizations or by typing in the website address found on a distributed survey invitation flyer. A paper version of the survey was prepared for participants from a university clinic for speech and hearing.
The second part of the study involved an individual follow-up interview. Initially, the intent was to garner enough participants for a focus group activity. Unfortunately, however, of all the survey respondents, only one expressed interest in participating in a focus group. Therefore, this respondent was selected and interviewed in order to explore the stressors, challenges, and supports available for the parents in greater depth. The interview was audio-taped and transcribed by the investigator.
Results
There were a total of 74 respondents. Among the respondents, 70 (94.6%) completed the survey online and 4 (5.4%) completed the paper form of the survey. Selected survey items and the resultant data are shown in Table 1.
Table 1
Selected Survey Items and the Resultant Data
Survey Items No. of Responses % Responses
Items for all the respondents (N = 74)
How would you rate your degree of stress on the following scale?
In the last month?
Very low 0 0.0
Low 8 10.8
Moderate 22 29.7
High 33 44.6
Very High 9 12.2
In the last year?
Very low 0 0.0
Low 5 6.8
Moderate 23 31.1
High 24 32.4
Very High 21 28.4
What would be the ratio of each factor that might be contributing to your stress level?
Raising a child(ren) with special needs
About 1–25% 9 12.2
About 26–40% 15 20.3
About 41–60% 15 20.3
About 61–80% 23 31.1
About 81–100% 11 14.9
Financial concerns
About 1–25% 15 20.3
About 26–40% 20 27.0
About 41–60% 12 16.2
About 61–80% 7 9.5
About 81–100% 15 20.3
Have you sought professional services (i.e., therapies) in dealing with your stress of raising a child(ren) with special needs?
Yes 30 40.5
No 43 58.1
If you answered No to the previous question, what was (were) your reason(s) for not seeking professional services (i.e., therapies)? (n = 43)
Unable to afford the service 5 11.6
Schedule conflict 7 16.3
Did not know about any service available 7 16.3
Unable to find a service that seemed
helpful for your needs 12 27.9
Counseling as a category of received service (n=30)
Type of service you have received:
Individual counseling 22 73.3
Couples counseling 3 10.0
Family counseling 7 23.3
How helpful was the service for dealing with your stress?
Very helpful 7 23.3
Somewhat helpful 12 40.0
Neutral 2 6.7
Somewhat unhelpful 2 6.7
Very unhelpful 3 10.0
Compared with your stress level before receiving service, how much has it changed after
receiving service?
Not changed at all 2 6.7
Greatly reduced 12 40.0
Somewhat reduced 7 23.3
Unsure 2 6.7
Somewhat increased 2 6.7
Greatly increased 2 6.7
Compared with your outlook on raising your child(ren) with special needs before receiving service, how much has it changed after receiving service?
Not changed at all 3 10.0
Greatly more optimistic 12 40.0
Somewhat more optimistic 4 13.3
Unsure 6 20.0
Somewhat more pessimistic 1 3.3
Greatly pessimistic 0 0.0
Group as a category of received service (n=16)
Group counseling 2 12.5
Support group 14 87.5
How helpful was the service for dealing with your stress?
Very helpful 6 37.5
Somewhat helpful 8 50.0
Neutral 1 6.3
Somewhat unhelpful 0 0.0
Very unhelpful 1 6.3
Compared with your stress level before receiving service, how much has it changed after receiving service?
Not changed at all 0 0.0
Greatly reduced 10 62.5
Somewhat reduced 3 18.8
Unsure 1 6.3
Somewhat increased 2 12.5
Greatly increased 0 0.0
Compared with your outlook on raising your child(ren) with special needs before receiving service, how much has it changed after receiving service?
Not changed at all 1 6.3
Greatly more optimistic 8 50.0
Somewhat more optimistic 2 12.5
Unsure 3 18.8
Somewhat more pessimistic 1 6.3
Greatly more pessimistic 0 0.0
Items for the respondents who sought a professional service(s) in the past for dealing
with their stress of raising their children with special needs (n=30)
What have you gained from receiving service(s)?
Peer support 8 26.7
Professional support 7 23.3
Network 10 33.3
Specific knowledge about the
child(ren)’s disability(ies) 14 46.7
Specific skills for dealing with
the child(ren)’s needs 13 43.3
What are some of the factors that you consider when choosing a service?
Cost (including transportation
and session fees) 20 66.7
Schedule/frequency 21 70.0
Format (e.g., individual vs. group vs.
psychoeducational vs. counseling) 16 53.3
How likely are you to seek an additional service(s) in the future?
Very likely 9 30.0
Likely 8 26.7
Unsure 6 20.0
Unlikely 1 3.3
Very unlikely 0 0.0
If you were to receive an additional service(s), what would be the most likely format/venue?
Individual counseling 15 50.0
Couples counseling 4 13.3
Family counseling 8 26.7
Group counseling 1 3.3
Support group 13 43.3
Parenting training individual sessions 4 13.3
Parenting training group sessions 6 20.0
Individual psychoeducational sessions 0 0.0
Psychoeducational group sessions 3 10.0
Coping skills—individual sessions 5 16.7
Coping skills—group sessions 3 10.0
Stress management—individual sessions 8 26.7
Stress management—group sessions 4 13.3
Note. Some of the items allowed multiple answers by a single respondent. Percentage of respondents for each item was measured based on the number of respondents corresponding to specific items.
Some of the 74 respondents did not provide responses for all items. The respondent group included 67 females (90.5%) and 63 (85.1%) participants who identified themselves as Caucasian/White. Thirty-five respondents (47.3%) were between ages 31 and 40, and 58 (78.4%) were married. Fifty-nine of the respondents (79.7%) had one child with special needs and 31 (41.9 %) reported the child’s disability as moderate.
In regard to stress levels, 33 respondents (44.6%) indicated that they had experienced a high degree of stress in the past month, and 45 (60.8%) indicated that they had experienced either a high or very high degree of stress in the past year. Twenty-three respondents (31.1%) indicated that raising their child with special needs contributed to about 61–80% of their total stress level, and 20 (27.0%) indicated that their financial concerns contributed to about 26–40% of their total stress level. In regard to help seeking, 45 (60.8%) indicated that they had never sought professional services (e.g., various possible therapies) to cope with the stress of raising a child with special needs. The most frequently cited (n = 12, 27.9%) reason for not seeking support services was that they were unable to find services that they perceived to be helpful for their needs.
Among the 30 respondents who had sought professional services, 22 (73.3%) indicated that they had sought individual counseling (which also was the most used type of service). The second most used type of service was support groups, in which 14 respondents (46.7%) indicated that they had joined or were current members of a support group. Among those who had received individual, couple, family, or any combination of counseling, 19 (73.1%) indicated that their stress levels were reduced to some or a great extent after receiving such service(s) and 16 (61.6%) responded that their outlook on raising their child with special needs became somewhat or greatly more optimistic.
Specifically, among the 16 (53.3%) who had received either group counseling, participated in support groups, or both, 13 (81.3%) indicated that their stress levels were somewhat or greatly reduced and 10 (62.5%) indicated that their outlook on raising their child) with special needs became somewhat or greatly more optimistic. Finally, 14 (46.7%) responded that they had gained specific knowledge about the child’s disability from receiving the services and 13 (43.3%) responded that they had gained specific skills for coping with the child’s needs.
Although the respondents in this latter subgroup had participated in a wide variety of support services, it appears that most were psychoeducational in nature. Seventeen respondents (56.7%) also reported that they were either likely or very likely to seek additional services in the future. The three most selected types of services that these respondents would most likely seek were individual counseling (n=15, 50.0%), support groups (n=13, 43.3%), and family counseling (n=8, 26.7%). Session schedule and frequency, cost (including transportation and session fees), and format of the service were all important factors considered in use of support services.
The second part of the study was an interview with the mother of a son with cerebral palsy in order to gather information about personal experiences, particularly those contributing to her level of stress. The interview was conducted at a house close to the hospital to which she periodically brought her son for treatment. At the time of the interview, Amy (a pseudonym), the mother, was 39 years old, and Michael (a pseudonym), her son, was two years old. Amy was Caucasian, between 31 and 40 years old, married, and had one child with special needs; therefore, she was “typical” of the majority of the respondents to the survey. Specific interview questions were not prepared in advance. Rather, Amy was asked to convey her most important and/or strongest experiences and emotions as a mother of a child with special needs.
A wide variety of issues were discussed during the interview, but the most pressing issue mentioned by Amy was the lack of available resources for parents of children with special needs. Amy related that large cities might have many resources available, “but especially not my little small town—the resources are so limited.” She talked about how in attempting to acquire information and resources to aid in Michael’s care, she had asked many different people. Importantly, she did considerable research on her own, primarily using the Internet. She felt that many, or perhaps most professionals did not know more than she did, regardless of their formal education and training. She gave the example of having told one of Michael’s doctors about Euro-Pēds, a facility specializing in physical therapy for children with cerebral palsy and other neuromuscular disorders. The doctor did not know about this resource. Amy also related how shocked she was when a receptionist at a local mental health facility was not aware of a “respite” fund provided by the facility. She expressed that it was “disheartening that these people are supposed to guide me, and they just couldn’t.” Then she went on to describe a situation in which parents of children with special needs could not obtain the service they wanted because they did not use the technical term:
I was told that there were even situations where people who aren’t articulate would call and say, ‘I need a
babysitter.’ And they say, ‘We don’t do babysitting services.’ Click. Because they didn’t say ‘respite,’ they were turned away…. It’s their job to be in tune with, maybe there’s something I’m not getting here. Let me figure out what’s wrong with this person that’s calling my mental health facility.
Amy was often disappointed in seeking resources and help, probably because of the lack of understanding and education among professionals.
Amy lamented that resources external to the family should not cause more stress because parents of children with special needs already are overwhelmed by feelings of guilt, helplessness and stress. She believed that Michael was not the cause of her issues, but rather that the actual problems were the by-products of his having a disability:
It’s not always directly related to the child, but all the side effects that how they affect you… A lot of it is just the overwhelming feeling that sometimes you wake up in the morning and say, ‘I can’t believe that he has so many problems.’ And you feel sorry for him, and you feel stressed out about it.
Amy also felt guilty about not being able to spend as much time as she would have liked with her other two children; the demands of Michael’s situation dominated all her plans. Amy had tried to be with her other children whenever she could, but still felt that she was not doing enough for them. Thus, she believed that Michael’s disability affected not only her, but also everyone else in the family. Amy also felt tremendous pressure when talking to Michael’s doctors:
Michael’s doctors say, ‘We don’t know if he can ever walk. But we don’t know if he won’t. It’s gonna be up to you, Mom. It’s gonna be, if he’s got the potential to do it. You’re the one that’s gonna push him…’It’s a lot of pressure and I don’t think that these doctors meant to give me that unneeded pressure… But I work very hard to push Michael, you know, everyday. But it scares me. It scares me that, ‘Am I pushing him enough? Am I pushing him too hard?’
Obviously Amy (and other parents of children with special needs like her) suffers from high levels of stress from both internal and external factors. To Amy, taking care of Michael was like “not knowing how to swim and you get thrown into a pool with another person who doesn’t know how to swim.” When Michael was born, Amy had to teach herself how to raise a child with special needs because “these children don’t come with an instruction manual…or a book of resources.” She believed that knowledge about Michael’s disability would be particularly important in order for her to take care of him properly and effectively. She also was aware that the process of accepting her son’s disability and learning how to take care of a child with special needs could be “a nightmare for some people,” because “even someone with formal medical training struggles with these children.” Amy related that she thought a support group to provide opportunities for the parents of children with special needs to discuss and share experiences and feelings would be beneficial. She also believed that inviting a professional such as a social worker to the group who could help the parents fill out paperwork for requesting funds and other assistance would be beneficial because many parents of children with special needs struggle with understanding and completing formal documents properly. At the end of the interview, Amy indicated that she felt like she was contributing at least in a small way to improving the lives of parents of children with special needs by participating in the research and that the interview was helpful in reducing her stress.
Discussion
This preliminary research was conducted to gather data, collect descriptive personal information, and, from the data, suggest future practices for gaining understanding of the unique needs of parents of children with special needs. Suggested in the results of this exploratory study, is that counseling services for parents of children with special needs are both warranted and needed. The format of such services likely should be group counseling because of lower cost and potential for mutual support among group members. Such group counseling sessions should be in part psychoeducational and in part intended to foster support to meet the goals of knowledge and skill acquisition for parenting children with special needs and sharing personal experiences with others. Individual and/or family counseling might be used as a follow-up service, especially for parents or families of children with special needs who appear to need intensive care. Finally, parents of children with special needs should be able to choose how they would like to interact, such as by phone, home visit, or face-to-face because they often struggle with finding child care for when they are away from home. Having support group meetings at each other’s homes also can be an option so that parents can take turns watching children during meetings.
Limitations of this study included a small number of male participants. Whether more responses from fathers would have changed the results is only a matter of speculation. Thus, future research that includes significantly more input from fathers of children with special needs is needed. Also, to be noted is that some participants reported confusion about terms such as psychoeducation, which may have influenced their responses. Therefore, future research should identify specific services rather than the categories of services. Any online survey is limited to those who have access to the Internet and are comfortable using computers. Future studies can overcome this limitation to a great extent by incorporating multiple methods involving several types of data collection. Finally, the case interview was perhaps the most valuable part of the study in terms of revealing the reality and challenges faced by parents of children with special needs. Thus, qualitative, phenomenological research also would be beneficial, especially for understanding the unique and complex concerns of parents of children with special needs.
References
Barnett, D., Clements, M., Kaplan-Estrin, M., & Fialka, J. (2003). Building new dreams: Supporting parents’ adaptation to their child with special needs. Infants and Young Children, 16, 184–200.
Ergüner-Tekinalp, B., & Akkök, F. (2004). The effects of a coping skills training program on the coping skills, hopelessness, and stress levels of mothers of children with autism. International Journal for the Advancement of Counselling, 26, 257–269.
Goddard, J. A., Lehr, R., & Lapadat, J. C. (2000). Parents of children with disabilities: Telling a different story. Canadian Journal of Counselling, 34, 273–289.
Heiman, T. (2002). Parents of children with disabilities: Resilience, coping, and future expectations. Journal of Developmental and Physical Disabilities, 14, 159–171.
Looman, W. S., O’Conner-Von, S. K., Ferski, G. J., & Hildenbrand, D. A. (2009). Financial and employment problems in families of children with special health care needs: Implications for research and practice. Journal of Pediatric Health Care, 23, 117–125.
Chiharu Hensley, NCC, is a professional counselor in Nagasaki, Japan. The author thanks Dr. Devika Dibya Choudhuri for generously and patiently guiding me through the entire process of the current study while I was a Master’s counseling student at Eastern Michigan University, and those who willingly participated in the study. Correspondence can be addressed to Chiharu Hensley, 9-1-403 Manabino 2-chome, Nagayo-cho, Nishisonogi County, Nagasaki, Japan, 851-2130, chiharu.hensley@gmail.com.
Sep 2, 2014 | Article, Volume 1 - Issue 3
Sachin Jain, Santiago Silva
One of the major flaws in current psychological tests is the belief that a prediction/diagnosis can be made that would tell an individual whether he is heterosexual, homosexual or bisexual. What is needed within the profession, however, is an assessment that has the sensitivity to help clients explore their sexual orientation. A pilot 100-item Sexual Orientation Scale was developed after interviewing 30 self-identified gay men who considered themselves happy/satisfied. The items summarized the thoughts and feelings of these 30 men during the discovery process and ultimate acceptance of their sexual orientation. The scale was then completed by 208 male participants. The Cronbach’s Alpha Coefficient was calculated for the initial 100-item version of the Sexual Orientation Scale along with item analysis and factor analysis. These statistical manipulations were computed to help eliminate items that did not discriminate well. The final version of the Sexual Orientation contains 43 items. Implications for the use of this scale and future directions in research are further explored.
Keywords: sexual orientation, scale development, males, assessment, exploration
As children grow up in our society, they are introduced to a wide range of knowledge about sexual behavior by their parents, siblings, and peers. Part of their education addresses the ideas of sexual orientation and/or preference. The inherent messages in this education are that a person is either heterosexual (sexually attracted to members of the opposite sex), homosexual (sexually attracted to members of the same sex), or bisexual (sexually attracted to members of both sexes).
Historical Overview of Sexual Orientation
A number of theories on the origin of homosexuality have attempted to define homosexuality. A number of these theories (c.f. Drescher, 2008; Ellis, 1936; Freud, 1922/2010; Krafft-Ebing, 1887/1965; Nuttbrock et al., 2009) place sexual orientation within the context of an individual’s overall sex role identity. These individuals link sexual attraction for men toward women with a masculine sex role orientation and sexual orientation toward men with a feminine sex role orientation (Axam & Zalesne, 1999; Mata, Ghavami & Wittig, 2010; Storms, 1980). Sexual orientation refers to a particular lifestyle (behavior) that an individual displays. Storms (1980) and Moradi, Mohr, Worthington, & Fassinger (2009) found most theories about the nature of sexual orientation emphasize either the person’s sex role orientation or erotic orientation. Although these assumptions have had a major impact on the development of theories, research, clinical practice, and even popular stereotypes, neither assumption has been adequately tested in past research. A homosexual person is one defined as having preferential erotic attraction to members of the same sex and usually (but not necessarily) engaging in overt sexual relations with them (Crooks & Baur, 2008; Marmor, 1980).
Cass (1984) and Harper & Harris (2010) identified four steps in the discovery process that people experience as they begin to identify their sexual orientation:
1. Individuals come to perceive themselves as a homosexual by adopting a self-image of what it means to be homosexual.
2. Individuals take this self-image a step further and allow it, through interaction with others, to become a homosexual identity.
3. Individuals assume the necessary affective, cognitive, and behavioral strategies in order to effectively manage this identity in everyday life.
4. Individuals find a way with which to incorporate the new identity into an overall sense of self.
Assessment of Sexual Orientation
Fergusson & Horwood (2005) wrote a review of the multitude of methods that have been used to assess sexual orientation. Conceptualization of sexual orientation as dichotomous (i.e., heterosexual and homosexual) was overturned over 60 years ago by Kinsey, Pomeroy, and Martin (1948) and by Kinsey, Pomeroy, Martin, and Gebhard (1953). These studies resulted in the development of a 7-point scale in which 0 represented exclusive heterosexuality and 6 represented exclusive homosexuality. Three on the scale indicated equal homosexual and heterosexual responsiveness. Individuals were rated on this continuum based upon their sexual behavior and physical reactions (i.e., physical attraction to desired partners) (Coleman, 1987; Fergusson & Horwood, 2005).
Although this notion that people fall in a continuum better represented the realities of the world (Bagley & Tremblay, 1997; Silenzio, Pena, Duberstein, Cerel, & Knox, 2007), the Kinsey Scale has many limitations for accurately describing an individual’s sexual orientation. The scale assumes that sexual behavior and erotic responsiveness are the same within individuals. In response to this criticism, Bell and Weinberg (1978) used two scales in their extensive study of homosexuality. They examined two scales: one for sexual behavior, and one for erotic fantasies. Bell & Weinberg (1978) found discrepancies between the two ratings. Paul (1984) and Garnets & Kimmel (2003) also reported discrepancies in approximately one-third of their homosexual samples. It was reported that most men saw their behavior as more exclusively homosexual than their erotic feelings (Coleman, 1987; Fergusson & Horwood, 2005; Schwartz, Kim, Kolundzija, Rieger & Sanders, 2008).
Coleman (1987) and Fergusson & Horwood (2005) suggested that while this dichotomous and continuous view of sexual orientation represented an improvement in assessment of sexual orientation, several clinicians and researchers have recommended additional dimensions (Fox, 2003). These dimensions are those based upon both the biological sex of the partner and the biological dichotomous sex of the individual.
As the literature on psychological testing and homosexuality unfolded, it became clear that tests were not very effective in creating special scales, signs or scoring patterns that could differentiate homosexuals from heterosexuals (Garnets & Kimmel, 2003; Paul, Weinrich, Gonsiorek, & Hotvedt, 1982). Homosexuality was no longer being studied as an illness. Contrastingly, literature has brought forth strong data that dismiss the notion that homosexuality is a disorder (Cass, 1984; Coleman, 1982; Harper & Harris, 2010; Henchen & O’Dowd, 1977; Morin & Miller, 1974; Tripp, 1975; Troiden, 1977; Weinberg, 1978).
One of the major flaws in current psychological tests is that there is a belief that a prediction/diagnosis can be made that would tell an individual whether he is heterosexual, homosexual, or bisexual. It is the authors’ belief that it is inappropriate to predict what kind of lifestyle an individual will/should follow. What is more feasible is to assist an individual as he or she explores the experience of uncertainty. Therefore, an instrument is needed that has the sensitivity to help clients explore their sexual orientation, not one that identifies levels of disturbance.
Purpose of the Study
The purpose of the study was to construct an instrument that would help counselors in assisting clients who wish to explore sexual orientation. The instrument was to:
• Identify issues that need to be addressed by the client during the discovery of sexual orientation.
• Focus on issues such as self-definition, self-acceptance, fears, sexual fantasies, and understanding of lifestyle.
• Provide an information base for counselors as they help their clients unfold significant characteristics of their personality.
• Provide counselors a tool for helping clients meet the challenges they face now and will face in the future.
Method
Participants
The volunteer population of this study consisted of males who were either a) receiving personal counseling at a university counseling service, community mental health agency, and/or private practice; b) enrolled in introductory psychology classes at universities or community colleges; or c) participating in local men’s groups (i.e., Jaycees, Lions Clubs, support groups, etc.).
Two universities in California, eight universities in Texas, and one university in Wisconsin assisted in the collection of data. Three mental health agencies and four private counseling centers also were recruited for assistance in data collection. The private counseling centers served primarily gay and lesbian clients from the Dallas/Ft. Worth area.
Directors and/or counselors at the mental health sites mentioned above were visited. The purpose of the study was explained and they were asked if they would approach their clients (straight and gay) to determine their willingness to participate in the study. If the counselors were willing to speak to their clients, they were given instructions to share with clients who agreed to participate. They were instructed to give the client the research packet and return the completed information in the enclosed addressed/stamped envelope. Seventy-five agreed and completed packets from this group of mental health agencies were obtained.
Permission from psychology professors at the universities and/or colleges to address their introductory psychology classes was obtained for recruiting more subjects. The purpose of the study was shared with the class, willing participants were moved to another classroom, and they completed the research packet. One hundred and six packets were completed through this procedure.
Men’s groups were approached to obtain additional participants. Groups such as Jaycees, Lions Clubs, and Gay Men’s Support Groups were contacted and visited. A presentation was made that addressed the purpose of the study. Willing participants were provided with information packets, which they returned in enclosed envelopes. Thirty-three completed packets from representatives of the men’s groups were received. Twenty-eight of the 33 came from gay men’s support groups.
Demographic information from the personal data form was summarized and examined across the variable of sexual orientation on the following factors: educational level, socio-economic status, age, ethnicity, self-rating on the Kinsey Sexuality Scale and whether or not the participant was currently in counseling or psychotherapy. The males in the sample identified themselves as being either homosexual (gay) or heterosexual (straight). The males self-identified as gay or straight by rating themselves on the seven-point Kinsey Sexuality Scale (0=exclusively heterosexual to 6=exclusively homosexual). Straight responses were identified as those of which the men rated themselves as zero (0) or one (1) and gay responses were identified as those in which the men rated themselves as five (5) or six (6) on the Kinsey scale. Only six subjects rated themselves as 2, 3 and 4. The scales completed by these 6 subjects were not used in this study.
The sample consisted of a total of 208 men from cities in Texas, Wisconsin, and California: 132 were between the ages of 18–25 (63.5%); 52 were between the ages of 26–33 (25.0%); and 24 were between the ages of 34–40 (11.5%). According to the Kinsey Scale Rating, 104 were straight (50%) and 104 were gay (50%). Of the men who participated in the sample, 85 (40.9%) had received counseling and 123 (59.1%) had not.
Procedure
The first procedure consisted of the development of the items for the Sexual Orientation Scale. In order to achieve this task, thirty gay men who described themselves as being happy/satisfied with the gay lifestyle were interviewed. The men were identified via personal contacts and gay organizations. Their input was used to develop items for the Sexual Orientation Scale.
Three small group meetings of approximately two hours each with about ten men were scheduled. Each meeting began with a statement of the purpose of the groups and the study. It was explained that data was being collected to formulate a scale that would help people clarify questions about their sexual orientation. It was explained that the scale was not designed to label whether someone was gay or straight, but simply to identify issues surrounding sexual orientation. Time was allotted for questions and answers.
Participants were asked for permission to record the group session. When permission was obtained, participants were asked about their experience of the discovery process of their sexual orientation (e.g., “What struggles did you experience?” “What questions did you ask yourself during this discovery process?” “What were you feeling?” “Did you get in touch with any fears?” “What kind of sexual fantasies did you experience?”). These questions were asked in order to help the participant recall their discovery process. Participants were allowed to ask each other questions and/or identify with what was being shared in a casual and informal atmosphere. Recordings of the three small group meetings provided the source for the 100 items that represented thoughts and feelings the men experienced during their discovery process. These 100 items consisted of Phase 1 of the Sexual Orientation Scale development.
After the pilot scale was developed, packets were sent out to university counseling services, psychotherapists in private practice, and community mental health agencies. The packet consisted of: (a) a personal data form, (b) the 100-item Sexual Orientation Scale, (c) an informed consent form, and (d) an addressed and stamped envelope. Data on the 100-item Sexual Orientation Scale also was collected from different men’s groups and from the introductory psychology classes both at universities and community colleges.
Two hundred and eight packets were completed. Coincidently, 104 responses were from gay individuals, and 104 were straight responses. The responses were then transferred onto Scantrons and submitted for analysis.
Instrument Development
Item construction. Tests are composed of a number of items that are used to measure a particular subject. According to Wesman (1971), an item may be defined as a scoring unit. Creating an item should be taken seriously because each item in a test produces a unit of information regarding the person who takes the test.
Writing a test item is an involved process. Test items need to be subjected to constant evaluation in order to ensure, as much as possible, that they are measuring what they are intended to measure. The items developed for the Sexual Orientation Scale represent two variables: self-image and eroticism. These variables have been continuously identified in sexual-orientation literature (Cass, 1984; Coleman, 1982; Eliason & Schope, 2007; Grace, 1979) as variables that must be examined when attempting to answer questions regarding sexual orientation. Self-image is defined as involving self-definition, self-acceptance, fears and an understanding of lifestyle. Eroticism is defined as sexual fantasies.
Item analysis. According to Anastasi (1988), items on an instrument may be analyzed quantitatively, in regards to their statistical properties. When examining items qualitatively, content validity is considered as well as the evaluation of items in terms of effective item-writing procedures. Quantitative analysis primarily includes the measure of item difficulty and item discrimination (Anastasi, 1988).
Item difficulty answers the question: How hard or easy was a particular item for the group of participants? Item discrimination refers to the degree to which an item differentiates correctly among test takers in that behavior that the test is designed to measure (Anastasi, 1988, p. 210). Item discrimination was calculated as a correlation coefficient between the item score and the total score. Correlation coefficient indicates the strength and direction of a linear relationship between two random variables.
Results and Discussion
Item Design for the Sexual Orientation Scale
In designing the Sexual Orientation Scale, two areas of interests were salient. They were self-images and eroticism. The literature on sexual identity formation strongly supported the examination of these two interest areas during the discovery process of one’s sexual orientation. The importance of examining self-images and eroticism was further supported in the early stages of this study that resulted in the identification of the initial 100 items of the Sexual Orientation Scale.
Thirty self-identified gay men were interviewed regarding their discovery process. While reviewing interviews, items were generated that represented their thoughts and feelings. Examination of items clearly indicated that issues such as self-acceptance, understanding fears, and eroticism were being confronted during the discovery process.
Next, these 100 items were then subjected to an item analysis that resulted in identifying 45 items with item discrimination indices of 0.50 or higher. These 45 items were then further subjected to a factor analysis and an alpha coefficient.
An arbitrary decision was then made to use a 0.5 or higher factor loading in examining items. A strict convention of 0.5 or higher was used in order to identify the most discriminating items. The factor analysis identified the same items the item analysis identified. The alpha coefficients were as follows: overall= 0.924; straights= 0.723, gays= 0.653.
The factor analysis also identified four factors that were consistent with issues identified by both the literature and the initial 100 items. After reviewing the items in these factors, they were labeled as:
attraction to same sex
attraction to opposite sex
self-acceptance of gay behavior/attitudes
fears
Item Analysis
Anastasi (1988) pointed out that items on an instrument may be analyzed qualitatively, in terms of their content and form, and quantitatively, in terms of their statistical properties. The item analysis performed on the initial 100 items of the Sexual Orientation Scale focused on a quantitative analysis and more specifically on the measures of item difficulty and item discrimination.
Item difficulty refers to the percentage of subjects that endorse certain items on the scale. The closer the difficulty level approaches 0.50, the more differentiations the items can make (Anastasi, 1988).
Item discrimination refers to how effective the item discriminates between the two groups. Therefore, the higher the item discrimination score, the more effectively the item will differentiate between the two groups (gays/straights). Table 1 shows how the sample was grouped in order to establish an item-to-total score correlation, which is identified as a useful exercise to select items.
Based on the total score, the respondents were divided into quintiles (groups of approximately 40 subjects). A total score was established by assigning a value of 1 to true responses and a value of 2 to false responses. A true response indicated the way a gay man would respond. An item difficulty, identified in the item analysis as proportion of subjects that responded correctly to the item (PROP) and item discrimination, identified in the item analysis as a point biserial correlation coefficient (RPBI), were calculated for each item.
Table 2 outlines the item difficulty and item discrimination score and the scoring key of the initial 100 items. The asterisks identify the scores for the 43 items on the final version of the Sexual Orientation Scale. Of the final 43 items, approximately 76.7% of the items have a difficulty score that range from 0.40 to 0.60. Since Anastasi (1988) stated that the closer the difficulty level approaches 0.50, the more differentiations the item can make, it is safe to infer that the majority of the items on the Sexual Orientation Scale possessed good potential for differentiating between responses of the two groups. The remaining 23.3% of the items were not far behind. None of the item difficulty scores were less than 0.32 or higher than 0.76. This shows that the result of these items do not differentiate as well, but well enough to contribute to the overall reliability and validity of the Sexual Orientation Scale.
According to Anastasi (1988), the items that have low correlations with total score should be deleted and the items with the highest average inter-correlations will be retained. These items were retained because they are the ones that discriminate well and increase the validity of the test.
Analysis showed that 43 items on the initial 100-item Sexual Orientation Scale scored 0.50 or higher on the item discrimination index. Since the item discrimination refers to the degree to which an item differentiates correctly among test takers in the behavior that the test is designed to measure, one could assume the majority of the items on the Sexual Orientation Scale effectively differentiates between the two groups (gay/straight) that were tested.
Results of Item Analysis
The 45-item version of the Sexual Orientation Scale was a result of an item analysis done on the initial 100-item scale. The original data analysis identified 17 factors. An item discrimination index of 0.50 or higher was used to identify the items for the final version of the scale. Items that exhibited a higher level of commonality were selected. The 55 items that were deleted did not discriminate as well.
The 45 items were then submitted to the following statistical procedures: (a) Cronbach’s alpha coefficients were calculated for the overall sample, for the straight sample, and for the gay sample; and (b) a factor analysis was conducted via the running of five, four, three and two factor solutions on the overall sample (N=208). The factor analysis was done for the purpose of further validating the Sexual Orientation Scale.
Naming of the Factors
After creating and reviewing a SCREE Plot with the Eigen values of the 45 items, the researchers identified a bend that began to occur around the three, four and five factors. All factor solutions were investigated, and a decision was made to use the four-factors solution because (a) the items fit the four factors very well, and (b) the addition of a fifth factor accounted for negligent increase in the total variance. Every item in each factor carried a common theme.
The items in the four-factor solution were reviewed. Finally, two of the 45 items did not have a factor loading of 0.5 or higher. In keeping with the arbitrary decision of only using those items with a 0.5 factor loading or higher (for the purpose of implementing a stricter convention), items 36 and 15 were deleted. The final version of Sexual Orientation Scale resulted in having 43 items.
Table 3 summarizes the four factors solution by identifying the sorted rotated factor loadings of each item in each factor. Items 12, 22, 29, 39, 42, 45, 46, 49, 50, 59, 64, 65, 66, 67, 68, 72, 73, 80, 83, 88, 97, 98, and 99 loaded on Factor One with factor loadings ranging from 0.58 to 0.73. The common theme was sexual attraction to members of the same sex. The items in Factor One identified issues such as being attracted to nude males, erotic thoughts about men, masturbatory fantasies involving men only, relationships with males, etc. Therefore, Factor One was named “Attraction to Same Sex.”
Items 6, 14, 17, 41, 44, 48, 60, 63, 70, and 82 of Factor Two in Table 3 also had sexuality as their common theme. However, the sexual attraction addressed in the above items was towards members of the opposite sex. Their factor loadings ranged from 0.55 to 0.78. The items in Factor Two brought to surface issues dealing with erotic fantasies about women only, thoughts about women that led to sexual arousal, etc. Due to a common theme in these items, Factor Two was named “Attraction to Opposite Sex.”
Factor Three in Table 3 was comprised of items 16, 25, 33, 56, 89, and 91. These items had factor loadings ranging from 0.53 to 0.78. In examining these items, it was evident that the common theme surrounding the items was that of self-image and self-concept. The items in Factor Three addressed issues such as self-expression, expression of affection to another male, the acknowledgement of individual differences and the normalcy of being attracted to other men. Factor Three was named “Self-Acceptance of Gay Behaviors/Attitudes.”
Items 31, 32, 81, and 90 in Table 3 loaded onto Factor Four with loadings that ranged from 0.52 to 0.60. The common theme among these items was fear, which pertained to issues faced more often than not by gay men. The items addressed concerns in areas such as wanting to be sexually active with other men, jealousy, noticeable reactions to other men and fear of being gay. Because of the obvious common theme, Factor Four was named “Fears.”
It is vital to note that the four factors identified via the factor analysis represented those themes continually found in sexual orientation literature. Cass (1979 & 1984), Grace (1979) and Coleman (1982) consistently addressed the importance of examining the variables of self-images and eroticism during the discovery process of one’s sexual orientation. All these factors are clearly identified in the 43 items in the four-factor solution done on the final version of the Sexual Orientation Scale.
Reliability
A Cronbach’s alpha coefficient was performed in order to establish the reliability of the final 43-item version of the Sexual Orientation Scale. An alpha coefficient was done on the overall sample (N=205), the straight sample (N=103), and the gay sample (N=102). The overall sample has an N of 208. Three completed scales (1 straight respondent and 2 gay respondents) were eliminated because they did not complete the initial 100 items. The alphas for the 43-item version were 0.93 for the overall sample, 0.72 for the straight sample and 0.65 for the gay sample.
Construct-Related Validity
Internal consistency is a procedure used to establish construct validity. A statistical procedure used in this study to establish internal consistency was Cronbach’s Alpha Coefficient; this statistic also was used to establish instrument reliability (Miller, 1987). Table 4 shows the alphas which clearly exhibit the homogeneity for the items on the Sexual Orientation Scale.
A factor analysis was performed on the 45 items to identify the prevalent factors. After the factor analysis was done, four factors were identified as the most important factors that need to be examined when struggling with the uncovering discovery process of an individual’s sexual orientation. They are Attraction to Same Sex, Attraction to Opposite Sex, Self-Acceptance of Gay Behavior/Attitudes, and Fears. The items and other data on each factor are summarized in the following table. Normative data also was generated on the overall sample, the gay sample, and the straight sample. This was done for interpretation purposes. Table 4 summarizes the established normative data.
Limitations
This study contained methodological features which resulted in limitations. The major areas of limitations were (a) the sampling procedures and (b) generalizability.
Sampling Procedures
The initial 100-item Sexual Orientation Scale resulted from interviews with self-identified gay men who stated they were happy/satisfied as gay men. Only 30 men were interviewed. Although this number was an acceptable number, a larger number of men interviewed may have provided additional insights.
Of this group of 30 men (participating in the development of the items for the Sexual Orientation Scale) from cities in the Rio Grande Valley (RGV) in South Texas, 90% were Hispanic college graduates. Being a Hispanic gay man in the RGV in South Texas is difficult. The machismo attitude is somewhat prevalent in this area. This, coupled with a strong Catholic belief about homosexuality makes life as a gay man very secretive in this area. Thus, it is important to note that the initial 30 subjects were men who are openly gay, educated, motivated, and obvious risk takers. The sample group, therefore, may not have represented the “typical” gay man in the United States. Moreover, a different or more thorough perspective about what is involved in the discovery process with respect to sexual orientation might have risen if there had been a more diverse group of gay men in terms of ethnicity and geographical area.
The sample size (N = 208) utilized for the statistical item analysis was small. Although acceptable for this study, a much larger sample would likely improve the scale’s reliability and validity. The sample in this study did not include women. Women were excluded because it was suspected that gay men and lesbian women experience a different discovery process and that a parallel, yet different study is necessary for females.
Lastly, the two samples (gay/straight) are not actually directly comparable because, in essence, they were not selected in the same way. For example, a large percentage (65.4%) of the gay sample compared to 16.3% of the straight sample was enrolled in counseling. One can ascertain that most of the gay samples were selected from university counseling centers, mental health agencies, and the private sector. Contrastingly, the straight sample was selected from introductory psychology classes and from the membership of men’s groups (civic and/or support).
Generalizability
The generalizability of the results of this study is limited to men who are between the ages of 18 and 40 and who are either receiving counseling services from university counseling centers, mental health agencies, or the private sector, or who are in introductory psychology classes or members of men’s groups. The generalizability of this study is further limited to Hispanic and Caucasian men who met the research criteria.
Recommendations
The following are recommendations to either improve the present study’s design or identify areas for future research:
• Obtain a more culturally diverse sample by including representatives of other ethnic groups along with representatives from the Hispanic and Caucasian groups. This would increase the potential of gathering different perspectives and insights as well as increase the generalizability of the results.
• Utilize a larger more diverse sample in order to compare the reliability and validity data obtained in this study with other studies. A test re-test might be considered so as to verify the reliability of the Sexual Orientation Scale.
• In order to minimize a client’s tendency to answer the way they think their therapist or counselor wants them to, a lie/consistency scale may need to be established for the Sexual Orientation Scale. This may be done by including items that emphasize the same information, but written in a different manner.
Once the Sexual Orientation Scale has undergone further empirical investigation and eventual modifications, the use of the scale in counselor training programs should be considered. This would be done in hopes of (a) educating future counselors in how to assist clients who are confused about their sexual orientation, (b) increasing one’s sensitivity to and knowledge about gay/lesbian issues, and (c) requiring to some extent that future counselors accept and understand their own biases in regards to individual differences and more specifically to gays and lesbian.
References
Anastasi, A. (1988). Psychological testing. New York, NY: Macmillan.
Axam, H. S., & Zalesne, D (1999). Simulated sodomy and other forms of heterosexual horseplay: Same sex sexual harassment, workplace gender hierarchies, and the myth of the gender monolith before and after oncale. Yale Journal of Law and Feminism, 11, 155–253.
Bagley, C., & Tremblay, P. (1997). Suicidal behaviors in homosexual and bisexual males. Crisis, 18(1), 24–34.
Bell A. P., & Weinberg, M. S. (1978). Homosexualities: A study of diversity among men & women. New York, NY: Simon and Schuster.
Cass, V. C. (1979). Homosexuality identity formation: A theoretical model. Journal of Homosexuality, 4, 219–234.
Cass, V. C. (1984). Homosexuality identity formation: Testing a theoretical model. The Journal of Sex Research, 20, 143–167.
Coleman, E. (1982). Homosexuality & psychotherapy: Developmental stages of the coming-out crocess. Journal of Homosexuality, 7, 36–48.
Coleman, E. (1987). Assessment of sexual orientation. Boca Raton, FL: Haworth.
Crooks, R., & Baur, K. (2008). Our sexuality. Belmont, CA: Wadsworth.
Drescher, J. (2008). A history of homosexuality and organized psychoanalysis. Journal of the American Academy of Psychoanalysis and Dynamic Psychiatry, 36(3), 443–460.
Ellis, H. (1936). Studies in the psychology of sex. New York, NY: Random House.
Eliason, M. J., & Schope, R. (2007). The health of sexual minorities: Shifting sands or solid foundation? Lesbian, gay, bisexual, and transgender identity formation. New York, NY: Springer.
Fergusson, D. M., & Horwood, L. J. (2005). Sexual orientation and mental health in a birth cohort of young adults. Psychological Medicine, 35(7), 971–981.
Fox, R. C. (2003). Bisexual Identities. In L. Garnets & D. Kimmel (Eds.), Psychological perspectives on lesbian, gay, and bisexual experiences (pp. 86–129). New York, NY: Columbia University Press.
Freud, S. (1922/2010). Group psychology and the analysis of the ego. (J. Strachey, trans.). Charleston, S.C: Nabu Press.
Garnets, L., & Kimmel, D. C. (2003). Psychological perspectives on lesbian, gay, and bisexual experiences. New York, NY: Columbia University Press.
Grace, J. (1979, November). Coming out alive. Paper presented at the Sixth Biennial Professional Symposium of the National Association of Social Workers. San Antonio, Tx.
Harper, S. R, & Harris, F. (2010). College men and masculinities: Theory, research, and implications for practice. Hoboken, NJ: John Wiley.
Henchen, J. D., & O’Dowd, W. T. (1977). Coming out as an aspect of identity formation. Gay Academic Union Journal: Gai Saber, 1, 18–22.
Kinsey, A. C., Pomeroy, W. B., & Martin, C. E. (1948). Sexual behavior in the human male. Philadelphia, PA: W. B. Saunders.
Kinsey, A. C., Pomeroy, W. B., Martin, C. E., & Gebhard, P. H. (1953). Sexual behavior in the human female. Philadelphia, PA: W. B. Saunders.
Krafft-Ebing, R. V. (1887/1965). Psychopathia sexualis. (H. E. Wedick, Trans.). New York, NY: Putnam.
Marmor, J. (1980). Homosexual behavior. New York, NY: Basic.
Mata, J., Ghavami, N., & Wittig, M. A. (2010). Understanding gender differences in early adolescents’ sexual prejudice. The Journal of Early Adolescence 30(1), 50–75.
Miller, D. (1987). Strategy making and structure: Analysis and implications for performance. Academy of Management Journal, 30, 7–32.
Moradi, B., Mohr, J. J., Worthington, R. L., & Fassinger, R. E. (2009). Counseling psychology research on sexual (orientation) minority issues:Conceptual and methodological challenges and opportunities. Journal of Counseling Psychology, 56(1), 5–22.
Morin, S. F., & Miller, J. S. (1974, March). On fostering positive identity in gay men: Some developmental issues. Paper presented at the meeting of the American Orthropsychiatric Association, San Francisco, CA.
Nichols, P., & Fulkerson, D. (2010). Informing design patterns using research on item writing expertise. Menlo Park, CA: SRI International.
Nuttbrock, L., Bockting, W., Mason, M., Hwahng, S., Rosenblum, A., Macri, M., & Becker, J. (2009). A further assessment of Blanchard’s typology of homosexual versus non-homosexual or autogynephilic gender dysphoria. Archives of Sexual Behavior,40(2), 247–257.
Paul, W., Weinrich, J. D., Gonsiorek, J. C., & Hotvedt, M. E. (1982). Homosexuality: Social, psychological and biological issues. Beverly Hills, CA: Sage.
Paul, J. P. (1984). The bisexual identity: An idea without social recognition. Journal of Homosexuality, 9(2/3), 65–77.
Silenzio, V. M., Pena, J. B., Duberstein, P. R., Cerel, J., & Knox, K. L. (2007). Sexual orientation and risk factors for suicidal ideation and suicide attempts among adolescents and young adults.American Journal of Public Health 97(11), 2017–2019.
Storms, M. D. (1980). Theories of sexual orientation. Journal of Personality and Social Psychology, 38, 783–792.
Tripp, C. A. (1975). Homosexual matrix. New York: McGraw-Hill.
Troiden, R. R. (1977). Becoming a homosexual: Research on acquiring a homosexual identity. Unpublished manuscript, State University of New York at Stony Brook, Stony Brook, New York.
Weinberg, T. (1978). On “doing” and “being” gay: Sexual behavior and homosexual male self-identity. Journal of Homosexuality, 4, 143–156.
Wesman, A. G. (1971). Writing the test item. In R.L. Thorndike (Ed.), Educational measurement. Washington, DC: American Council on Education.
Sachin Jain, NCC, is an Assistant Professor, Department of Counseling, Oakland University, Rochester, MI. Santiago Silva, is a Clinical Professor & Director of the Counseling Assessment Preparation Clinic (CAP) at the University of Texas-Pan American. Correspondence can be addressed to Sachin Jain, Oakland University, Department of Counseling, 2200 N Squirrel Road, Rochester, MI, 48309, sacedu@yahoo.com.
Sep 2, 2014 | Article, Volume 2 - Issue 1
Robert C. Reardon, Stephen J. Leierer, Donghyuck Lee
A six-year retrospective study of a university career course evaluated the effect of four different class schedule formats on students’ earned grades, expected grades and evaluations of teaching. Some formats exhibited significant differences in earned and expected grades, but significant differences were not observed in student evaluations of instruction. Career services providers, including curriculum designers, administrators and instructors, will find the results of this study helpful in the delivery of services, especially with high-risk freshman students.
Keywords: career, teaching, course, instruction, evaluation, grades
While individual counseling has been shown to be effective in helping students develop career decision-making skills (Brown & Ryan Krane, 2000; Reese & Miller, 2006; Whiston & Oliver, 2005; Whiston, Sexton, & Lasoff, 1998), undergraduate career courses also can be effective interventions (Folsom & Reardon, 2003; Reardon, Folsom, Lee, & Clark, 2011; Whiston et al., 1998).
Although college career courses have been shown to offer substantial benefits (Brown & Ryan Krane, 2000; Osborn, Howard, & Leierer, 2007; Reed, Reardon, Lenz, & Leierer, 2001; Reese & Miller, 2006; Whiston & Oliver, 2005; Whiston et al., 1998), the content and format of such courses vary greatly (Folsom & Reardon, 2003). The present study sought to focus on one aspect of such career course variability: alternative class schedule formats.
Effective career classes can be characterized by these features: (a) structured course approaches appear to be more effective than unstructured approaches (Smith, 1981); (b) individual career exploration should be a cornerstone of the course (Blustein, 1989); and (c) five components (written exercises, individualized interpretations and feedback, in-session occupational exploration, modeling, and building support for choices within one’s social network) (Brown & Ryan Krane, 2000; Brown et al., 2003).
What is the effect that class schedule might have on course effectiveness? Only one study (Vernick, Reardon, & Sampson, 2004) has examined this issue, and the results showed that such courses should be designed to meet more than once a week and avoid over-exposure to materials and activities so as not to overwhelm the student. Extending this concept, we hypothesized that certain course schedule formats (weekly meeting frequency and term length) could make a difference in student learning and evaluation of teaching.
Alternative Career Class Schedules
This study focused on a course based on cognitive information processing theory incorporated into the course textbook, Career Planning and Development: A Comprehensive Approach (Reardon, Lenz, Sampson, & Peterson, 2000). All sections of the course followed a prescribed curriculum comprising a mixture of lectures, panel presentations, small and large group instructional activities, personal research, and field work; however, the classes differed in terms of the class meeting schedule (class duration, number of weekly meetings, and number of weeks a class met during an academic term).
We examined 57 course sections that met over a six-year period and were team-taught by lead instructors and co-instructors with an instructor/student ratio of about 1:8. Lead instructors included both professional staff and faculty who supervised the co-instructors. During the time of this study, four class schedule formats were used. In the case of a 16-week semester, the class met once per week for 3 hours; twice per week for 1.5 hours; or three times weekly for 1 hour. A fourth schedule option was for a 6-week term with the class meeting four times weekly for about 8 hours per week. In the 16-week semester, the class met once per week for 3 hours on Wednesdays (W); twice per week for 1.5 hours on either Monday/Wednesday or Tuesday/Thursday (MW/TuTh); or three times weekly for 1 hour on Monday, Wednesday, and Friday (MWF). A fourth schedule option was for a 6-week term where the class met four times weekly for about 8 hours per week on Monday, Tuesday, Wednesday, and Thursday (MTuWTh). In summary, we sought to evaluate the influence of these four class schedule formats upon the educational experience of the students as measured by expected grades, earned grades, and student evaluations of teaching.
Course Measures
The following section gives details about the three measures of student learning and perceptions of teaching used in this study.
Earned Grade (EG)
Although a student’s grade point average has limitations as a measure of academic achievement, class grades are nevertheless a widely accepted method of quantifying students’ level of educational achievement and future success in graduate school or employment (Plant, Ericsson, Hill, & Asberg, 2005). Specific to career development, Reardon, Leierer, and Lee (2007) showed that grades might be useful measures of career course interventions, “especially if the treatment variables are carefully described and the grading procedures are fully explained and replicable by other researchers” (p. 495). For this study, we assumed that a student’s final EG would accurately reflect learning in the course.
Expected Grade (XG)
Grade expectations are a complex phenomenon that combines realistic data-driven grade expectations with unjustified optimism or wishful thinking (Svanum & Bigatti, 2006). The XG reflects the student’s assessment of course demands and optimism about successfully meeting those demands. This grade prediction may be informed or uninformed; however, after completing multiple assignments over the course of the semester, Svanum and Bigatti (2006) noted that students lower the value of their XG such that it will be only moderately inflated and will reliably predict their final EG. Because students in our course had the course grading scale in the syllabus, a signed performance contract, and predicted their grades during the last week of the semester when 85% of their grade had already been accounted for, we hypothesized that in aggregate their predictions would be only moderately inflated and thus a reliable predictor of their earned grades and success in the course. We felt this grade variable was important as a measure of students’ confidence in their mastery of the career development subject matter and the problem-solving skills taught in the course, and therefore a valid measure of the relative effectiveness of different class schedule formats.
In addition, comparing EG and XG informs us about students’ self-evaluation of learning and their actual performance in the course. When there is not a significant difference between the two scores, we might suppose that students have a fairly accurate understanding of their performance on completed assignments and those still to be graded. By contrast, a significant difference between XG and EG indicates a discrepancy between students’ self-evaluations of graded and as-yet-ungraded assignments and the official final grades. If XG is significantly higher than EG in a section, one may conclude that the academic work has been undervalued by the instructor or overvalued by the students. Conversely, if XG is significantly lower than EG, one might conclude that students’ estimates were conservative or instructors recognized a level of performance not seen by the students.
Student Evaluation of Teaching (SET)
Student evaluation of classes and teaching effectiveness is standard practice at most postsecondary institutions. There is substantial anecdotal and experimental evidence supporting the usefulness of SETs (Centra, 1993; Marsh & Dunkin, 1992; Marsh & Roche, 1997). Certain student ratings forms provide important feedback that can be used to improve teaching performance (Greenwald & Gillmore, 1997; Marsh & Roche, 1997; McKeachie, 1997), and when asked most faculty members support the use of SETs as a tool for teaching improvement (Baxter, 1991; Griffin, 1999; Schmelkin, Spencer, & Gellman, 1997). Although SET is not without its critics, it appears to be a pragmatic way to access and compare student perceptions of teachers’ effectiveness and therefore a potential measure of the relative efficacy of different class schedules.
In an effort to better evaluate students’ course experiences, the influence of EG (Goldman, 1985) and XG (Greenwald & Gillmore, 1997) on SET is receiving considerable attention in the literature. The present study provided an opportunity to examine the relationship of SET to both EG and XG relative to four different class schedule formats.
Research Questions
In seeking to discover if particular class schedules were more effective in a team-taught career course, we evaluated grades and participant feedback from undergraduate students. The goal was to determine if any of the four differing class schedules produced significant differences in the course evaluation measures EG, XG, and SET. Although we were examining these measures from the students’ perspective and such measures are typically scored at the individual student level, we chose to examine class section level scores because XG and SET data were only available to us in this way.
The first group of research questions examined differences between mean evaluative measures, aggregated by class format and averaged for classes that met one (W), two (MW/TuTh), or three times per week (MWF) for 16 weeks, or four times per week (MTuWTh) for 6 weeks.
Research Question 1: Were there any significant differences in the career course evaluation measures among the four class formats?
RQ 1.1: Are there differences in mean EG between formats?
RQ 1.2: Are there differences in mean XG between formats?
RQ 1.3: Are there differences in mean SET between formats?
The second group of research questions explored the differences between the evaluation measures (EG, XG, and SET) within the sections.
Research Question 2: Within any given format, are there significant differences between the mean of the aggregated class evaluation measures?
RQ 2.1: Is the mean XG significantly different than the mean EG?
RQ 2.2: Is the mean XG significantly different than the mean SET?
RQ 2.3: Is the mean EG significantly different than the mean SET?
Method
Participants
Over a 6-year period, 1,479 students were enrolled in 57 sections of a career course to fulfill elective requirements for the baccalaureate degree. The class met in a standard classroom in academic buildings on the campus. Although the class was offered for variable credit, over 95% of the students took it for 3 credit hours. The number of students per section ranged from 19–34 with a mean of 26.5.
Ethnic diversity was generally proportional to the general student population of the university: Caucasian, 74%; African American, 12%; Hispanic American, 7%; Other, 4%; Asian, 3%; and American Indian, .4%. The course typically enrolled about 60% females and 40% males, including freshmen (15%), sophomores (45%), juniors (20%), and seniors (20%). Depending on the semester, between 15% and 25% of the course was composed of students with officially undeclared majors, and the large percentage of sophomores was the result of academic advisors referring these undeclared students to the class. While almost 40% of the members in a typical class reported satisfaction with their present career situation, about 60% were unsure, dissatisfied, or undecided.
Course Grading Procedures
Student grades were computed using scores earned on assignments contained in the performance contract. This contract was comprised of 28 different graded activities spread across the three units of the course. Given the use of the performance contract, students in this course should have had a very good idea of what their final grade would be when they filled out the SET and estimated their grade, because only two of the 28 activities accounting for 125 of 653 total points were still ungraded at that point.
Student Evaluation of Teaching Ratings
We used a standardized instrument for SETs, the Student Instructional Rating System (SIRS; Arreola, 1973), a student course on form developed at Michigan State University (Davis, 1969) and adapted for use at our university. SIRS provided an opportunity for instructors to obtain reactions to their instructional effectiveness and course organization and to compare these results to those of similar courses offered within the university.
The SIRS consisted of 32 items and 25 of these items enabled students to express their degree of satisfaction with the quality of instruction provided in the course by using a 5-point Likert scale. For example, the course was well organized could be marked strongly agree, agree, neutral, disagree, or strongly disagree. One item on the SIRS was of special interest in this study: What grade do you expect to receive in this course? A, B, C, D, or F.
We also employed a second instructional rating instrument, the State University System Student Assessment of Instruction (SUSSAI) which had been used at the university for five years prior to this study. This instrument consisted of eight items focused on class and instructor evaluation. One item was of special interest in this study: Overall assessment of instructor: Excellent=4, Very Good=3, Good=2, Fair=1, Poor=0.
Data Collection
After obtaining permission from the university institutional review board, we received the archived career course grade data for a six-year period. We aggregated the grades of these 1,479 students by class schedule and averaged the results to achieve a mean EG for each class schedule format.
The data relating to students’ perceptions of what they had achieved and the quality of instruction they had received was collected as follows: On the last week of class, while filling out their teacher evaluations, all students in a section were asked to indicate the grade they expected to receive and the results were tallied and averaged to determine a class mean XG. These class averages of 57 sections were forwarded to the researchers, and the results were tallied and averaged to find the mean XG for each class schedule format. In addition, we retrieved overall class ratings of instructors for an ad hoc sample of career classes over the 6-year period. These data enabled us to examine the relationships between mean EG and XG, EG and SET, and XG and SET.
Procedures
In this team-taught course where all instructors were involved in making large- and small-group presentations, each co-instructor had primary responsibility for evaluating the progress of students in his or her small group and assigning a grade, while the lead instructor of the team had overall responsibility for course presentations and management. In completing the SIRS and SUSSAI items for the SET, students were asked to provide a composite rating of the instructional team for their section. SETs were completed anonymously during the final two class meetings while instructors were out of the room and then returned by a student proctor to the university’s office of evaluation services.
Data Analysis
We examined how different class formats influenced mean EG, XG, and SET. The independent variable of class schedule format had four levels. The first three levels met over the course of a 16-week fall or spring semester for either 3 hours once a week (W), 1.5 hours twice a week (MW/TuTh), or 1 hour three times a week (MWF). The final level met for 2 hours four times a week over the course of a 6-week semester (MTuWTh). Because the assumptions related to independence for the three evaluative measures could not be met (i.e., the evaluations for each class section were correlated), we analyzed the data using a split-plot design.
Results
As is the case for other ANOVA and MANOVA tests, the dependent variables were assumed to be normally distributed. We tested the dependent variables to determine if they were normally distributed by computing skewness and kurtosis of each of the dependent variables to see if they fell between −1.0 and +1.0. Both the SET and EG scores did not violate the assumptions of normality as measured by skewness and kurtosis. However, while the skewness of XG did fall within the appropriate range, the kurtosis score was 1.04. Although this score is above 1.00, we believe this minor violation does not seriously affect the results and their interpretation.
Research Question 1
Using the split-plot MANOVA, we found a significant interaction of the three evaluative measures across the four class formats F (6, 106) = 4.47, p < .0005, η2 = .20. Specifically, there was a significant difference in EG between the four course formats, F (3, 53) = 19.15, p < .0005, partial η2 = .52. The EG for schedule MTuWTh (M = 3.50) was significantly higher (p < .005) than that of formats W, MW/TuTh, and MWF (M = 3.25, 3.32, and 3.31, respectively). Next, there was a significant difference in XG between the four course formats, F (3, 53) = 3.62, p = .019, η2 = .02. The means for XG for the W, MW/TuTh, MWF, and MTuWTh were 3.71, 3.57, 3.34, and 3.64, respectively. There was not a significant difference for XG between formats W, MW/TuTh, and MTuWTh. However, there was a significant difference between format MWF and format MTuWTh (p = .036), and format MWF was trending lower when compared with format W (p = .097) and format MW/TuTh (p = .051). Finally, there was not a significant difference on SET scores across the four formats, F(3, 53) = 1.36, p = ns. The mean SET scores for formats W, MW/TuTh, MWF, and MTuWTh were 2.88, 3.15, 3.31, and 3.11, respectively.
Research Question 2
When we compared evaluation measures within each format, we found significant differences with each one, F (2, 52) = 23.61, p < .0005, η2 = .47. We found XG significantly greater than EG within schedule format W (.46, p = .002) and format MW/TuTh (.35, p < .0005). By contrast, the difference between XG and EG was smaller and not statistically significant within format MWF (.13, p = ns) and format MTuWTh (.13, p = ns). This lack of a significant difference between EG and XG indicates that these students earned grades very similar to the grades they expected to receive. It is apparent that the students and instructors used similar evaluation and grading methods. Stated another way, this finding suggests that students in classes meeting more frequently per week have a slightly more accurate perception of how they are doing in the class.
We also found that mean XG was significantly greater than mean SET for format W (.83, p =.003), format MW/TuTh (.42, p < .0005), and format MTuWTh (.53, p < .0005). However, there was not a significant difference between XG and SET for format MWF (.13, p = ns). Finally, in comparing the difference between mean EG and mean SET within each of the four formats, we found a significantly higher EG only for format MTuWTh (.40, p < .0005). No significant differences were observed for formats W, MW/TuTh, and MWF, which had differences of .37, .07, and .13, respectively.
In summary, we found significant differences in the evaluation measures of XG, EG, and SET across the four different career course formats. Class sections which met four times a week for 6 weeks had a significantly higher EG than classes meeting one, two, or three times a week for a 16-week semester. Interestingly, formats W, MW/TuTh, and MTuWTh all had mean XG scores over 3.55, while format MWF’s XG was not only lower than the other formats, but significantly lower than that of format MTuWTh. Finally, mean SET scores were not significantly different from one another. Notably, they were all well above the rating of “good” (good = 2.0), with a mean of 3.15 on a 4-point scale. Means for the sections ranged between 2.88 and 3.31; thus we concluded that students found the instruction to be very good or excellent.
Discussion
Career course interventions have been developed to help students improve their academic and career decision-making skills. Comprehensive career courses offered for academic credit represent a cost-effective intervention that could be described as a “mega-dose” of career services (Reardon et al., 2011). While the benefits of college career courses are clear, it is unclear what contributions specific class formats (differing by length of class period, number of classes per week, length of course in weeks) might make to their effectiveness. Thus, the purpose of our study was to analyze the influence of different schedule formats on earned and expected grades and students’ evaluation of their instructors.
Previous studies on career development classes have described various limitations (see Gold, Kivlighan, Kerr, & Kramer, 1993; Reese & Miller, 2010), and we attempted to address these in the following ways. First, although we did not directly address random selection and random assignment issues, we aggregated class section scores instead of individual student scores, thus reducing the effect of individual outliers. By using the aggregate mean for each career planning section, individual students’ evaluation of the teacher remained anonymous yet the evaluation of the course section remained intact. The second limitation described by other researchers is the small number of participants in the career class analyzed. Over a six-year period we were able to collect data from almost 1,500 students from 57 sections of the course. The third limitation we attempted to address was the lack of equal representation of different ethnic groups. While we did not have equal percentages of students from different ethnicities, the demographic composition of our sample closely matched the composition of our university.
Perhaps the greatest strength of this study’s design was the replication of the intervention. That is, because the course structure and specific assignments were very similar for all sections, in effect the replication of the career course occurred across all 57 of the course sections analyzed. In each section, the course content and procedures were clearly specified and grades were based on the successful execution of a performance contract by the student.
Earned and Expected Grades
We examined how schedule influenced mean earned grade (EG) and expected grade (XG) scores. Like Vernick et al. (2004), we found that sections meeting only once per week over 16 weeks (format W) had the lowest EG, though not significantly lower than formats MW/TuTh and MWF. By contrast, schedule MTuWTh had a significantly higher EG than all the other formats, suggesting that a 6-week semester of 2-hour class meetings four times a week was more conducive to learning than a 16-week semester of classes meeting one, two, or three times per week for 3 hours, 1.5 hours, or 1 hour, respectively; that is, the “mega-dose” of career development interventions given in the course were intensified with MTuWTh.
Further analysis of the difference between mean section EG and XG scores enables us to compare the students’ view of their performance in the course with their actual performance. Ideally, we would prefer that there not be a significant difference between XG and EG in order to increase students’ confidence about the fairness of the grading and their sense of having mastered the material in the course. Expanding on these points, when the section mean XG was significantly higher than the mean EG, students could have left the course with a sense of failure and disappointment. Interestingly, in this study schedules W and MW/TuTh had significantly higher mean XG than mean EG, indicating an incongruity between the expected and earned grades. By contrast, for both schedules MWF and MTWF, the difference between mean XG and mean EG was not significant. One might conclude that fewer course meetings per week increased the difference between XG and EG scores.
Student Evaluation of Teaching
With regard to SET, there were no significant differences between the four class schedule formats, although we had suspected this might be the case. Perhaps a significant difference between section means for SET and XG would describe an incongruity between the students’ estimate of instruction quality and their evaluation of their own performance in the course. If XG were significantly higher than SET, this finding might indicate that students in these sections believed their performance was more related to their abilities and efforts rather than course instruction. By contrast, sections with significantly lower XG than SET scores may have rated instructors’ presentation of material higher than their own performance in the course. Interestingly, for schedules W, MW/TuTh, and MTuWTh, XG was significantly higher than SET, suggesting that students evaluated themselves more favorably than they did their instructors. We found it curious that for schedule MWF alone, XG was not significantly higher than SET.
Finally, EG is assigned to the student by the instructor, while SET is assigned to the instructor by the student. By comparing mean EG with SET, we can examine the relationship between an instructor’s evaluation of his or her students with students’ evaluation of the instructor. When EG is greater than SET, this means that instructors evaluated their students more favorably than they themselves were evaluated; conversely, when SET is greater than EG, students evaluated instructors more favorably than they themselves were evaluated. For schedules W, MW/TuTh, and MWF, there were no significant differences between mean EG and SET scores. However, for MTuWTh, in which students achieved a significantly higher mean EG than the other formats, the EG also was significantly higher than the SET, suggesting that this high-performing group had higher expectations for their instructors than they felt the instructors met.
Limitations
Because this study is field research, there are a few limitations to discuss. First, participants were undergraduates taking a career planning course from one university. The advantage to using this approach was consistency of teaching content, training and quality control of teaching personnel, administration of tests, and assignments, thus reducing the possibility that course differences were responsible for random error variance. But, because these results come from only one university’s career course, caution should be exercised when generalizing them to other courses.
Second, participants were not randomly selected. In fact, random assignment was impossible given the students’ autonomy in selecting this course. Random selection is seldom an option in field research at an educational institution, but this fact does restrict the robustness and generalizability of results to other populations (Babbie, 2001).
Third, participants in the study may have been experiencing more career-related difficulties than other students who did not elect to take the course. It is to be expected that participants perceived a career course as more important to their progress than nonparticipants, which limits generalizability of these findings (Smith & Glass, 1987).
Fourth, because the data were collected over a six-year period, it is difficult to determine the effect of historical events on the behavior and attitudes of participants (Smith & Glass, 1987; Van Dalen, 1979). For example, students from the initial semester of the study took the class at the height of the tech bubble, while others took the class in the shadows of the 9-11 tragedy. Although we were not able to control for these events, we acknowledge that researchers and practitioners must be aware of the influence of external events upon any college course.
Implications
There are several implications regarding the findings of this study. The significant differences found between schedule formats in the outcomes of EG and XG serve to remind instructors, those who supervise them, and those managing career courses about the potential impact of this variable. For example, these findings indicate that classes meeting one time per week for three hours are not characterized by higher earned grades, and by implication this means student learning. Additional studies should isolate and evaluate format variables such as length of the entire course, number of classes per week, and length of individual classes so that those evaluating teachers might consider this in their evaluations. At the same time, the absence of any differences in student evaluations of teaching across the four schedule formats is reassuring for those teaching and supervising instructors, at least in a course that was as highly structured and standardized as the one in this study.
Career services providers, curriculum designers, administrators, and instructors may wish to consider these findings when making decisions about the design and delivery of career courses, especially for high-risk freshmen (Osborn et al., 2007). Students meeting for four classes a week over a 6-week semester earned and expected significantly higher grades overall than students meeting over a 16-week semester. Taking the 6-week intensive course during the summer term before beginning the freshman year could both increase students’ chances of academic success and their confidence in navigating the college experience.
References
Arreola, R. A. (1973). A cross-institutional factor structure replication of the Michigan State University SIRS faculty evaluation model. College Student Journal, 7, 38–42.
Babbie, E. (2001). The practice of social research. Belmont, CA: Wadsworth/Thomson Learning.
Baxter, E. P. (1991). The TEVAL experience, 1983–88: The impact of a student evaluation of teaching scheme on university teachers. Studies in Higher Education, 16, 151–179.
Blustein, D. L. (1989). The role of career exploration in the career decision making of college students. Journal of College Student Development, 30, 111–117.
Brown, S. D., & Ryan Krane, N. E. (2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 740–766). New York, NY: John Wiley & Sons.
Brown, S. D., Ryan Krane, N. E., Brecheisen, J., Castelino, P., Budisin, I., Miller, M., & Edens, L. (2003). Critical ingredients of career choice interventions: More analyses and new hypotheses. Journal of Vocational Behavior, 62, 411–428.
Centra, J. A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco, CA: Jossey-Bass.
Davis, R. H. (1969). Student Instructional Rating System (SIRS) Technical Bulletin. East Lansing, MI: Michigan State University, Office of Evaluation Services.
Folsom, B., & Reardon, R. (2003). College career courses: Design and accountability. Journal of Career Assessment, 11, 421–450.
Gold, P. B., Kivlighan, D. M. Jr., Kerr, A. E., & Kramer, L. A. (1993).The structure of students’ perceptions of impactful, helpful events in career exploration classes. Journal of Career Assessment, 1, 145–161.
Goldman, L. (1985). The betrayal of gatekeepers: Grade inflation. Journal of General Education, 37, 97–121.
Greenwald, A. G., & Gillmore, G. M. (1997). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52, 1209–1217.
Griffin, B. W. (1999). Results of the faculty survey on student ratings of instruction: Preliminary report. Statesboro, GA: Georgia Southern University, Student Ratings Committee.
Marsh, H. W., & Dunkin, M. (1992). Students’ evaluations of university teaching: A multidimensional perspective. In J. C. Smart (Ed.), Higher education: Handbook on theory and research (Vol. 8, pp. 143–234). New York, NY: Agathon Press.
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52, 1187–1197.
McKeachie, W. J. (1997). Student ratings: The validity of use. American Psychologist, 52, 1218–1225.
Osborn, D. S., Howard, D. K., & Leierer, S. J. (2007).The effect of a career development course on the dysfunctional career thoughts of racially and ethnically diverse college freshmen. Career Development Quarterly, 55, 365–377.
Plant, E. A., Ericsson, K. A., Hill, L., & Asberg, K. (2005). Why study time does not predict grade point average across college students: Implications of deliberate practice for academic performance. Contemporary Educational Psychology 30, 96–116. doi:10.1016/j.cedpsych.2004.06.001
Reardon, R. C., Folsom, B., Lee, D., & Clark, J. (2011). The effects of college career courses on learner outputs & outcomes: Technical report No. 531. Tallahassee, FL: Center for the Study of Technology in Counseling and Career Development, Florida State University. Retrieved from http://career.fsu.edu/techcenter/whatsnew/TechRept53.pdf.
Reardon, R. C., Leierer, S. J., & Lee, D. (2007). Charting grades over 26 years to evaluate a career course. Journal of Career Assessment, 15, 483–498. doi:10.1177/1069072707305767.
Reardon, R. C., Lenz, J. G., Sampson, J. P., Jr., & Peterson, G. W. (2000). Career development and planning: A comprehensive approach. Pacific Grove, CA: Wadsworth-Brooks/Cole.
Reed, C., Reardon, R., Lenz, J., & Leierer, S. (2001). Reducing negative career thoughts with a career course. Career Development Quarterly, 50, 158–167.
Reese, R. J., & Miller, C. D. (2006). Effects of a university career development course on career decision-making self-efficacy. Journal of Career Assessment, 14, 252–26.
Reese, R. J., & Miller, C. D. (2010). Using outcome to improve a career development course: Closing the scientist-practitioner gap. Journal of Career Assessment, 18, 207–219.
Schmelkin, L. P., Spencer, K. J., & Gellman, E. S. (1997). Faculty perspectives on course and teacher evaluations. Research in Higher Education, 38, 575–592.
Smith, G. E. (1981). The effectiveness of a career guidance class: An organizational comparison. Journal of College Student Personnel, 22, 120–124.
Smith, M. L., & Glass, G. (1987). Research and evaluation in education and the social sciences. Englewood Cliffs, NJ: Prentice-Hall.
Svanum, S., & Bigatti, S. (2006). Grade expectations: Informed or uninformed optimism, or both? Teaching of Psychology, 33, 14–18.
Van Dalen, D. (1979). Understanding educational research: An introduction. New York, NY: McGraw-Hill.
Vernick, S. H., Reardon, R. C., & Sampson, J. P. Jr. (2004). Process evaluation of a career course: A replication and extension. Journal of Career Development, 30, 201–213.
Whiston, S. C., Sexton, T. L., & Lasoff, D. L. (1998). Career-intervention outcome: A replication and extension of Oliver and Spokane. Journal of Counseling Psychology, 45, 150–165.
Whiston, S. C., & Oliver, L. W. (2005). Career counseling process and outcome. In W. B. Walsh & M. Savickas (Eds.), Handbook of vocational psychology (3rd ed., pp. 155–194). Hillsdale, NJ: Erlbaum.
Robert C. Reardon, NCC, NCCC, is Professor Emeritus at the Career Center at Florida State University. Stephen J. Leierer is an Associate Professor at East Carolina University. Donghyuck Lee is an Assistant Professor at Konkuk University in Seoul, Korea. Correspondence can be addressed to Robert C. Reardon, Career Center, Florida State University, 100 S. Woodward St., Tallahassee, FL 32306-4162, rreardon@admin.fsu.edu.
Sep 2, 2014 | Article, Volume 1 - Issue 2
John McCarthy, Michelle Bruno, Teresa E. Fernandes
The prevalence of depression increases markedly during adolescence, yet many youth are not receiving the support that they need. One factor that has been speculated as contributing to low rates of care is a lack of mental health literacy about depression and viable sources of support. This pilot study focused on mental health literacy as it relates to adolescent depression and suicidality and represented a pseudo-replication of Burns and Rapee (2006). Overall, participants (N=36) in this study were able to differentiate depressed vignettes from non-depressed vignettes and identify common symptoms of depression in their assessments. Also, sources of optimal help identified by participants varied upon the perceived degree of seriousness of the difficulties. Such results offer implications regarding the potential benefit of including adolescents in a more direct way when providing outreach or offering services.
Keywords: adolescents, mental health literacy, depression, suicidality, support
Depression in adolescence is of particular relevance, as it can continue into adulthood yet often goes undiagnosed and untreated (Wagner, Emslie, Kowatch, & Weller, 2008). According to the Diagnostic and Statistical Manual of Mental Disorders-Text Revision (DSM-IV-TR) (APA, 2000), the diagnostic criteria and duration mirror adult depression in many respects. As in adult depression, adolescent depression can include a variety of symptoms, at least one of which must be either depressed mood or loss of pleasure/interest. Furthermore, the DSM-IV-TR stipulates that, if depressed mood is chosen, it may be substituted by irritable mood in adolescents.
The rate of depression increases six-fold between the ages of 15–18 (Hankin, 2006). Approximately eight percent of teenagers—an estimated two million youth from 12–17 years of age—suffered at least one major depressive episode in 2007. Only 39% received some form of treatment for depression in the preceding 12 months. The rate of receiving professional help was much lower among those youth without health insurance (17%). Among all teenagers who obtained treatment, over half (59%) saw a counselor for assistance with their depression. Nearly 37% and 27% of youths saw a psychologist or general practitioner/family doctor, respectively (Substance Abuse and Mental Health Services Administration, 2009).
Given the prevalence of mental illness and its impact on society, it is no surprise that there is a growing interest in mental health literacy, a term first used by Jorm et al. (1997). Defined as the “knowledge and beliefs about mental disorders which aid their recognition, management or prevention” (p. 182), mental health literacy also includes knowledge about treatment and from whom to seek help. It has been found, for instance, that family and friends can be vital in the recognition of depressive symptoms (Langlands, Jorm, Kelly, & Kitchener, 2008a). However, Highet, Thompson, and McNair (2005) saw that family members usually recognized symptoms of the individual in hindsight. The general public often does not possess the knowledge base to help someone who is developing a psychotic illness (Langlands, Jorm, Kelly, & Kitchener, 2008b). Kitchener and Jorm (2002) found that individuals who took part in their Mental Health First Aid course showed improvement in recognizing disorders, and their views about treatment of disorders became more in line with those of professionals in the mental health field. In addition, the course reduced their stigma attached to mental disorders, increased their feelings of confidence in providing help, and increased the help provided to others.
Few studies have been conducted on younger populations and mental health literacy. Burns and Rapee (2006) noted, “While there is growing literature on the mental health literacy of adults, to date there has not been a parallel interest in the mental health literacy of young people” (p. 227). Wright et al. (2005) looked at young adults’ (ages 12 to 25) ability to pinpoint depression and psychosis and their recommendations for help to be sought. Nearly half of the participants were able to label the depressed vignette as depressed, but only a quarter of participants were able to label psychosis. People who were given the depressed vignette were less likely to choose a correct form of treatment than those given the psychosis vignette. Psychologists and psychiatrists were recommended more frequently for the psychosis vignette than for the depressed vignette, and a family doctor or general practitioner was chosen more often for the depressed vignette than for the psychosis vignette.
Adolescents have been found more likely to consider themselves “very confident” (Jorm, Wright, & Morgan, 2007a, p. 67) to help a peer in need with girls rating themselves as more confident than boys. In addition, across vignettes, confidence in providing help to a peer with a problem was higher for depression (without alcohol misuse) and social phobia than for psychosis and depression with alcohol misuse (Jorm et al, 2007a).
Jorm, Wright, & Morgan (2007b) found differences among Australian youth in the type of help sought for mental disorders. Participants were read vignettes describing youth of similar ages who were experiencing various disorders, then were asked a series of questions that included where they would turn with similar problems. For the vignette describing a teen suffering from depression, adolescents aged 12–17 chose family (54%) most often as a source of help and opted for mental health professional or service most infrequently (2%). Nearly one-third of young adults ages 18–25 selected family (31%) or a general practitioner/medical doctor (31%) on a similar vignette regarding depression. Overall the perceived barriers to help-seeking were personal in nature and did not relate to systemic characteristics, as they noted, “For young people, it is embarrassment or concern about what others think…” (p. 559).
Burns and Rapee (2006) used a vignette-based approach to measure mental health literacy among high school students in Australia. In their study, they utilized the Friend In Need questionnaire, created by the authors for that specific study. This instrument offers five short vignettes of teenage students, two of whom (“Tony” and “Emily”) represented youth meant to be clinically depressed. One of the two vignettes (“Emily”) offered a reference to suicidal ideation. The remaining three vignettes were of students facing difficulties, though were not intended to reflect depression.
They found that over two-thirds of participants (68%) accurately labeled “Emily” as depressed, while about one-third (34%) recognized “Tony” as depressed. Female participants were more likely to make a depressed diagnosis in both the “Tony” and “Emily” vignettes than the male participants. Female participants also showed more worry for the depressed vignettes than male participants. Among help-seeking sources, counselors were chosen most often for the helpers of the depressed teens, and this category was followed by friends and family/relatives.
To our knowledge, no study has been conducted on the mental health literacy of U.S. teens as it pertains to adolescent depression. With this point in mind, the current study represents a replication of Burns and Rapee (2006) and offers an initial sample involving older adolescents’ perspectives in the assessment, recovery time, and help-seeking recommendations regarding depression. Our central study questions were consistent with Burns and Rapee and the questions posed by the Friend in Need Questionnaire.
Procedures
Both prior to and after receiving approval by the university’s institutional review board, two of the authors met with the principal of the school where the data was collected. It was determined that eight sections of the school’s psychology and anthropology classes would be appropriate to the topic of study and ages of interest, and the primary author contacted the teachers and shared the following information with them: the parental/guardian consent form, the student consent form, details concerning the data collection process, and pertinent dates of the consent form deadlines and actual administration of the instrument used in this study. Teachers distributed the consent forms to students, who, if interested in possibly participating in the study, took them to their parents/guardians. Signed parental/guardian consent must have been completed and returned to the teachers in a four-day time period, which occurred prior to the date of the administration of the instrument. In both the parental/guardian and participant consent forms, it was made clear that the questionnaire was not a formal test and would take an estimated 25–40 minutes to complete.
On the day of the data collection, one of the two primary authors (JM and MB) went to the classroom, collected the completed parental/guardian consent form, read an abbreviated student consent form to the potential participants after giving a hard copy to them, and asked for questions at the conclusion. Students with unsigned parental/guardian consent forms were given an alternate class assignment, while those students who consented to be in the study completed the Friend in Need Questionnaire. No extra credit was granted for participation in the study. Participants completed the questionnaire in their classrooms. In a few instances, participants and the author/administrator were asked to move to a nearby vacant room for the data collection.
Approximately five classes were visited for data collection, and a total of 36 students, 21 of whom were young men, participated in the study. Most participants completed the questionnaire in approximately 20 minutes. The questionnaires were completed in an anonymous manner. In the coding process, a number was given to each questionnaire for tracking purposes only. Finally, the two authors also offered to return to the class after the data administration to further discuss the study; however, no teachers chose this option.
Instrument
Adolescents’ mental health literacy was assessed using the Friend in Need Questionnaire (Burns & Rapee, 2006). As previously described, the questionnaire presents five vignettes of young people experiencing various difficulties and solicits both close-ended and open-ended responses from participants. Specifically, participants are instructed to read each vignette and respond to the following general questions: (a) How worried would you be about the person’s overall emotional well-being? (b) What do you think is the problem of the person? (c) What aspects of the vignette provided the strongest hints that the person was having difficulties? (d) How long will it take this adolescent to feel better? and (e) Does this person need help from others to cope with his/her problems? The final question also has a supplemental, open-ended question regarding who the helper would be. The respondents are posed with all of these questions for each of the five vignettes. The complete Friend in Need Questionnaire can be found in Burns and Rapee (2006).
A coding system was devised for the open-ended responses, specifically on the responses asking about the youth’s problem, aspects of the vignette that provided hints, and the appropriate helper. For the question concerning the youth’s problem, the responses were filtered into two categories: “depressed” or “not depressed.” To qualify as “depressed,” the respondents needed to write the words “depressed/depression” or “suicide/suicidal.” Any other problems listed were considered to be “not depressed.” On the question regarding hints of the problem in the vignette, the coder was looking for responses that fit into diagnostic criteria for depression. The two depressed vignettes each had five diagnostic criteria imbedded in them, and this question tried to tease out whether respondents could identify these key criteria. Hence, the responses were categorized into the five diagnostic criteria of each vignette, with other responses not qualifying. The question that asked about the appropriate helper was split into nine possible categories of helpers. A few respondents, whose answers occurred rarely, were not included in the analyses.
Results
The findings are described in order of the items presented in the Friend in Need Questionnaire. The first question assessed whether adolescents could label a cluster of depressive symptoms in a case vignette as depressed. Respondents were asked, “What do you think is the matter with [name]?” This open-ended question elicited a variety of responses from respondents. Only responses that included “depressed,” “depression,” “suicide,” or “suicidal” were coded as a label of depression. In reviewing the responses to the two vignettes concerning students (Tony and Emily) depicted as depressed, it was evident that the majority of participants accurately labeled the vignettes, as 75% accurately identified Emily as depressed and 58% accurately labeled Tony as depressed.
The majority of respondents also accurately identified the non-depressed vignettes as such. Specifically, over 94% of respondents accurately identified Mandy as not being depressed. All participants (100%) accurately identified Jade as non-depressed, and over 97% accurately identified Nick as not being depressed. Frequencies of depressive codes for all vignettes are included in Table 1. Separate chi-square analyses were conducted to examine any differences in ratings of each vignette between male and female participants. Results indicated that no such differences exist on any of the five vignettes.
Second, in regard to respondents expressing greater worry for youth in the depressed vignettes versus the non-depressed vignettes, the Friend in Need Questionnaire instructed participants to rate their concern on a five-point scale with higher scores indicating more worry. The scores for the depressed vignettes (Emily and Tony) and non-depressed vignettes (Mandy, Jade, and Nick) were collapsed to produce mean scores of level of worry. A general linear model was used to compare sex differences (participant) in the intensity of worry scores for depressed and non-depressed vignettes. Results indicate that no significant differences existed between male (M = 3.40, SD = .38) and female participants (M = 3.45, SD = .33) regarding ratings of worry for the depressed (p < .58). No significant differences were found regarding male (M = 1.80, SD = .41) and female participants’ (M = 1.81, SD = .39) ratings of worry of the non-depressed vignettes either (p < .82).
The third question pertained to the length of recovery in the depressed and non-depressed vignettes. The respondents rated each vignette on the perceived length of time it would take the character to feel better on a four-point Likert scale from 1 (one or two days) to 4 (longer than a few months). Higher scores indicate a perception that more time is needed to feel better. Despite the use of a Likert scale, some respondents chose two answers or marked in between two options. When this occurred, the score was adjusted to reflect an average. For example, if someone circled, both “3” and “4,” a score of “3.5” was entered. This decision was made to maintain as many respondents as possible, given the small number of the sample. Overall, the respondents rated the depressed vignettes with a mean score of 3.67 (SD = .37), which indicates a recovery period of between “one or two months” and “longer than a few months.” This finding compared to a lower mean score of 1.97 (one or two days, SD = .45) for the non-depressed vignettes. Scores on the two depressed vignettes and scores on the three non-depressed vignettes were collapsed to create a composite mean score of recovery time for depressed (dependent variable) versus non-depressed vignettes (dependent variable).
A two-way MANOVA was conducted to determine if sex differences (of respondents) made a difference in the length of the recovery for both scenarios (depressed versus non-depressed). The overall model was statistically significant for the recovery time between the depressed and non-depressed vignettes F (1, 34) = 651.31; p = .01. The MANOVA did not reveal a significant interaction between participant gender and recovery time of vignettes (p < .27). Female respondents rated both the depressed vignettes (M = 3.82, SD = 24) and non-depressed vignettes (M = 2.03, SD = .43) higher than male respondents who rated the vignettes as 3.57 (SD = .53) and 1.93 (SD = .47) respectively, but this difference was not statistically significant.
Fourth, participants were asked to identify the elements of the vignette that demonstrated whether the fictitious teens were having emotional troubles. The two depressed vignettes (Emily and Tony) contained criteria of a Major Depressive Episode as described in the DSM-IV-TR (APA, 2000). In the case of Emily, respondents readily identified indicators of suicide (91%) and self-worth (72%). Respondents were less likely to identify symptoms of loss of interest (19%), fatigue (22%), and mood (19%) in this case. (See Table 2 for more complete results.) In the case of Tony, a majority of respondents identified loss of interest (75%) and weight loss (58%). Respondents were less likely to identify Tony’s fatigue (44%), insomnia (39%), and diminished ability to think or concentrate (39%).
Finally, after noting which symptoms were strong indicators of problems, respondents answered an open-ended question about sources of help to aid the person in the vignette. For all five vignettes, participants answered whether they thought the person in the vignette needed help from another person. The options included “no,” “yes,” or “don’t know.” If the respondents endorsed that the person did need help, they were asked to answer a follow-up question indicating who they think should help the person. For the depressed vignettes, 58% of respondents indicated that Tony needed help, and 75% indicated the same for Emily.
In regard to the type of helpers, participants’ responses were broken down into nine categories of helpers, including counselor; friends; family; professional; psychologist; psychiatrist; doctor; teacher; and someone who has had the same difficulty. Some coding decisions included how to categorize responses not explicitly in the list. Some of these included counseling, school counselor, and guidance counselor, which were included in the category of counselor. For the friend category, other responses included “peers” and “someone who knows him/her well.” For family, “parents,” “relatives,” “siblings/brother/sister” also were included. Non-specific terms were included in the professional category, including specialist, shrink, therapist, psychotherapist, and family therapist. Other responses included in the psychiatrist category were “doctor for depression/depressed kids” and “doctor who prescribes antidepressants.” Some responses that were not coded included third party, new people, anyone, role model, someone he/she doesn’t know, and everyone.
Nearly half of the participants (47%) identified the family as the suggested primary helper for Tony, while over one-third (36%) of participants suggested a counselor. The same percentage (36%) identified the family and a psychiatrist, respectively, for Emily, as the best sources of help (see Tables 3-4 for more complete results).
Discussion
The primary purpose of this study was to examine the level of teenagers’ mental health literacy specific to adolescent depression. Because it was a pilot study that involved a relatively small sample size, the findings are admittedly limited in generalizability. However, even with the small sample size, the results offer initial points of comparison to Burns and Rapee’s (2006) larger scale study. First and perhaps foremost, the level of detection of adolescent depression was relatively high in the present study, yet no significant differences were found as they related to gender. Over half of the participants correctly labeled both depressed-based vignettes (Emily and Tony) as being depressed, and three in four participants indicated that Emily was depressed. To their credit, participants rated both depressed vignettes as highest in terms of depression.
This finding is noteworthy. In Burns and Rapee (2006), the corresponding findings of correctly identifying depression in Emily and Tony were 68% and 34%, respectively. The higher rating of Emily as depressed was similar in both settings, yet the rating of Tony as depressed was sizably different with American participants being more inclined to have viewed the fictitious student as depressed.
A closer investigation of this finding points to critical symptoms chosen in the participants’ assessment. The vignette of Emily featured pointed comments of suicidality, and, to no surprise, it was this characteristic that was almost uniformly (92%) expressed by participants when asked about the “strongest hints that something was wrong.” The element of suicidality also was foremost in Burns and Rapee (2006) in reference to Emily, but its expression was lower (77%) among the Australian sample. At least two possibilities are present. First, it is conceivable that the Australian teenagers were not as concerned about the suicidal ideation as the U.S. participants in the present study. A second possibility is that the awareness of suicidality among adolescents has increased in more recent years in the U.S., prompting a higher rate among the U.S. teenagers.
Suicidality was absent in the vignette of Tony. However, other signs of depression were present, and these symptoms included anhedonia, fatigue, weight loss, insomnia, and diminished ability to think/concentrate. Both U.S. participants in the present study and Australian participants in Burns and Rapee (2006) placed “diminished loss of interest” as the primary symptom of an emotional difficulty at nearly identical rates (73% and 75%, respectively). The same held true for the second-rated symptom (weight loss) in both samples, again expressed by nearly the same percentage (58% in the present study and 61% in Burns and Rapee). The consistency in the ranking and percentages of both samples reflects the teenagers’ recognition of lowered interest levels and appetite difficulties leading to weight loss when an adolescent is experiencing depression. In actuality, both behaviors do indeed tend to be two of the six most frequent symptoms among teenagers who are depressed (Roberts, Lewinsohn, & Seeley, 1995).
To their credit, participants in the present study also were able to differentiate the depressed vignettes from the non-depressed vignettes. Mandy was feeling upset over a relationship termination initiated by her former boyfriend that occurred three days prior. Jade expressed family disruption and had become intoxicated at a recent party. Meanwhile, Nick was coping with the loss of a grandparent. None of these vignettes offered significant amounts in the way of genuine depression, and by and large, the majority of participants detected that their respective problems were not severe. A mere 6% of participants indicated that Mandy was depressed. Similarly, none of the participants indicated that Jade was depressed, and only 3% of them assessed Nick to be depressed. This finding offers support for the overall level of mental health literacy of the sample as it pertains to adolescent depression. Moreover, in comparison to the Australian participants in Burns and Rapee (2006), the American sample fared somewhat better: They found that, though none of their participants found Jade to be depressed, 11% and 9% of teenagers in their study did relate Nick and Mandy, respectively, to be depressed.
The participants in the present study demonstrated significantly more concern and anticipated a longer recovery period for the students in the depressive vignettes than in the non-depressed vignettes. In our study, a significant difference was accurately found in estimated recovery time.
The average duration of an initial depressive episode is eight months when no treatment is received (Brent & Birmaher, 2002). These findings add support to the conclusion that the sample possessed a considerable level of literacy. Given the fact that, to our knowledge, this pilot study is the first to assess mental health literacy for adolescent depression among American teenagers, no point of comparison exists. With this point in mind, the finding was relatively surprising. The adolescents in the present study were astute in their detection, concern, and estimated time of recovery, which could be related to a knowledge set based on their classroom education or acquired in other ways (i.e., having a friend who was depressed). Regardless of the mode of acquisition, the adolescents in this study offered greater concern for the fictitious students in the midst of a depressive episode and estimated their recovery more accurately than those students in the non-depressed vignettes.
It was mildly surprising that, unlike Burns and Rapee (2006) and Gifford-May (2002), no significant difference was found in regard to gender and mental health literacy. Burns and Rapee found that girls “clearly demonstrated” higher literacy in their abilities to not only correctly label the depressed vignettes, but also in their expression of greater concern over the students in those same vignettes (p. 232). One point of speculation on their part dealt with the higher levels of depression experienced by young women in later adolescence (Lewinsohn, Rohde, & Seeley, 1998). However, given the absence of significant differences in gender within the sample in the present study, it raises the possibility that young men in the U.S. are more insightful regarding adolescent depression than anticipated.
Burns and Rapee (2006) indicated that the primary reason for raising the mental health literacy of adolescents “is to increase the likelihood that young people can access the most appropriate help when needed” (p. 233). Taken from combined data from 2005 and 2006, an estimated 12% of American youth aged 12–17 obtained professional help for emotional or behavioral problems, and females were more likely than males to receive professional help (Office of Applied Statistics, 2008). However, the literature points to the fact that many other teenagers in need of mental health assistance for various disorders do not receive it. In fact, a mere 39% of those adolescents suffering a depressive episode receive treatment (Office of Applied Statistics, 2009).
The recommended sources for help in our sample were family and counselor, respectively, for Tony, and family and psychiatrist, both at equal percentages, for Emily. For the vignette of Emily, counselor ranked sixth of the nine helping sources. This finding is in contrast to the real-world conditions where nearly 60% of those teenagers with depression in 2007 saw or talked to a counselor in their treatment (Office of Applied Statistics, 2009).
Though the reasoning behind the choices of the helping sources was not sought, the selections lead to intriguing possibilities. First, in the case of Tony, the primary helping source was family, despite information in the vignette that the family system was deteriorating over a parental separation. Even if that played no role in the participants’ responses, the choice of family in soliciting help is striking in that parent-adolescent conflicts increase during early adolescent years (Laursen, Coy, & Collins, 1998). Suicidal adolescents reported difficulty in communicating with parents, tremendous stress in their home life, and a distressed relationship with one or both parents (Bostik & Everall, 2006). However, this finding is consistent with a qualitative study (McCarthy, Downes, & Sherman, 2008) pointing to beneficial parental partnerships that developed during depressive episodes and were instrumental in the teenager getting professional help. Counselor, the second recommended helper choice in the vignette of Tony, may not be as surprising. The school from which the data were collected does have a staff of professional school counselors, and this finding may speak to the participants’ level of comfort with counselors.
The topic of recommended helper was much different in the vignette of Emily, as the choices were much more equal in terms of the percentages. The selection of psychiatrist as the second recommended helper may point to the participants’ perception of the potential for harm and their connotation that a physician with mental health expertise and prescription privileges was needed. In a similar vein, the designations of psychologist and professional were closely behind psychiatrist in recommended helpers, again suggesting the participants’ notion that highly trained professionals who likely have a doctoral degree were needed to aid Emily. This finding mirrors recent research, as 27% of those adolescents having a depressive episode saw their family physician or a general practitioner. Roughly the same number sought help from a psychiatrist or psychologist (Office of Applied Statistics, 2009).
Surprisingly, friends were the third most common choice of helper in a case of a student marked by suicidal ideation. With the potential for harm in this student, friends may not be the best source for initial help. However, participants in the present study may have thought that friends would be supportive during an emotionally difficult period. Finally, the lower ranking of the counselor designation may be connected with a perception that a counselor is sought for less complex difficulties.
Burns and Rapee (2006) found that counselor and friend were the two primary overall recommended sources of help. In regard to counselors, they noted that this finding may be reflective of the “access and familiarity” that adolescents in many Australian schools possess with this type of professional (p. 233). Overall, however, the participants in their study offered far lower rankings of a psychologist, professional, or psychiatrist as a source of help in the depressed vignettes. This finding could point to a familiarity by American teenagers with medical professionals, particularly with the prevalence rate of medication prescribed to this population in the U.S. compared to European countries (Levin, 2008).
Limitations
Limitations are clearly evident in this study. As previously noted, the small sample size that is consistent with pilot studies restricts generalizability. The sample size also may have been composed of more sophisticated students in mental health, as many students in the sample were enrolled in a psychology class. Burns and Rapee (2006) pointed out that the vignette-based instrument of the Friend in Need Questionnaire is consistent with the manner in which other mental health literacy studies have been conducted. However, they added, “The extent to which such data can be translated into what actually is likely to happen in the real world is unclear” (p. 234). They also noted that a subsequent challenge for research in this area includes the development of research modalities that examine literacy in a naturalistic setting, such as interviews with adolescents. This suggestion connects to Dundon’s (2006) call to bring forth the “voice of the adolescent” that has been lacking in the research on adolescent depression (p. 384).
Implications
This pilot study represents a point of entry in studying American teenagers’ mental health literacy in regard to teen depression. Participants in this study showed the ability to correctly differentiate depressed vignettes from non-depressed vignettes and, in their assessment, indicated relevant symptoms of depressive symptoms faced by adolescents. Overall they also expressed sources of help that varied upon the perceived degree of seriousness of the difficulties. The outcomes offer implications regarding the potential benefit of including adolescents in a more direct way when providing outreach or offering services. They demonstrated an accurate understanding of when more intense levels of care could be beneficial.
The study produced results that also warrant further exploration of the relationships between youth and parents during adolescence. Although this developmental period can be marked by tumultuous relationships between them, there may be wisdom in providing communication skills to strengthen such relationships. Such efforts could result in more disclosure of depressive symptoms to parents, which may expedite the process of getting help as opposed to sharing such struggles only with peers. In addition to implications for teens and parents, this research can help shape additional studies in expanding the understanding of literacy.
Future research calls for additional mental health literacy investigations marked not only by larger sample sizes, but also by an in-depth investigation of adolescents of various racial/ethnic differences within the sample. Higher rates of adolescent depression have been found in youth of Latino descent (Guiao & Thompson, 2004; Twenge & Nolen-Hoeksema, 2002), for instance, and it would be important to evaluate the mental health literacy levels among respective backgrounds. With teenage depression being a pressing matter in adolescent mental health, the domain of mental health literacy in regard to this disorder is a vital one that merits additional research.
References
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
Bostik, K. E., & Everall, R. D. (2006). In my mind I was alone: Suicidal adolescents’ perceptions of attachment relationships. International Journal for the Advancement of Counselling, 28(3), 269–287. doi: 10.1007/s10447-006-9013-6
Brent, A., & Birmaher, B. (2002). Adolescent depression. The New England Journal of Medicine, 347(9), 667–671. doi:10.1056/NEJMcp012249
Burns, J. R., & Rapee, R. M. (2006). Adolescent mental health literacy: Young people’s knowledge of depression and help seeking. Journal of Adolescence, 29, 225–239. doi:10.1016/j.adolescence.2005.05.004
Dundon, E. E. (2006). Adolescent depression: A metasynthesis. Journal of Pediatric Health Care, 20, 384–392. doi: 10.1016/j.pedhc.2006.02.010
Gifford-May, D. (2002). Adolescents’ knowledge and beliefs about depression, coping strategies and barriers to help-seeking: Gender and ethnic difference (Unpublished master’s thesis). Macquarie University, Australia.
Guiao, I., & Thompson, E. A. (2004). Ethnicity and problem behaviors among adolescent females in the United States. Health Care for Women International, 25(4), 296–310. doi:10.1080/07399330490278330
Hankin, B. L. (2006). Adolescent depression: Description, causes, and interventions. Epilepsy & Behavior, 8, 102–114. doi:10.1016/j.yebeh.2005.10.012
Highet, N., Thompson, M., & McNair, B. (2005). Identifying depression in a family member: The carers’ experience. Journal of Affective Disorders, 87, 25–33. doi:10.1016/j.jad.2005.02.- 016
Jorm, A. F., Korten, A. E., Jacomb, P. A., Christensen, H., Rodgers, B., & Pollitt, P. (1997). “Mental health literacy”: A survey of the public’s ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Medical Journal of Australia, 166, 182–186.
Jorm, A. F., Wright, A., & Morgan, A. J. (2007a). Beliefs about appropriate first aid for young people with mental disorders: Findings from an Australian national survey of youth and parents. Early Intervention in Psychiatry, 1, 61–70. doi:10.1111/j.1751-7893.2007.00012.x
Jorm, A. F., Wright, A., & Morgan, A. J. (2007b). Where to seek help for a mental disorder? National survey of the beliefs of Australian youth and their parents. The Medical Journal of Australia, 187, 556–560.
Kitchener, B. A., & Jorm, A. F. (2002). Mental health first aid training for the public: Evaluation of effects on knowledge, attitudes and helping behaviour. BioMed Central Psychiatry, 2, 10. doi:10.1186/1471-244X-2-10
Langlands, R. L., Jorm, A. F., Kelly, C. M., & Kitchener, B. A. (2008a). First aid for depression: A Delphi consensus study with consumers, carers and clinicians. Journal of Affective Disorders, 105, 157–165. doi:10.1016/j.jad.2007.05.004
Langlands, R. L., Jorm, A. F., Kelly, C. M., & Kitchener, B. A. (2008b). First aid recommendations for psychosis: Using the Delphi method to gain consensus between mental health consumers, carers, and clinicians. Schizophrenia Bulletin, 34(3), 435–443. doi:10.1093/schbul/sbm099.
Laursen, B., Coy, K. C., & Collins, W. A. (1998). Reconsidering changes in parent-child conflict across adolescence: A meta-analysis. Child Development, 69, 817–832. doi:10.1111/j.14678624.1998.00817.x
Levin, A. (2008, November 21). U.S. youth more apt than Europeans to be prescribed psychotropics. Psychiatric News, p. 18.
Lewinsohn, P. M., Rohde, P., & Seely, J. R. (1998). Major depressive disorder in older adolescents: Prevalence, risk factors, and clinical implications. Clinical Psychology Review, 18(7), 765–794. doi:10.1016/S0272-7358(98)00010-5
McCarthy, J., Downes, E. J., & Sherman, C. A. (2008). Looking back at adolescent depression: A qualitative study. Journal of Mental Health Counseling, 30(1), 49–68.
Office of Applied Statistics. (2009, May 11). Major depressive episode and treatment among adolescents. The NSDUH Report, pp. 1–4.
Roberts, R. E., Lewinsohn, P. M., & Seeley, J. R. (1995). Symptoms of DSM-III-R major depression in adolescence: Evidence from an epidemiological survey. Journal of the American Academy of Child and Adolescent Psychiatry, 34, 1608–1617. doi:10.1097/00004583-199512000-00011
Substance Abuse and Mental Health Services Administration, Office of Applied Studies. (May 11, 2009). The NSDUH Report: Major Depressive Episode and Treatment among Adolescents. Rockville, MD.
Substance Abuse and Mental Health Services Administration, Office of Applied Studies. (September 28, 2008). Mental health service use among youths aged 12 to 17: 2005 and 2006. Rockville, MD.
Twenge, J. M., & Nolen-Hoeksema, S. (2002). Age, gender, race, socioeconomic status, and birth cohort differences on the Children’s Depression Inventory: A meta-analysis. Journal of Abnormal Psychology, 111, 578–588. doi:10.1037/0021-843X.111.4.578
Wagner, K. D., Emslie, G. J., Kowatch, R. A., & Weller, E. B. (2008). An update on depression in children and adolescents. Journal of Clinical Psychiatry, 69(11), 1818–1828. doi:10.4088/JCP.v69n1118
Wright, A., Harris, M. G., Wiggers, J. H., Jorm, A. F., Cotton, S. M., Harrigan, S. M., & McGorry, P. D. (2005). Recognition of depression and psychosis by young Australians and their beliefs about treatment. The Medical Journal of Australia, 183, 18–23.
John McCarthy, NCC, is a Professor in the Department of Counseling at the Indiana University of Pennsylvania (IUP). Michelle Bruno is an Assistant Professor in the same department at IUP. Teresa E. Fernandes, NCC, is a counselor at the Meadows Psychiatric Center, Centre Hall, PA. Correspondence can be addressed to John McCarthy, Indiana University of Pennsylvania, Department of Counseling, 206 Stouffer Hall, Indiana, PA, 15705, john.mccarthy@iup.edu.
