Online Learning and the Development of Counseling Self-Efficacy Beliefs

Oct 6, 2014 | Article, Volume 2 - Issue 2

Joshua C. Watson

This study examined the relationship between enrollment in online counseling courses and students’ counseling self-efficacy beliefs. Results indicate that students enrolled in online courses report statistically significant higher self-efficacy beliefs than students in traditional FTF courses. Online instructional method may increase counselor self-efficacy, self-confidence, and personal motivation. Implications for research and counselor education are discussed.

Keywords: online, counselor education, self-efficacy, beliefs, instructional methods

In recent years, the increasing popularity of online instruction has dramatically changed the delivery of college curricula nationwide (Artino & McCoach, 2008). The growing popularity of this medium has led some (Keeton, 2004; Tallent-Runnels et al., 2006) to project that online education will soon become the largest source of higher education in America. According to a recent study conducted at over 2,500 colleges and universities nationwide, approximately 5.6 million students were enrolled in at least one online course during the 2009 fall semester (Allen & Seaman, 2010). This figure, as noted by Allen and Seaman, represents a 21% increase over the number reported the previous year, the largest ever year-to-year increase in the number of students studying online. The increased student interest in online course delivery has led more and more institutions to look into adopting online instruction as the next logical step in educational delivery systems (Bejerano, 2008; O’Malley & McCraw, 1999). Highlighting this trend, 66% of the 4,160 degree-granting postsecondary institutions in the nation offered college-level courses via online instruction during the 2006–2007 academic year (National Center for Education Statistics, 2008). Only recently has this trend carried over to the field of counseling and counselor training (Albrecht & Jones, 2001; Benshoff & Gibbons, 2011; Chester & Glass, 2006).

Despite the growing interest from students in online course delivery, counselor educators traditionally have been hesitant to incorporate online learning into the counselor education curriculum (Greenidge & Daire, 2005; Karper, Robinson, & Casado, 2005). Both best practice considerations and the questionable research supporting the effectiveness of online instruction have played a role in explaining the hesitancy to apply components of online learning in counseling education and practice (Granello, 2000; Krieger & Stockton, 2004). According to Murphy, MacFadden, and Mitchell (2008), several educators have questioned whether online education is appropriate for the clinical professions in which the development of basic relationship-building skills is a key curricular component. One of the common concerns often voiced has been whether or not “skills-based” or “techniques” courses could be offered sufficiently online (Wantz et al., 2003). The fundamental argument of these counselor educators is that the personal, high touch nature of counseling cannot adequately be simulated at a distance. However, this perspective appears to be changing slightly. While most counselor education programs have long espoused traditional, face-to-face (FTF) didactic instruction, Quinn, Hohenshil, and Fortune (2002) noted that a growing number have come to realize that the integration of technology and online instruction into their curriculum is essential in order to remain competitive.

As a result of this shift in thinking, there is now a growing interest in the use and practicality of online instructional approaches in graduate counselor training programs (Benshoff & Gibbons, 2011; Clark & Stone, 2002; Hayes & Robinson, 2000; Patrick, 2005). Several authors describing the effective implementation of online education have begun publishing their work in the counselor training literature. Translating theory into practice, complete online accredited programs have begun emerging nationwide. A review of the Council for the Accreditation of Counseling and Related Programs (CACREP) website indicates online accredited programs existing in a variety of programs including clinical mental health counseling (Adams State University, Capella University, Wake Forest University, Walden University); community counseling (Regent University); school counseling (Adams State University, Capella University, Regent University, Wake Forest University); marriage, couple, and family counseling (Capella University); and counselor education and supervision (Regent University). Furthermore, many other programs nationwide have begun experimenting with the inclusion of various forms of online education. According to the results of a recent study conducted among CACREP accredited counselor training programs, nearly 50% of the programs surveyed had already begun integrating online instruction, either totally or partially, into their existing curriculum (Wantz et al., 2003). In terms of quality of instruction, 38% of the faculty members surveyed reported an increase in program quality as a result of adding online courses to the existing curriculum. These numbers should continue to increase as counselor educators become more familiar with existing and emerging technologies, and their departments and universities provide them with the support needed to effectively offer these courses (Baltimore, 2000; Nelson, Nichter, & Henriksen, 2010; Tyler & Sabella, 2004).

In view of the fact that the use of online technology in the preparation and training of professional counselors is still a relatively new phenomenon, it is not surprising that there is currently a general lack of empirical research concerning the effectiveness of using technology as a training tool in counselor education programs (Karper et al., 2005; Myers & Gibson, 1999; Nelson et al., 2010; Wantz et al, 2003). Though limited in the counselor training literature, a growing number of studies in which the effectiveness of online education is compared to traditional FTF instruction have been published in a variety of other disciplines (Ellis, Ginns, & Piggot, 2009; Lim, Kim, Chen, & Ryder, 2008). In these studies, researchers have traditionally assessed the effectiveness of online education by examining a limited set of measurable student characteristics, abilities or strategies thought to be related to either the successful completion of a single course (Waschull, 2005) or the final course grade earned (O’Hanlon, 2001). Although many academic programs evaluate student competence primarily using these outcomes, they may not represent the most salient outcomes for counseling students, who must not only acquire the requisite knowledge needed to be a counselor, but also transfer that knowledge to their work with actual clients.

In addition to academic proficiency, a longstanding goal of most counselor education programs has been the fostering of students’ confidence in their abilities to effectively work with the clients they intend to serve (Bernard & Goodyear, 2008; Hensley, Smith, & Thompson, 2003). Generally speaking, those counselors who are more confident in their ability to use their clinical skills in real life settings often provide a higher quality of counseling services to the clients they serve (Barnes, 2004; Bradley & Fiorini, 1999). As a result, it might be advantageous for counselor educators to measure effectiveness of various course delivery methods (e.g., FTF, online, or hybrid) by examining the beliefs their students hold in regard to their ability to become effective counselors.

Self-Efficacy

One of the major approaches often used when investigating the process of gaining competence and self-confidence in particular domains of behavior has been self-efficacy theory (Bandura, 1989). Also called perceived ability, self-efficacy refers to the confidence people have in their abilities to successfully perform a particular task (Bandura, 1986; 1997). In other words, an individual may regard him or herself to be quite capable in one area, but much less capable in another. Although self-efficacy is not equivalent to competence, researchers consistently have demonstrated that higher perceived self-efficacy is related to higher performance attainment (Bandura, 1986). Thus, individuals with a strong sense of efficacy are more likely to challenge themselves with difficult tasks and to have a higher level of intrinsic motivation. In addition, these individuals typically put forth a higher degree of effort in order to reach their goals and will attribute failure to things that are in their control rather than blaming external factors.

Although self-efficacy refers to individuals’ confidence in their abilities to complete tasks or reach goals, it is not based entirely on actual experience with performing tasks in the past. In addition to personal performance, vicarious learning, verbal persuasion and affective states all help facilitate the development of self-efficacy beliefs (Bandura, 1986). Furthermore, research in this field has shown that the selection of a specific self-efficacy measure that closely matches the desired task performance criteria is often a better predictor of performance outcomes than a more general self-efficacy measure (DeTure, 2004). Consequently, the focus of self-efficacy researchers has broadened in scope in recent years to include investigations of ability perceptions across several domains. Included in this body of research are studies of counseling self-efficacy among beginning counselors.

Counseling Self-Efficacy

Counseling self-efficacy (CSE), according to Larson and Daniels (1998), is best described as the beliefs or judgments an individual has about his or her capability to effectively counsel a client in the near future. It is an important factor related to the level of anxiety novice counselors experience as well as the amount of effort they put forth to learn advanced counseling behaviors (Larson, 1998). As a result, some counselor educators and researchers have suggested that increasing counseling trainees’ self-efficacy is a worthwhile training goal (Larson et al., 1992) and that examinations of this construct should be included in both the research and evaluation of counselor competency and training effectiveness (Yuen, Chan, Lau, Lam, & Shek, 2004).

In a review of the CSE literature, numerous training interventions such as academic coursework, assigned readings, classroom discussions, self-reflection, modeling, supervision and hands on experiences associated with practica and internships all have been found to help improve competency and perceptions of self-efficacy (Easton, Martin, & Wilson, 2008; Larson & Daniels, 1998). These activities provide learning experiences that facilitate task performance and vicarious learning which Bandura (1986) noted as critical to the development of self-efficacy. Although these activities have become commonplace in traditional FTF courses, their effectiveness in developing student competency when employed in online courses is unclear. Given that CSE has been demonstrated to play a central role in counselor trainee development (Stoltenberg, McNeill, & Delworth, 1998) it will be interesting to note how the implementation of new online instructional methods impacts the development of CSE beliefs among student trainees. With this in mind, the purpose of the current study was to compare the counseling self-efficacy beliefs of students enrolled in both traditional FTF and online counselor education programs. In particular, it was hypothesized that students who enrolled in traditional FTF courses would report greater perceptions of counseling self-efficacy than those enrolled in online courses.

Method

Participants
To recruit participants, a random sample of 1,000 graduate student members of the American Counseling Association (ACA) obtained from ACA Member Services received an invitation to participate in this research study. Overall, 429 individuals responded to the invite, resulting in a 42.9% response rate. A total of 56 surveys were discarded from further analysis due to significant missing data or the fact that the participant was no longer a graduate student. This resulted in a final sample size of 373 individuals being used in this study.

Of the 373 sample participants, 307 (82.3%) were women, 63 (16.9%) were males, and 3 participants (0.8%) did not indicate their gender. The average participant was 37.12 years of age (SD = 10.96). When asked to report ethnicity, 45 (12.1%) participants identified as African American, 5 (1.3%) identified as Asian-Americans, 11 (2.9%) identified as Hispanic, 5 (1.3%) identified as Native American and 307 (82.3%) identified as Caucasian. Two hundred and ninety seven students reported working on their master’s degree and 77 reported enrollment in an advanced degree (specialist or doctorate) program. To establish comparison groups, participants were asked to indicate whether or not they had taken their core skill development courses (e.g., helping skills, group counseling, assessment, practicum, internship) required for their graduate counseling program of study online. For this study, an online course was defined as one in which at least 80% of the course content was delivered via the Internet (Allen & Seaman, 2010). Using this criterion, 207 students reported no prior online enrollment and 166 reported previous online enrollment in core skill development courses.

Instrumentation
To assess the counseling self-efficacy of counselor trainees, participants were asked to complete the Counseling Self Estimate Inventory (COSE). The COSE (Larson et al., 1992) consists of 37 self-report items, most of which are worded as “I” statements, that require trainees to rate the degree to which they estimate how well they will perform as counselors in future counseling sessions. Each of the items are rated on a 6-point Likert scale with values ranging from 1 (strongly disagree) to 6 (strongly agree). To prevent the influence of response set bias, both positively and negatively worded items are included, with the negatively worded items reverse scored so that higher scores would indicate a higher estimate of one’s counseling self-efficacy.

Overall, the COSE provides scores for a single higher order factor and five second order factors which were identified through an initial exploratory factor analysis. However, in a review of the literature, Larson et al. (1992) recommended that researchers use the single higher order factor score rather than the second order factor scores when assessing overall counseling self-efficacy. Based on their suggestion, only the full scale COSE score was used in this study. An initial report by Larson et al. noted an internal consistency of .93 for the COSE and a three-week test-retest reliability of .87. A reliability analysis conducted to examine the internal consistency of the COSE instrument for the current study yielded an alpha coefficient of .94, suggesting that the COSE was a reliable instrument for the given sample.

Procedures
Following institutional review board approval, electronic invitations to participate in this study were sent to a random sample of graduate students enrolled in counselor training programs nationwide using the contact information made available by ACA member services. Included in the study invitation were a brief overview of the study and a description of what individuals would be asked to do if they chose to participate. To make access to the survey more convenient, an embedded hyperlink was included in the electronic invitation. All individuals accessing the study site were asked initially to read an informed consent document and indicate their agreement to participate before proceeding. To protect participant anonymity, the survey did not ask for any personally identifying information. Participants were encouraged to answer as honestly as possible. The majority of participants were able to complete the survey in less than 20 minutes.

Data Analysis
An analysis of covariance (ANCOVA) was computed using SPSS 16.0 to compare mean differences in CSE scores by instructional method (online vs. traditional FTF instruction) while controlling for students’ previous counseling experience. Previous counseling experience was thought to be related to the dependent variable (DV) because researchers (see Larson et al., 1992) have noted a strong positive relationship between CSE and counseling experience in prior studies. Current student data also indicates that a significant number of students enrolled in online programs are working professionals (Ivey, 2011), thus the likelihood exists that many of the participants in this study may currently be working in the counseling field or have done so previously. Therefore, following Warner’s (2008) recommendation that variables strongly correlated with the DV be included as covariates in any statistical model because they produce a smaller error term and a larger F ratio for assessing the main effect of the treatment variable, a measure of previous counseling experience (defined as any contact with clients in a professional mental health-related role) was added to the analysis. Finally, a multiple regression analysis was computed to determine the best linear combination of variables among gender, age and degree level (master’s or advanced) for predicting COSE scores.

Results

Before running the ANCOVA, a preliminary analysis was conducted to test the homogeneity of slopes assumption by examining the interaction between the treatment variable (instructional method) and the covariate (previous counseling experience). Results indicated that the interaction was not statistically significant, F(1, 369) = .498, p = .48, and the assumption had been met. Having satisfied the homogeneity of slopes assumption, an ANCOVA was performed to determine whether a difference in counseling self-efficacy existed between students who had completed FTF instruction and online instruction courses after controlling for the variance accounted for by previous counseling experience. The results indicate that there was a significant difference in COSE scores, F(1, 370) = 4.61, p = .03, η2 = .02. A comparison of the adjusted group means, as displayed in Table 1, reveals that students who enroll in online counselor education courses self-report significantly stronger CSE beliefs. According to Cohen (1988), η2 effect sizes can be interpreted as being either small (.01), medium (.06), or large (.17). Using these benchmarks, the computed effect size of .02 would be categorized as small in terms of the variance accounted for in COSE scores by instructional method.

Discussion

The purpose of this exploratory study was to examine whether differences in counseling self-efficacy exist between students enrolled in FTF and online core counseling skill courses after controlling for the effect of previous counseling experience. It was hypothesized that students enrolled in the online courses would self-report lower levels of counseling self-efficacy. To test this hypothesis, students enrolled in both types of courses were administered the Counseling Self Estimate Inventory (COSE) and their results compared. The results of an analysis of covariance suggest that students who enrolled in online core counseling skill courses as part of their training program reported stronger counseling self-efficacy beliefs than the students who were trained in traditional FTF courses when prior counseling experience was controlled, thus disproving our research hypothesis.

There are a number of explanations that can be posited for this unexpected result. For one, the results may be a function of the manner in which the data was collected. By design, the COSE is a self-report measure. As such, students are asked to give their perceptions of their abilities as a counselor. According to Kruger & Dunning (1999), there is a natural tendency for individuals to overrate their abilities in the absence of any specific skill demonstration on which to accurately judge their own level of competence. As Lepkowski, Packman, Smaby, and Maddux (2009) note, the unrealistic sense of confidence some individuals have may cause them to attempt counseling tasks for which they are not fully trained nor prepared. One of the biggest criticisms of online instruction in training counselors is that it does not afford students adequate opportunity to develop their basic counseling skills. Typically, online students have fewer structured opportunities to gain practical experience applying the skills and concepts they are learning in their courses. As a result, they may feel confident in their abilities despite the fact they lack the actual hands-on training and experiences needed to appropriately validate their perceptions.

Another explanation could be that online instructional methods might require students to employ learning strategies that are more conducive to improving self-efficacy. Previous research suggests that both the structure and content of online courses often make them conducive to self-paced learning (Appleton & Orr, 2000; Vrasidas & McIsaac, 2000), allowing students to work at a pace that provides them with the opportunity to develop a deeper understanding of the course content (Biggs, 1999). In addition, the increased autonomy inherent in the online environment has been found to positively relate to a higher sense of intrinsic motivation in many students (Wadsworth, Husman, Duggan, & Pennington, 2007). In other words, those students who enroll in online courses may be more invested in the learning process and thus more likely to put forth the additional effort required to be successful. In so doing, they might experience a greater sense of confidence in their ability to effectively counsel clients in the future.

In terms of the multiple regression analysis performed, it was found that both age and degree level were significant predictors of students’ counseling self-efficacy beliefs. Specifically, older students and those who were working on an advanced degree (specialist or doctorate level) appeared to be more confident in their counseling abilities. A possible explanation could be that these students have a greater pool of experiences, both personal and professional, to draw upon when working with clients. Melchert, Hays, Wiljanen, and Kolocek (1996) found that both level of training and clinical experience positively contributed to students reporting significantly higher self-efficacy beliefs for counseling skills. It could be that more advanced students feel increasingly comfortable dealing with a wider variety of issues and client populations than their less experienced colleagues. The fact that gender was not a significant predictor appears to support previous research on gender differences in self-efficacy. As noted by Lepkowski et al. (2009), initial gender differences found early in the training process appear to dissolve as additional experience is gained.

Limitations and Recommendations for Future Research

Limitations and caveats need to be noted. First, although a significant effect for method of instruction was found, the small effect size computed could potentially mitigate the relative importance of this finding. Second, differences in teaching style of the various instructors of the students surveyed may serve as a confounding variable affecting the interpretation of these results. It could be that any differences noted in students’ self-efficacy beliefs could be related more to the efforts of the instructor than the instructional platform used to deliver the course material. In this study, faculty status (full-time or part-time/adjunct) was not addressed. For many online programs adjunct faculty are employed. While many are practicing counselors and may possess more current clinical experience than their full-time educator peers, their investment in the total development of their students may be less than that of full-time faculty members who spend significantly more time training and developing students to become counseling professionals. Additionally, while some instructors may be implementing online components to their courses for the first time, others may be more experienced and thus using more advanced technologies that allow them to better simulate actual counseling experiences for students and create a more interactive and dynamic learning environment. Finally, sampling issues need to be addressed. In the current study participants were recruited from a mailing list of current graduate students provided by the American Counseling Association. When they sign up for membership, individuals have the opportunity to make their contact information available or not. Since the participants in this study were limited to students who chose to make their contact information available, complete random sampling was not possible, thus limiting the generalizability of the results.

Recommendations for Research and Practice

The results of this study suggest that students enrolled in online core counseling skills courses report higher perceptions of counseling self-efficacy than those taking traditional FTF courses. While, depending on perspective, this result is encouraging for proponents of online education, it suggests that further research is warranted. In particular, future researchers seeking to expand on this initial exploration should consider examining the influence of specific course content as well. In this study, students were separated into groups based on whether or not they enrolled in online core graduate counseling skills courses. There was no control for the content of these courses or the types of learning experiences they would be exposed to for either method. An additional suggestion would be that counselor educators and researchers develop a better understanding of the types of students who choose to enroll in online programs. As noted earlier, many students who enroll in online programs already have experience in their chosen profession and may feel more comfortable and confident with their abilities as a counselor. Although this logic appears rational, research supporting this belief does not exist and should be addressed in future studies. A final suggestion would be to include students who have completed entire programs online. In this study, the majority of students in the online course group also had completed traditional FTF courses as well. As more universities begin offering complete online programs and these programs gain accreditation status, it will be interesting to note how the counseling self-efficacy beliefs of students in these programs relate to those who are enrolled in more traditional programs.

Without a doubt, the benefits of using online instruction are numerous for both educators and students. When used appropriately, it has the potential to profoundly impact and enhance counselor training and ultimately the counseling services provided to clients (Chandras, 2000). However, those wishing to integrate online elements into their courses are urged to do so cautiously. Not all courses may be suited for total online presentation. Counselor educators are therefore encouraged to be mindful of the ways in which they attempt to integrate elements of online instruction into their courses. As Bentley (2007) notes, “because counselors play a crucial role in helping individuals deal with a vast array of mental health and developmental issues, it is critically important that they are well prepared upon entering the profession to work effectively with clients and provide quality services” (p. 1). As gatekeepers for the profession, counselor educators have an ethical mandate to ensure that the students they train are not entering the profession with a sense of ability that exceeds their skill level so that they do not harm the clients they will serve. Thus, the decision to incorporate online instruction should be based on its ability to provide students with the platform to further develop their clinical skills and not simply on the novelty of the approach. As for students, they are urged to consider their own personal learning style and evaluate whether or not it is in agreement with the structure of an online format. Despite the benefits an online learning environment may provide, the evaluation of its effectiveness still remains not only a personal decision, but one that requires the attention of counselor educators, and accreditation and state licensure boards.

References

Albrecht, A. C., & Jones, D. G. (2001). High tech/high touch: Distance learning in counselor preparation. Alexandria, VA: Association for Counselor Education and Supervision.
Allen, I. E., & Seaman, J. (2010). Learning on demand: Online education in the United States, 2009. Retrieved from http://sloanconsortium.org/publications/survey/pdf/learningondemand.pdf
Appleton, M., & Orr, D. (2000). Meeting the needs of distance education students. In C. Bruce and P. Candy (Eds.). Information literacy around the world: Advances in programs and research. Wagga Wagga, Australia: Charles Sturt University Press.
Artino, A. R., & McCoach, B. D. (2008). Development and initial validation of the online learning value and self-efficacy scale. Journal of Educational Computing Research, 38, 279–303. doi: 10.2190/EC.38.3.c
Baltimore, M. L. (2000). Ethical considerations in the use of technology for marriage and family counselors. The Family Journal: Counseling and Therapy for Couples and Families, 8, 390–393. doi: 10.1177/1066480700084010
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Upper Saddle River, NJ: Prentice Hall.
Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist, 44, 1175–1184. doi:10.1037//0003-066X.44.9.1175
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman.
Barnes, K. L. (2004). Applying self-efficacy theory to counselor training and supervision: A comparison of two approaches. Counselor Education & Supervision, 44, 56–69.
Bejerano, A. R. (2008). The genesis and evolution of online degree programs: Who are they for and what have we lost along the way? Communication Education, 57, 408–414.
Benshoff, J. M., & Gibbons, M. M. (2011). Bringing life to e-learning: Incorporating a synchronous approach to online teaching in counselor education. The Professional Counselor: Research and Practice, 1, 21–28. Retrieved from https://tpcwordpress.azurewebsites.net/wp-content/uploads/2011/03/Benshoff.pdf
Bentley, P. D. (2007). Mindfulness and counseling self-efficacy: The mediating role of attention and empathy. Retrieved from Dissertations and Theses database. (AAT 3273312).
Bernard, J. M., & Goodyear, R. K. (2008). Fundamentals of clinical supervision (4th ed.). Boston, MA: Allyn & Bacon.
Biggs, J. B. (1999). Teaching for quality learning at university: What the student does. Buckingham: Society for Research into Higher Education and Open University Press.
Bradley, C., & Fiorini, J. (1999). Evaluation of counseling practicum: National study of programs accredited by CACREP. Counselor Education and Supervision, 38, 62–64.
Chandras, K. (2000). Technology-enhanced counselor-training: Essential technical competencies. Journal of Instructional Psychology, 27, 224–227.
Chester, A., & Glass, C. (2006). Online counseling: A descriptive analysis of therapy services on the Internet. British Journal of Guidance and Counseling, 34, 145–160. doi:10.1080/03069880600583170
Clark, M. A., & Stone, C. B. (2002). Clicking with students: Using online assignments in counselor education courses. Journal of Technology in Counseling, 2. Retrieved from http://jtc.columbusstate.edu/vol2_2/clarkstone.htm
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
DeTure, M. (2004). Cognitive style and self-efficacy: Predicting student success in online distance education. The American Journal of Distance Education, 18, 21–38.
Easton, C., Martin, W. E., & Wilson, S. (2008). Emotional intelligence and implications for counseling self-efficacy: Phase II. Counselor Education and Supervision, 47, 218–232.
Ellis, R. A., Ginns, P., & Piggott, L. (2009). E-learning in higher education: Some key aspects and their relationship to approaches to study. Higher Education Research & Development, 28, 303–318.
Granello, P. (2000). Integrating wellness work into mental health private practice. Journal of Psychotherapy in Independent Practice, 1, 3–16. doi: 10.1300/J288v01n01_02
Greenidge, W. L., & Daire, A. P. (2005). The application of gaming technology in counselor training programs. Journal of Technology in Counseling, 4. Retrieved from http://jtc.columbusstate.edu/Vol4_1/Daire2/Daire2.htm
Hayes, B. G., & Robinson III, E. H. (2000). Assessing counselor education students’ attitudes toward computers and multimedia instruction. Journal of Humanistic Counseling, Education & Development, 38, 132–141.
Hensley, L. G., Smith, S. L., & Thompson, R. W. (2003). Assessing competencies of counselors-in-training: Complexities in evaluating personal and professional development. Counselor Education & Supervision, 42, 219–230.
Ivey, E. (2011). Online education and the non-traditional university student. Retrieved from: http://c21u.gatech.edu/online-education-and-non-traditional-university-student
Karper, C. M., Robinson, E. H., & Casado, M. (2005). Computer-assisted instruction and academic achievement in counselor education. Journal of Technology in Counseling, 4. Retrieved from http://jtc.columbusstate.edu/Vol4_1/Karper/Karper.htm
Keeton, M. T. (2004). Best online instructional practices: Report of phase I of an ongoing study. Journal of Asynchronous Learning Networks, 8, 75–100.
Krieger, K. M., & Stockton, R. (2004). Technology and group leadership training: teaching group counseling in an online environment. The Journal for Specialists in Group Work, 29, 343–359. doi:10.1080/01933920490516044
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own competence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77, 1121–1134.
Larson, L. M. (1998). The social cognitive model of counselor training. The Counseling Psychologist, 26, 219–273. doi:10.1177/0011000098262002
Larson, L. M., & Daniels, J. A. (1998). Review of the counseling self-efficacy literature. The Counseling Psychologist, 26, 179–218. doi:10.1177/0011000098262001
Larson, L. M., Suzuki, L. A., Gillespie, K. N., Potenza, M .T., Bechtel, M. A., & Toulouse, A. L. (1992). Development and validation of the Counseling Self-Estimate Inventory. Journal of Counseling Psychology, 39, 105–120. doi:10.1037/0022-0167.39.1.105
Lepkowski, W. J., Packman, J., Smaby, M. H., & Maddux, C. (2009). Comparing self and expert assessments of counseling skills before and after skills training, and upon graduation. Education, 129(3), 363–371.
Lim, J., Kim, M., Chen, S. S., & Ryder, C. E. (2008). An empirical investigation of student achievement and satisfaction in different learning environments. Journal of Instructional Psychology, 35, 113–119.
Melchert. T. P., Hays., V. L., Wiljanen, L. M., & Kolocek, A. K. (1996). Testing models of counselor development with a measure of counseling self-efficacy. Journal of Counseling & Development, 74, 640–644. doi:10.1002/j.1556-6676.1996.tb02304.x
Murphy, L., MacFadden, R., & Mitchell, D. (2008). Cybercounseling online: The development of a university-based training program for e-mail counseling. Journal of Technology in Human Services, 26(2), 447–469. doi:10.1080/15228830802102081
Myers, J. E., & Gibson, D. M. (1999). Technology competence of counselor educators. Counseling and Personnel Services (CG029454), 1–11.
National Center for Education Statistics (2008). Distance education at degree-granting postsecondary institutions 2006–07. Retrieved from http://nces.ed.gov/fastfacts/display.asp?id=80
Nelson, J. A., Nichter, M., & Henriksen, R. (2010). On-line supervision and face-to-face supervision in the counseling internship: An exploratory study of similarities and differences. Retrieved from http://counselingoutfitters.com/vistas/vistas10/Article_46.pdf
O’Hanlon, N. (2001). Development, delivery, and outcomes of a distance course for college students. Library Trends, 50, 8.
O’Malley, J., & McCraw, H. (1999). Student’s perceptions of distance learning, online learning and the traditional classroom. Online Journal of Distance Learning Administration, 2(4). Retrieved from http://www.westga.edu/~distance/omalley24.html
Patrick, P. K. S. (2005). Online counseling education: Pedagogy controversies and delivery issues. In G. R. Walz, & R. K. Yep (Eds.), Compelling perspectives on counseling (pp. 239–242). Alexandria, VA: American Counseling Association.
Quinn, A. C., Hohenshil, T., & Fortune, J. (2002). Utilization of technology in CACREP approved counselor education programs. Journal of Technology in Counseling, 2(2). Retrieved from http://jtc.colstate.edu/Vol2_2/quinn/quinn.htm
Stoltenberg, C. D., McNeil, B., & Delworth, U. (1998). IDM supervision. San Francisco, CA: Jossey-Bass.
Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., & Liu, X. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76, 93–135. doi:10.3102/00346543076001093
Tyler, M. J., & Sabella, R. A. (2004). Using technology to improve counseling practice: A primer for the 21st century. Alexandria, VA: American Counseling Association.
Vrasidas, C., & McIsaac, M. S. (2000). Principles of pedagogy and evaluation for web-based learning. Educational Media International, 37, 105–112. doi:10.1080/095239800410405
Wadsworth, L. M., Husman, J., Duggan, M. A., & Pennington, M. N. (2007). Online mathematics achievement: Effects of learning strategies and self-efficacy. Journal of Developmental Education, 30, 6–14.
Wantz, R. A., Tromski, D. M., Mortsolf, C. J., Yoxtheimer, G., Brill, S., & Cole, A. (2003). Incorporating distance learning into counselor education programs: A research study. In J.W. Bloom & G. R. Walz (Eds.), Cybercounseling & cyberlearning: An encore (pp. 327–344). Alexandria, VA: American Counseling Association.
Warner, R. (2008). Applied statistics: From bivariate through multivariate techniques. Thousand Oaks, CA: Sage.
Waschull, S. B. (2005). Predicting success in online psychology courses: Self-discipline and motivation. Teaching of Psychology, 32, 3. doi:10.1207/s15328023top3203_11
Yuen, M., Chan, R., Lau, P., Lam, M. P., & Shek, D. T. (2004). The Counseling Self-Estimate Inventory (COSE): Does it work in Chinese counselors? Counseling Psychology Quarterly, 17, 177–194. doi:10.1080/09515070410001728280

Joshua C. Watson, NCC, is an Associate Professor at Mississippi State University-Meridian. Correspondence can be addressed to Joshua C. Watson, Mississippi State University-Meridian, 1000 Highway 19 N, Meridian, MS 39307,
jwatson@meridian.msstate.edu.

Predicting Student Outcome Measures Using the ASCA National Model Program Audit

Oct 6, 2014 | Article, Volume 2 - Issue 2

Lauren E. Palmer, Bradley T. Erford

This study explored the prediction of student outcome variables from the ASCA national model level of program implementation. A total sampling of schools from two suburban school districts was conducted. Outcome variables were measures of math and reading achievement scores, attendance and graduation rates. Such measures play a central role in promoting school counselors as an integral part of the educational process.

Keywords: ASCA national model, outcome variables, attendance, graduation rates, achievement scores

At a time when accountability within the school counseling profession is at the forefront, school counselors are required to present evidence which validates the effectiveness of daily practices. The American School Counselor Association (ASCA) created a framework for implementing comprehensive developmental school counseling programs through specification of standards and competencies. But does degree of implementation of the ASCA National Model (2005) affect the ability of school counselors to meet student needs? This assertion is supported by correlative or indirect research which indicates the positive effects of fully implemented school counseling programs (Brigman & Campbell, 2003; Carrell & Carrell, 2006; Lapan, Gysbers, & Kayson, 2007; Lapan, Gysbers, & Petroski, 2001; Lapan, Gysbers, & Sun, 1997; McGannon, Carey, & Dimmitt, 2005; Nelson, Gardner, & Fox, 1998; Sink, 2005; Sink & Stroh, 2003; Whiston & Wachter, 2008).

A focus on attaining the goals of a comprehensive program is essential to initiate systemic change and to establish the school counseling program as an integral part of the total educational process. School counselors develop and refine their roles in order to meet the diverse needs of students and the school community. Transitioning from the traditional guidance program, or no program at all, to a comprehensive, developmental school counseling program is a demanding task, but is attainable through collaboration among school counselors and stakeholders. A program audit is a fundamental step in this process as well as in evaluating where a counseling program currently is and establishing where the program aims to go in the future.

ASCA (2005) presented a standardized framework for creating a comprehensive school counseling program that supports the academic, career and personal/social development of students throughout their academic careers. According to ASCA, a school counseling program is comprehensive, preventative and developmental in nature. This framework provides school counselors with an all-inclusive approach to program foundation, delivery, management and accountability. Similar to the Education Trust (2009) and College Board (2009), ASCA promotes a new vision for the school counseling profession which reflects accountability, advocacy, leadership, collaboration, and systemic change within schools, positioning professional school counselors as essential contributors to student success.

The extant literature reveals much support for the positive benefits of school counseling programs for students. As a result of fully implemented school counseling programs, students enjoy higher grades (Lapan et al., 2001; Lapan et al., 1997), better school climate (Lapan et al., 1997), higher satisfaction with education (Lapan et al., 2001; Lapan et al., 1997), more relevant education (Lapan et al., 2001), higher ACT scores (Nelson et al., 1998), and greater access to more advanced math, science, technical and vocational courses (Nelson et al., 1998). Studies also have provided evidence of fewer classroom disruptions and improved peer behavior among students who participated in comprehensive school counseling programs (Brigman & Campbell, 2003; Lapan, 2001; Lapan et al., 2007; Lapan et al., 1997; Sink, 2005; Sink & Stroh, 2003).

The program audit is an evaluation tool used to determine the extent to which components of a comprehensive program are implemented and helps to make decisions concerning future directions that a school counseling program will take. A program audit, or process evaluation, assists school counselors in implementing the standards and components of a comprehensive school counseling program, in addition to identifying areas for improvement or enhancement (ASCA, 2005). ASCA has suggested that a program audit be completed annually to determine the strengths and weaknesses evident within a school counseling program with regard to the four main elements of the ASCA National Model: foundation, delivery system, management system and accountability. Specific criteria under each component are used as a way to evaluate implementation of the school counseling program.

This study examined the prediction of student outcomes, including achievement scores, attendance and graduation rate, using level of implementation of the ASCA National Model (2005) as a predictor variable. It was hypothesized that level of program implementation would be a significant predictor of student outcomes at each of the three school levels: elementary, middle and high school. The study also determined coefficients alpha for the ASCA Program Audit for the total sample and each academic level.

Method

Participants
A nonrandomized cluster sampling of two public school districts located in Maryland was conducted to select participants for the study. These two public school systems housed a total of 111 elementary schools, 30 middle schools, and 23 high schools for a total of 164 schools. Each participating school had at least one professional school counselor and a school counseling program in place. In the instances where multiple school counselors were assigned to schools, the data were provided by the guidance chair or lead counselor. School counselors from two alternative schools responded, but were eliminated from the sample due to dissimilarity with the traditional high schools and small sample size. Thus, a total of 78 (70%) elementary schools, 17 (57%) middle schools, and 18 (78%) high schools participated for a total sample of 113 schools (69%) within the two participating school districts.

Instrument
The ASCA Program Audit (ASCA, 2005) served as an independent variable for this study. The audit takes approximately 30–45 minutes to complete the 115 prompts and uses a Likert-type scale to evaluate the components of a counseling program along the continuum of “None” (meaning not in place), “In progress” (perhaps begun, but not completed), “Completed” (but perhaps not as yet implemented), “Implemented” (fully implemented), or “Not applicable” (for situations where the component does not apply). For the purposes of this study, these response choices were coded 0, 1, 2, 3, and 0, respectively. Once a program audit is completed, the information can be used to determine implementation strengths of the program, areas of the program which need strengthening, and short-range and long-range goals for implementation improvement.

This is the first published study to use the complete ASCA Program Audit as a study independent variable. One other study, a dissertation (Wong, 2008), used facets of the ASCA Program Audit as an independent variable. Wong constructed a survey which was modified from the ASCA Program Audit in a study designed to describe the relationship between comprehensive school counseling programs and school performance. Wong’s use of regression analysis yielded a positive relationship and predictive model between these two variables, but no information regarding the psychometric characteristics of the scale. Internal consistency information from the current study’s sample is provided in the results section.

Procedures
The method used to select participants was a nonrandomized cluster sampling of two districts from among 24 public school districts located in Maryland. Once IRB approval was received, letters were mailed out over the summer and early in the academic year to school counselors of elementary, middle and high schools within each of the two school districts selected for participation. Inclusion of school counselor supervisors assisted in the distribution and administration of this study and increased return rates of completed program audits. The school counselors of each participating school were provided with the program audit from the ASCA National Model (2005), a statement of rationale for the study and a consent form. The school counselors completed the program audit during the months of June through February with instructions to retrospectively evaluate implementation of the school counseling program components at the end of the previous (2009-2010) academic year. Demographic data, graduation rates, attendance and scores from the Maryland State Assessment (MSA) for grades 5, 8 and 10 were obtained from 2009-2010 Maryland Report Cards as retrieved from the Maryland State Department of Education website (http://mdreportcard.org/).

The dependent variable of achievement was measured using MSA math and reading scores and defined operationally as the percentage of those students of a given grade not meeting the criterion for passing (i.e., percentage of students receiving only basic scores), separately for the reading and math components. The MSA is administered to students in grades 3–5 at the elementary level, grades 6–8 at the middle school level and during the 10th grade in high school. Fifth grade scores, 8th grade scores and 10th grade scores (English and algebra) were used for these analyses, reasoning that these scores reflected the cumulative intervention of prolonged exposure to the school’s curricular experience.

The dependent variable of attendance was defined as the percent of average daily attendance including ungraded students in special education programs (Maryland State Department of Education, 2010). The dependent variable of graduation rate was defined by MSDE as the percentage of students who received a Maryland high school diploma during the school year. More specifically, the graduation rate is calculated by “dividing the number of high school graduates by the sum of the dropouts for grades 9 through 12, respectively, in consecutive years, plus the number of number of high school graduates (MSDE, 2010, para 1).” Since graduation rate and dropout rate in this sample were highly correlated (r = -.752, p < .001, n = 18), graduation rate was used in the analysis, while dropout rate was excluded as redundant.

Analysis
The data from the demographic and program audit forms were coded into an SPSS database. The total audit score was used to determine the level of program implementation. Data marked as “N/A” or “none” were coded as 0 to reflect no attempt at implementation, even though the actual audit reported them separately. “In progress” was coded as a 1, “completed” was coded as a 2, and “implemented” was coded as a 3. The total audit score was the simple sum of scores for the 115 responses. Appropriate Pearson family correlation coefficients were applied to analyze relationships between the total audit score (program implementation), student-to-counselor ratio and school outcome measures. Simple linear regression analyses were conducted to determine whether degree of model program implementation was a significant predictor of student outcomes of achievement scores, attendance and graduation rate at each level: elementary, middle and high school.

Results
Of the 164 schools in the two participating school districts, 115 (70%) returned completed consent, demographic and program audit forms for analysis. Two high schools were eliminated because they were designated alternative schools. Thus, a total participation rate of 113 schools (69%) was obtained. Type I error (α) was set at the .05 level of probability for all analyses. Trends were indicated by probability levels of p < .10. Effect sizes for r or R were interpreted as follows (Cohen, 1988): .10 indicated a small effect; .30 indicated a medium effect; and .50 indicated a large effect.

This study provides the first reported analysis of internal consistency of a program audit (ASCA, 2005). Internal consistency was measured using Cronbach’s coefficient alpha. Alphas were calculated to determine the level of internal consistency of the total scale and each of the 17 sections of the program audit on the current total sample (n = 113), and separately for the elementary (n = 78), middle (n = 17) and high school (n = 18) samples. Table 1 provides a summary of these coefficients alphas for the total sample and disaggregated by elementary, middle and high school samples. For the total scale of 115 items, the α of .98 indicated an extraordinarily high degree of internal consistency. The program audit yielded an adequate degree of internal consistency for all 17 sections, ranging from α = .69–.99 for the total sample, with a median α of .89.

Correlation coefficients were calculated between the predictor and outcome variables and presented in Table 2 for the elementary and middle schools, and Table 3 for the high school samples. A cursory inspection of the outcome variables indicates strong intercorrelations, yielding magnitudes of r >.50 in all instances, which are large effect sizes. Correlations between the program audit predictor variable and outcome measures at the elementary and middle school levels were not significant (p > .05, see Table 2) and yielded small effect sizes ranging from .10 to .20 (adjusted for directional effects). However, at the high school level (see Table 3), significant correlations and large effect sizes were noted between the program audit predictor variable and high school outcome measures. Descriptive statistical analysis indicated that all variables were normally distributed with the one exception: the elementary reading outcome had a skewness index of 1.42.

To assess the proportion of variance in outcomes that could be uniquely accounted for by the ASCA program audit, simple linear regression analysis was used to test the hypothesis that a program audit would significantly predict each outcome variable at each of the three school levels (elementary, middle and high). These regression results are presented in Table 4. Program audit scores did not significantly predict any student outcome measure scores at either the elementary or middle school level, although the prediction of fifth grade reading achievement trended toward significance (p = .08). However, at the high school level (n = 18), program audit predicted English (β = -.645, t = -3.27, p < .05, large effect), algebra (β = -.517, t = -2.34, p < .05, large effect), and attendance (β = .506, t = 2.35, p < .05, large effect) outcomes, and a trend was noted in the prediction of high school graduation rate (β = .442, t = 1.97, p = .06, medium to large effect).

Discussion

The purpose of the study was to determine whether level of ASCA National Model (2005) program implementation predicted student outcomes (i.e., achievement scores, attendance and graduation rates). Results indicated that no prediction was observed at either the elementary or middle school levels on any of the outcome variables (reading, math or attendance). At the high school level, the data showed that as program implementation increased the percentage of students scoring basic on the MSA English and algebra decreased, which is a positive result. Likewise, at the high school level when program implementation increased, so did attendance and the related trend of increased high school graduation rates. Thus, the hypothesis that higher program implementation would predict better student achievement received mixed support and suggested a need for high school counselors to implement comprehensive developmental programs in order to benefit all students and improve important school and student outcomes.

Why these high school findings were not replicated at the elementary and middle school levels is puzzling, as the extant literature demonstrates a significant relationship between program implementation and student outcomes at all levels of schooling. One explanation may lie in the samples used for this study. The sample sizes used at the middle and high school levels were small, 17 and 18, respectively, reducing the power of the analyses, while the elementary sample was much larger (n = 78). A cursory inspection of the means and standard deviations from these three samples (see Table 1) indicates that the elementary sample had the lowest level of overall program implementation and the largest spread in scores (M = 215.1, SD = 66.3), compared to middle school (M = 222.4, SD = 53.7) and high school (M = 243.6, SD = 47.5). However, usually greater variations in scores lead to better predictions.

Study Limitations and Areas for Future Research

Additional inquiry regarding the implementation of comprehensive school counseling programs and student outcomes is necessary to determine the link between student outcomes and school counseling services. Some researchers have pointed out that previous investigations into this area of study have yielded deceiving results (Brown & Trusty, 2005; McGannon et al., 2005). For example, many of the studies used research designs and procedures that did not justify a causal relationship between counseling programs and positive outcomes. Indeed, the present study was correlational in nature, so causative inferences cannot be made. This study did not use a controlled treatment intervention and cannot determine a causal relationship between level of program implementation and more positive student outcomes. The small sample sizes of the middle school and high school counselors may have affected the results as well.

Various confounding variables exist in the current and previous studies, such as other co-occurring educational programs, and school organizational structure and leadership, all of which tend to influence academic achievement. Moreover, some of the data collected within these previous studies are self-reported and not cross-validated with multiple sources of information or informants. In studies that compare counseling programs and student achievement, Berliner and Biddle (1995) noted that researchers often fail to control for pupil expenditure, which is not always equivalent to socio-economic status as many presume because of high correlations (Brown & Trusty, 2005). Failure to control for socio-economic status also can confound the results which may be a factor in this study, although using only two large school systems may have provided some control for per pupil expenditure rates.

McGannon et al. (2005) emphasized the need for standardized achievement scores and other institutional data, intervention effect sizes and a measure of the quality of implementation of the program to be included in future studies to ensure worthy findings. Brown and Trusty (2005) recommended the use of proximal outcomes which include the target of interventions used with students (e.g., the development of specific ASCA competencies). Instead of using proximal outcomes, Brown and Trusty pointed out the overuse of distal outcomes (e.g., ACT scores, achievement test scores, school grades) which are affected by a number of factors rather than as a direct result of school counselor services. While proximal outcomes such as developing competencies including those within the ASCA model may be beneficial to report, the methods used to establish these competencies also becomes the focus of scrutiny.

Longitudinal and experimental design studies which include control and treatment groups are necessary to establish causal relationships. Correlational studies are often selected as the analysis tool of choice because of expediency and ease. Longitudinal studies take years to complete and are subject to student attrition. Experimental studies in schools also are complicated by trying to locate a school willing to serve as the control group (i.e., a school that does not have a counseling program in place or a school counselor on staff).

Outcome research plays a central role in promoting school counselors as an integral part of the educational process. It is critical for school counselors to use interventions and program components which provide positive student outcomes (McGannon et al., 2005) and to be knowledgeable of current research relevant to their position and the population they serve.

References

American School Counselor Association. (2005). The ASCA national model: A framework for school counseling programs (2nd ed.). Alexandria, VA: Author.
Berliner, D. C., & Biddle, B. J. (1995). The manufactured crisis: Myths, fraud, and the attack on America’s public schools. Reading, MA: Addison-Wesley.
Brigman, G., & Campbell, C. (2003). Helping students improve academic achievement and school success behavior. Professional School Counseling, 7, 91–99.
Brown, D., & Trusty, J. (2005). School counselors, comprehensive school counseling programs, and academic achievement: Are school counselors promising more than they can deliver? Professional School Counseling, 9, 1–8.
Carrell, S. E., & Carrell, S. A. (2006). Do lower student to counselor ratios reduce school disciplinary problems? Contributions to Economic Analysis & Policy, 5, 1–24.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
College Board. (2009). About us: The College Board. Retrieved from http://www.collegeboard.com/about/index.html
Education Trust. (2009). About the Education Trust. Retrieved from http://www.edtrust.org/dc/about
Lapan, R. T. (2001). Results-based comprehensive guidance and counseling program: A framework for planning and evaluation. Professional School Counseling, 4, 289–299.
Lapan, R. T., Gysbers, N. C., & Kayson, K. (2007). How implementing comprehensive guidance programs improves academic achievement for all Missouri students. Jefferson City, MO: Missouri Department of Elementary and Secondary Education, Division of Career Education.
Lapan, R. T., Gysbers, N. C., & Petroski, G. (2001). Helping 7th graders be safe and academically successful: A statewide study of the impact of comprehensive guidance programs. Journal of Counseling and Development, 79, 320–330.
Lapan, R. T., Gysbers, N. C., & Sun, Y. (1997). The impact of more fully implemented guidance programs on the school experiences of high school students: A statewide evaluation study. Journal of Counseling and Development, 75, 292–302.
McGannon, W., Carey, J., & Dimmitt, C. (2005). The current status of school counseling outcome research. Amherst, MA: Center for School Counseling Outcome Research, University of Massachusetts, Amherst.
Maryland State Department of Education. (2010). 2010 Maryland Report Card. [Data File]. Retrieved from
www.mdreportcard.org
Nelson, D. E., Gardner, J. L., & Fox, D. G. (1998). An evaluation of the comprehensive guidance program in Utah public schools. Salt Lake City, UT: Utah State Office of Education.
Sink, C. A. (2005). Comprehensive school counseling programs and academic achievement—A rejoinder to Brown and Trusty. Professional School Counseling, 9, 9–12.
Sink, C. A., & Stroh, H. R. (2003). Raising achievement test scores of early elementary school students through comprehensive school counseling programs. Professional School Counseling, 6, 350–364.
Whiston, S. C., & Wachter, C. (2008). School counseling, student achievement, and dropout rates: Student outcome research in the state of Indiana (Special Report). Indianapolis, IN: Indiana State Department of Education.
Wong, K. S. (2008). School counseling and student achievement: The relationship between comprehensive school counseling programs and school performance. [Doctoral dissertation]. Retrieved from Dissertation Abstracts International Section A: Humanities and Social Sciences. (AAT 3322290) .

Lauren E. Palmer, NCC, is a school counseling graduate student at Loyola University Maryland. Bradley T. Erford, NCC, is a Professor at Loyola University Maryland. Correspondence can be addressed to Bradley T. Erford, Loyola University Maryland, 2034 Greenspring Drive, Timonium, MD 21093, berford@loyola.edu.