Teaching Gatekeeping to Doctoral Students: A Qualitative Study of a Developmental Experiential Approach

Dec 16, 2020 | Volume 10 - Issue 4

Brenda Freeman, Tricia Woodliff, Mona Martinez

In addition to developing teaching, clinical supervision, and research skills, new entrants into the counselor education workplace will also face the challenging responsibility of gatekeeping. Gatekeeping can be both anxiety-provoking and time-intensive for new faculty members. To enhance the confidence and competence of new entrants into counselor education faculty positions, strong doctoral preparation in gatekeeping is critical. In this article, the authors describe a developmental experiential model to infuse gatekeeping instruction into counselor education and supervision doctoral courses. The model includes six experiential gatekeeping modules designed for instruction at three developmental levels. A phenomenological qualitative study of the model was conducted, leading to the discovery of four themes: importance of gatekeeping, behind the curtain, understandings vary by developmental level, and uneven responses to experiential learning. Developmental, pedagogical, and administrative implications for counselor educators are discussed.   

Keywords: counselor education, gatekeeping, doctoral preparation, experiential model, phenomenological

For new entrants into the counselor education higher education workplace, involvement in gatekeeping can be unavoidable and challenging. Although direct gatekeeping responsibilities may be conducted by associate and full professors in many institutions (Schuermann et al., 2018), assistant professors often teach courses in which gatekeeping issues arise. Evidence suggests that faculty perceptions of gatekeeping differ by academic rank (Schuermann et al., 2018), with untenured professors reporting greater concerns about gatekeeping than tenured faculty (Gaubatz & Vera, 2002). Bodner (2012) asserted that “faculty and supervisors may receive little guidance on how to implement such [gatekeeping] procedures in a highly ethical manner and/or how to approach complex and challenging gatekeeping dilemmas” (p. 60).

The gatekeeping role is taught during doctoral preparation. In the doctoral standards set by the Council for Accreditation of Counseling and Related Educational Programs (CACREP), Section B (Doctoral Professional Identity) requires the instruction of students in five core areas, two of which (teaching and supervision) include gatekeeping standards (CACREP, 2015). Supervision standard 2.i. requires programs to include in the curriculum “evaluation, remediation, and gatekeeping in clinical supervision” (CACREP, 2015, p. 35). Teaching standard 3.f. states that the curriculum must include “screening, remediation, and gatekeeping functions relevant to teaching” (CACREP, 2015, p. 36). The inclusion of gatekeeping in CACREP standards signals the importance of providing doctoral students with the knowledge, skills, and experiences necessary for them to be effective in their future role as gatekeepers.

There is a dearth of literature on pedagogy for teaching gatekeeping to doctoral students. Barrio Minton et al. (2018) conducted an analysis of select published articles and concluded that there has been a lack of focus on doctoral-level counselor education preparation. With limited publications centered on doctoral preparation and a generally minimal focus on pedagogy, the instructional approaches to prepare doctoral students for gatekeeping are largely unknown.

The purpose of our study was to design and deliver a developmental experiential model for increasing doctoral student competence in gatekeeping and to examine student reactions to these learning experiences. We have titled the gatekeeping instructional approach the Developmental Experiential Gatekeeping (DEG) Model. The DEG Model was designed and implemented at one CACREP-accredited counselor education and supervision (CES) doctoral program in the Western United States with a focus on preparing students for academic positions. This article presents the results of a phenomenological qualitative study of the experiences and reactions of doctoral students to the DEG Model. The insights gleaned from the study are discussed from the standpoint of improving pedagogy for gatekeeping instruction. The rationale for the study was that gatekeeping is a challenging aspect of counselor education teaching and supervision roles, particularly for new entrants into academia. Effective preparation in gatekeeping practices may not decrease the strain of dealing with difficult student remediation, suspension, and potential legal issues, but preparation is necessary to bolster strong gatekeeping and remediation practices.

Developmental Framework With Experiential Pedagogy

The DEG Model is an approach to instructing doctoral students in gatekeeping through the delivery of six curricular units divided into three developmental levels. The model was developed and implemented at a midsize institution (classified in the Carnegie system as an R1: Doctoral University – Very High Research Activity) with three counseling master’s programs and a doctoral program in counselor education and supervision located in the Western region of the United States. All programs were fully accredited under the CACREP 2016 standards (CACREP, 2015).

The DEG Model is grounded in both developmental and experiential pedagogy. The developmental framework, based in cognitive developmental theory, endorses sequential movement in learning processes within an established hierarchy (Bloom, 1956; Loevinger, 1976; Piaget, 1977). Higher levels are not attained without first accomplishing less complex levels of cognitive understanding. The development of formal operations, in which more sophisticated connections and abstract concepts are understood, is gradual and is based upon the interaction between cognition and experiences (Case et al., 2001; Eggen & Kauchak, 2001). Formal operations are situation specific (Eggen & Kauchak, 2001). Students may have reached formal operations in learning domains where they have a supporting framework of experiences, such as in post-internship counseling skills, and yet not function in formal operations in other content domain areas (such as research skills).

The experiential learning approach, reportedly a more powerful pedagogy than didactic instruction alone (Borowy & McGuire, 1983; Shreeve, 2008), is focused on gaining knowledge through direct experience. The process typically begins with preparation for the experience, followed by engaging in the experience, and culminating with reflection or testing of observations (Galizzi, 2014; Kolb & Kolb, 2009). Positive outcomes associated with experiential pedagogy include increased student engagement in the learning processes, improvements in cognitive functioning, greater acquisition of knowledge across a variety of subject areas (Galizzi, 2014; Greene et al., 2014; Tretinjak & Riggs, 2008), increases in historical empathy, improved critical thinking, and greater cultural open-mindedness (Greene et al., 2014). Borders et al. (1996) found didactic and experiential practices were related to a significant increase in student self-appraisal of supervision capacity. It is reasonable to assume that because experiential activities in supervision led to greater student competence, experiential activities in gatekeeping may also lead to greater student competence.

Research supports that experiential learning is an efficacious approach to teaching multicultural counseling (Kim & Lyons, 2003), particularly when the experiences closely emulate real world applications (Furr & Carroll, 2003; Granello, 2000). Although research on experiential learning related to teaching gatekeeping was not found, experiential learning in gatekeeping may be similar to multicultural counseling in that the experiential activities often used in the instruction of multiculturalism may be unfamiliar or uncomfortable for students. The DEG activities were unfamiliar experiences for doctoral students. Also parallel to instruction in multiculturalism, there is a gatekeeping culture that is unfamiliar to most doctoral students. Students must be introduced to the culture of gatekeeping, including the cultural norms and the development of a gatekeeping mindset.

Two assumptions were foundational to the pedagogy of the DEG Model. First, the authors assumed the DEG Model would have greater impact on student learning if delivered over more than one semester to allow time for integration of knowledge. Second, to maximize the advantages of experiential pedagogy, we assumed each DEG module should provide students with the opportunity for reflection after every experiential activity.

The DEG Model

The DEG Model was structured through a hierarchy informed by developmental principles (Bloom, 1956). Level 1 modules designed to meet the overall learning goal, To increase student understanding of concrete knowledge related to gatekeeping, dispositional assessment, and admissions, were delivered in a first-semester, first-year doctoral seminar course. Although experiential assignments were included with each module, the focus in Level 1 was on student acquisition of concrete knowledge (Bloom, 1956). The modules in Level 2 were integrated into an introductory course in clinical supervision and were designed to address Bloom’s Taxonomy (1956) comprehension and application levels. The learning goal for the Level 2 modules was To increase student knowledge and applied skills related to remediation and gatekeeping in clinical supervision. The Level 3 modules, designed to be consistent with Bloom’s Taxonomy (1956) analysis and synthesis levels, were infused into Doctoral Seminar II, a course with a focus on teaching pedagogy. The modules were designed toward the following goal: To develop student skills in analysis and synthesis of knowledge related to gatekeeping, with a focus on developing a systems understanding of gatekeeping. Each module described in the next section incorporated an experiential element and a written reflection.

DEG Modules
     The specific content domains for each module were driven by the literature. Table 1 includes descriptive material on the content for each module. The overall design of the DEG Model involved the infusion of six gatekeeping modules over a 16-month time frame in three sequential CES doctoral courses.

Table 1 

DEG Modules: Developmental Level, Content Domains, and Source Material

^aSource materials appear in order of recommended reading.

Grappling With Gatekeeping in Level 1, Module 1
In this module, for three consecutive classes (9 clock hours), first-year students were required to read and discuss journal articles on foundational gatekeeping topics selected by second-year students with guidance from the instructor. The structured class instruction and discussions on the readings were facilitated by the second-year students. The experiential component for first-year students was engagement in structured dialogue. The experiential component for second-year students was teaching gatekeeping and leading discursive discussion with first-year students under live faculty supervision. Students then reflected on the process. 

Dispositional Assessment in Admissions in Level 1, Module 2
     Armed with background knowledge from Module 1, students participated in the dispositional assessment training video for the Professional Disposition Competence Assessments—Revised Admissions (PDCA-RA; Freeman & Garner, 2020; Garner et al., 2020). The training video entails participant ratings of dispositions during admissions interview clips without training, followed by training in the assessment process, post-training rating of interview clips, and instructions on use of the PDCA-RA in actual admissions interviews. Following the PDCA-RA training, the doctoral students co-interviewed (with CES faculty) the master’s program applicants, using the PDCA-RA as the admissions dispositional assessment tool. This was followed by written reflections about the experience.

Gatekeeping Issues in Clinical Supervision in Level 2, Module 1
     This module was preceded by several weeks of instruction in clinical supervision theory and the assignment of one master’s-level supervisee to each doctoral student. Midway through the semester, students were instructed in best practices for giving evaluative formative and summative feedback in clinical supervision through the lens of the discrimination model (Bernard, 1997). The experiential component of this module consisted of students being required to deliver either formative or summative (positive or corrective) evaluative feedback to clinical supervisees related to the expected student dispositions under faculty supervision. Students then reflected on the process.

Mentoring Students Through Monitoring Remediation in Level 2, Module 2
     This module was designed to provide doctoral students with an experiential opportunity to partner with faculty in providing support for master’s students working on mild remediation issues. Examples of mild remediation issues included problems with class attendance or punctuality, difficulty adjusting to the professional expectations of graduate school, and challenges with interpersonal relationships in the classroom. The faculty team working in concert with the master’s student needing remediation determined the nature of the specified growth experiences for the master’s student. The doctoral students then implemented structured processes to support the remediation process, such as facilitating a reflective process on a student’s effort to become more culturally sensitive or serving as an accountability partner for a student working to become more conscientious. Doctoral students were not involved in working with any students where dismissal was a likely outcome. Doctoral students then wrote journal reflections on the experience.

The Ecological Gatekeeping Map in Level 3, Module 1
     With the developmental goal of synthesizing complex knowledge, students were tasked with creating an ecological gatekeeping map. The process began with didactic instruction in Bronfenbrenner’s (1977) ecological systems theory, followed by discussions of microsystems, mesosystems, exosystems, and macrosystems in higher education. The goal was to assist students in acquiring a systems perspective on gatekeeping, including subsystem interactions that influence the feasibility and outcomes of remediation, suspension, and dismissal of counseling students. As part of the module, students were introduced to the concept of the shadow organization (Allen & Pilnick, 1973). Allen and Pilnick (1973) described organizations as having two organizational structures—one being the visible structure obvious in the university organizational chart and the other (the shadow organization) consisting of the unwritten cultural expectations and daily behaviors of the institution. An example of the shadow organization influencing gatekeeping would be if the counseling handbook states that the program gatekeeps, but there is an unwritten culture in which the administration will not allow the program to dismiss even the most unethical student. Working as a team, the students had 6 weeks to interview administrators and faculty, collect policy and procedure documents, read and apply relevant literature, and prepare a group presentation of a visual ecological gatekeeping map.

Gatekeeping and Heartbreaking Adversity in Level 3, Module 2
     The final DEG module began with assigned readings of gatekeeping legal cases. Students were then charged with the responsibility to create a non-academic dismissal scenario, write and compile all documentation, and prepare to dramatize the scenario through a mock dismissal hearing. Roles adopted by students for the mock hearing included the fictitious master’s counseling student, the faculty member central to the dismissal scenario, the department chair, and the college dean. The mock hearing was enacted and was judged in real time by a university attorney and a university administrator (a dean or provost). Immediately following the hearing, the judges processed the hearing with the students, offering legal and procedural corrections. Students then reflected on the experience.

Method

The question “What are the lived experiences of doctoral students as they engage in gatekeeping instruction?” was addressed through qualitative methodology. Because we were interested in the subjective experiences of the student learners, the qualitative study was conducted using a phenomenological approach (Creswell, 2013; Moustakas, 1994). Investigation through deep exploration of lived experiences is part of the phenomenological paradigm (Creswell, 2014). Deep exploration of lived experiences with the gatekeeping experiential activities was congruent with the goal of understanding the journey of doctoral students to capture the essential meanings of gatekeeping. Husserl (2001) postulated that it was possible for researchers to bracket their own experiences to capture the essence of the experiences of others, which was one of the objectives in this analysis. The ontological assumption, informed by the constructivist paradigm, was that socially constructed multiple realities of gatekeeping exist (Mertens & Wilson, 2012).

The study was primarily conducted as scholarly inquiry into the developing professional identity of doctoral students relevant to the gatekeeping role. Aligned with the research question, the data analysis was accomplished through a phenomenological tradition, with a primary goal of revealing rich and concrete descriptions of the learning process and the translation of formal and experiential instruction into professional identity.

Subsequent to the analysis, the findings were also used to inform program development and pedagogy for counselor educators. This secondary use of the findings to inform program improvement is aligned with the values branch of program evaluation in which participant responses to program experiences are often viewed through a qualitative, constructivist perspective (Abma & Widdershoven, 2008). The use of the findings to inform counselor education pedagogy did not influence the interview protocol, data collection, or analysis process, which were conducted utilizing the phenomenological approach.

Participants
     For phenomenological studies, Creswell (2013) recommends between 3 and 15 participants. At the point of data collection, there were 12 students enrolled in the CACREP-accredited counselor education and supervision doctoral program where the DEG modules were delivered. The doctoral program was housed in the College of Education at a midsize university, classified in the Carnegie system as an R1: Doctoral University – Very High Research Activity.

Each of the 12 potential doctoral student participants had experienced some or all of the DEG modules, allowing the research team to gain insights from different levels of doctoral student professional identity development. Two students were removed from the participant pool because of a conflict of interest, yielding a participant pool of 10 students. Following human subjects research review board (IRB) approval, the 10 potential participants were contacted by email and invited to participate in the study. All 10 consented to be interviewed; however, one student was unavailable during the data collection window, leaving nine study participants.

As a precaution to mask the identity of the participants, specific demographics are not reported in this article. In general terms, the participants were primarily self-reported females, predominantly White, and ranged between 24 and 39 years old. Educationally, all participants had earned master’s degrees in counseling prior to entering the doctoral program. The students earned their counseling master’s degrees in institutions located in the West, South, Southwest, East, Midwest, and Rocky Mountain regions.

Procedure
     All nine doctoral student participants agreed to be interviewed and to allow electronic recording. Face-to-face interviews ranging in length from 30 to 60 minutes were conducted by a single member of the research team. No incentives were offered. Participants were informed that they could skip any of the interviewer questions. The items for the semi-structured interview protocol were first written by the lead author and then piloted with the second and third authors. The final items were determined by consensus of the research team. The interview protocol included nine items. Three were global items such as “Describe your learning experiences with gatekeeping and remediation in counselor education.” Of the remaining six items, each was dedicated to one of the DEG units. The interviewer first asked the student if they recalled having participated in the specific unit, followed by the prompt: “Please describe your experience with this unit. What was that learning experience like for you?” The same question was repeated for each of the six units.

Although the DEG Model was part of required coursework, participation in the study was strictly voluntary. To protect student participants from social pressure to participate in the study, all communications with participants were initiated by a single member of the research team with no evaluative relationship to the students. Further, the interviews were conducted during a time frame when no participants were enrolled in courses instructed by any member of the research team.

As a second source of data, student reflections were collected at the end of each unit. The reflections were ungraded and were used in the study to triangulate the interview data for the purpose of considering the consistency between the interview data and the reflections, part of the establishment of trustworthiness. The reflection data consisted of written, open-ended reflections on the experiences of students with each of the DEG modules. The reflections were submitted immediately following the experience with each DEG module. To scaffold the reflection process for students who found unstructured, open-ended reflections challenging, three prompts were offered: “Please share your reactions to the learning experience you engaged in today.” “What did you learn today that you consider to be important to your understanding of gatekeeping and remediation?” and “What questions come to mind as a result of engaging in this learning experience?”

Data Analysis
     The overarching purpose of the data analysis process is to bring structure and order into understanding the data for the purpose of addressing the research questions (Patton, 2015). In phenomenological research, there are many paradigms and differing worldviews on data analysis, including the issue of whether it is most suitable to analyze participant narratives through an ideographical approach or amass the data into qualitative themes (Moules et al., 2015). Accumulation of data with an analysis of themes was selected as the phenomenological data analysis approach. The results of the study were analyzed through Creswell’s (2014) approach to phenomenological analysis. Throughout the analysis, the research team bracketed their presuppositions and assumptions. The purpose of bracketing was to allow the voices of the participants, not the researchers, to dominate the analysis.

Following the interviews, the recordings were transcribed (using pseudonyms), and the transcriptions were reviewed for accuracy. The analyses of both the interviews and the reflections were conducted using NVivo12 (QSR International). The interview analysis was a three-part process that included open coding, thematic analysis, and thematic integration (Rossman & Rallis, 1998). The process began with reading and rereading the transcripts to deduce a list of core meanings for each transcript. This work was conducted by the lead author and verified by independent analysis of the second author. Once core meanings of individual transcripts were agreed upon, the meanings were cross-analyzed for repetition and clustered into themes and subthemes by the first and second authors working independently of one another. Team consensus was reached, and the data were then organized into a codebook. Data saturation was accomplished when it was determined that no new themes were emerging. The themes were then reviewed in relation to one another to clarify overlapping areas and collapse subthemes into broader themes. Direct quotes were extracted to support both textural and structural descriptions. After the analysis of the interview data, student reflections were analyzed using the codebook derived from the interview data. An “inconsistent” codebook category was created to code data inconsistent with the data found in the interviews. An “other coding” category was created to code data that reflected new concepts or themes not apparent in the interview data.

Reflexivity
     An important aspect of considering trustworthiness in phenomenological research is addressing bias (Creswell, 2013). The research team consisted of two White female researchers and one Hispanic and American Indian female researcher. One was a tenured full professor with extensive CES experience. Another had conducted research related to dispositional assessment. The third member of the research team had no specific background or personal experiences with gatekeeping. The team members had a wide range of experience in program evaluation and qualitative research. The shared assumptions of the research team were that understanding gatekeeping was an important professional obligation and that doctoral students with career aspirations of entering counselor education needed a solid foundation in gatekeeping.

Trustworthiness
     The process of establishing trustworthiness began with an understanding that the findings represented only one of many interpretations of the data (Corbin & Strauss, 2008). Early in the process, we consulted with a qualitative research expert who confirmed the analysis process (D. Barone, personal communication, December 2, 2018). Peer debriefing was used throughout the process (Creswell, 2014). The debriefing process included the research team presenting tentative findings at one regional and one national counselor education conference, a process that fostered research team deliberation on the interpretation of the data.

The areas for bracketing were identified prior to the interviews and consisted primarily of the delineation of the presuppositions and assumptions of the research team in order to avoid hindering the capacity of the team to listen to the participants. The actual bracketing was performed during the analysis stage by making notations of areas where presuppositions and assumptions might influence interpretation. Participants were not asked to bracket their assumptions. Direct quotes were heavily relied upon in the analysis to assure that the voices of the participants were heard throughout the process. An expert reviewer, a counselor educator not involved in the study, audited the results (Creswell, 2014; Patton, 2015), providing the team with feedback. Last, member checking was used to ascertain that we had not misunderstood or used participant statements out of context.

Results

The analysis yielded four themes: importance of gatekeeping, behind the curtain, understandings vary by developmental level, and uneven responses to experiential learning. Pseudonyms used during data collection were replaced with participant numbers for reporting purposes.

Importance of Gatekeeping
     The theme importance of gatekeeping describes the valuing of gatekeeping, remediation, and dispositional assessment by participants. Across all participants, gatekeeping and related processes were perceived as critically important. The rationale for valuing gatekeeping varied from participant to participant, with most offering more than one justification. Five participants positioned their responses within the professional mandate to protect the public. P1 stated:

I learned that some of my experiences as a counselor really influenced the importance that I put on gatekeeping . . . I’ve been doing counseling . . . so I had exposure to what it looks like when counselors in the field aren’t well suited or act from their own personal needs.

Two participants reflected that the protection of the public was particularly important because of the attraction of emotionally wounded individuals to the profession. As stated by P2:

[Gatekeeping and remediation] . . . are extremely important because people oftentimes I find go into the counseling field for the wrong reasons. Whether it’s a personal history with mental health issues and they’re trying to solve their own issues or because. . . maybe they like the power differential that is created in a helping relationship . . . they want to somehow take advantage.

Protecting counseling programs, universities, and the profession was also expressed as a reason for valuing gatekeeping. P3 stated: “The counseling profession is our own and needs to be protected,” later adding, “Despite how difficult it can be, if warranted, I want to play hardball to protect my students, other faculty, alumni, program, and the profession.”

Behind the Curtain
     Eight of the nine participants reported that they had limited awareness of gatekeeping and related processes in their master’s programs. P4 stated: “I mean, I’m sure we were gate checked in my master’s program, but I don’t really remember anything about it.” Participants discussed the process of learning about gatekeeping after the experience of being unaware of it in their master’s programs, noting that this process gave them a glimpse of what goes on behind the curtain. P9 described it as being given a different seat in the house, stating:

In my master’s program, I didn’t have any knowledge of anything like this . . . but now in my first year of the doctoral program, I feel like I have so much more of an understanding and kind of . . . like a different seat in the house. I can see how it all works and the importance of it.

Feelings associated with peeking behind the curtain were varied. P3 described it with positive affect: “So the first seminar class was really helpful. It was very much like the Wizard of Oz, pulling the curtain back and seeing what goes on behind everything in higher education.” P4 reported it to be an unsettling experience: “So our first year when we were learning about it, it was still a bit mysterious . . . kind of scary . . . I didn’t really know this process was going on . . . not like, so overtly. . . . it was kind of like, oh my God.”

Understandings Vary by Developmental Level
     All participant interviews reflected the theme understandings vary by developmental level. Some participants overtly addressed changes in developmental understandings, like P3, who said simply: “I thought it was tricky until it wasn’t.” She described her journey as becoming more comfortable over time. P5 reported: “I think the scaffolding was appropriate. . . . more content focused initially and then more at the process level with the application piece later on. It wasn’t like we were jumping right into applicability before we actually understood the different concepts.”

From the standpoint of developmental level, Level 1 students like P6 were inclined toward a concrete understanding of the concepts: “So my understanding of gatekeeping and counselor education is that it’s a process to make sure that the counseling students are where they’re supposed to be . . . academically and emotionally.” More advanced students like P1 reflected greater complexity in their understandings:

So part of our responsibility as counselors is to make sure the field is engaging ethically, and if we’re allowing people that are wounded in such a way that they’re not able to engage productively as counselors, then as a profession we’re acting essentially unethically. . . . Counseling is fundamentally about the person of the counselor and so we have to take that into account as counselor educators . . . gatekeeping or remediation become a big part of the more nebulous component of what makes a good counselor.

Another developmental issue was that the experiential frame or voice reflected by the participants varied throughout the process. Sometimes, particularly but not exclusively early in the developmental process, participants spoke with a student voice. At other points, participants reflected on their experiences through the perspectives of a clinical supervisor or counselor educator, reflecting a faculty voice. Sometimes participants shifted between the two voices. P5 directly addressed this issue:

So each of us was going through the process of being evaluated because there was a gatekeeping process for us as doctoral students . . . and so knowing that that was happening for us at the same time we were teaching it . . . it was just a pretty complex process.

P4’s comment on learning to give direct feedback in the clinical supervision unit reflects a conflicted voice:

But with a supervisee, it was different because you’re also in this evaluative role. . . . I wanted to like, be really supportive, you know . . . [but] I also had to evaluate their work. I wanted to be direct, but I also don’t want to give them a bad evaluation. It was just very difficult.

In this statement regarding the Level 1 module, P8 spoke through a counselor educator perspective:

I’m thinking about potentially becoming a faculty member . . . in interviewing at universities, I’d like to really try to understand their philosophy of gatekeeping and remediation to see if it could, like, be a good fit for me. If I went to a school and found out they didn’t do gatekeeping, I would have a really hard time being there . . . it’s just kind of like, “Well, what are we doing to ensure that the people we’re serving are protected?”

Uneven Responses to Experiential Learning
     Across all nine interviews, participants indicated a strong, positive response to experiential learning. However, some experiential elements were more powerful than others. Reflecting on the experience of participating in the PDCA-RA training video and the master’s admissions interviews, P7 stated: “I think it was just really, really fun to be a part of the training . . . and then to actually get the chance to do it again during admissions.” Teaching gatekeeping was described as a positive experience by P4:

Being forced to teach anyone anything is a good learning experience . . . a lot of pressure is on me. Like, oh, I really, really need to know this stuff so I can teach it pretty well. So, I definitely knew my presentation . . . so that was a good learning experience.

In relation to the mock hearing, P5 reflected: “I learned a lot. I was actually the student in the mock hearing and so I learned . . . from their perspective what they might experience, but I also learned from the other side of it too, from the institution side.”

Not all experiential activities were considered impactful. Three participants reflected that the remediation experiential module was confusing. The confusion may reflect on the module but could also be related to the concept that remediation is not a science and requires judgment, experience, and consultation with others. Stated by P8: “It was hard for me to tell [if the student made improvements] because I didn’t have like a clear baseline.” P1 reported: “I mostly ended up just having confusing conversations with the student.”

The ecological gatekeeping map also appeared to be lacking in experiential power. Although the group experience of working together on the module was deemed valuable, three participants could not recall what they learned from the experience. A word count showed participants gave shorter descriptions on the ecological map than on any of the other experiential units. It is possible that a deeper level of preparation in the ecological model would enhance the experiential learning. Understanding the system elements of higher education and how they overlap with gatekeeping is fraught with complexity, even for junior faculty.

Analysis of Reflections Data
     The data from the reflections were used to triangulate the interview data. In general, there was a high level of consistency between the reflections (submitted immediately following the modules) and the qualitative interviews (conducted after a time lapse). One interesting finding more evident in the reflections than in the interviews was the description of the emotional reactions to gatekeeping material. At the end of the analysis process, we created word clouds (pictorial displays of word frequencies) of the most common words used by participants. Through this process, we discovered there was a high frequency of a minimum of 12 emotionally laden words such as “scary” and “upsetting” in the data set, with more emotionality expressed in the reflections than in the interviews. Because the reflections were written, it appears that students were more likely to express emotional reactions in reflections than in the qualitative interviews. It is also possible that because the reflections were collected right after the experiential learning activities, emotional reactions were more accessible when the students wrote their reflections than at the time of the interviews.

Discussion and Implications

The CACREP expectation that counselor educators instruct doctoral students in gatekeeping and the awareness that new entrants to the counselor education workplace may experience considerable distress in their roles as gatekeepers inspired the study. Although gatekeeping and remediation may require a relatively small time commitment for new counselor educators, the nature of the work can be difficult and legalistic. The predominant goals of the study were to develop and infuse into the doctoral curriculum an experiential model for gatekeeping instruction and to gain insights into the lived experiences of doctoral students as they engaged in the learning modules.

The DEG Model is presented as one approach to doctoral instruction in gatekeeping. The experiential and developmental foundations for the approach are strongly supported in research, but literature on the application of these theories to the context of teaching gatekeeping to doctoral students was not available. Thus, the DEG Model and the qualitative study of the student learning experiences with the model are exploratory in nature. Nine students reported their perceptions and reactions to the DEG Model. An analysis of the lived experience of the students led to the discovery of four themes: importance of gatekeeping, behind the curtain, understandings vary by developmental level, and uneven responses to experiential learning. 

All nine participants were of one mind that gatekeeping, dispositional assessment, and remediation are important. Given that all nine students were from different master’s programs representing institutions located in various regions of the country, this finding suggests that gatekeeping has assumed a position of primacy as an essential function in counseling academic programs and an expected role for counselor educators. Earlier gatekeeping research reported hesitancy in trainees related to gatekeeping because of factors such as program culture, lack of protection for the gatekeepers, and confusion about the standards for gatekeeping (Shen-Miller et al., 2015). The results of this study suggest a possible shift in the perspective of new entrants to the counselor education workplace. In addition, state licensure boards have underscored the importance of gatekeeping the profession. Shen-Miller et al. (2015) also found that trainee ambivalence about the gatekeeping role mirrored faculty ambivalence, suggesting that faculty modeling of appropriate gatekeeping and remediation may be a critical factor in the changing attitudes of doctoral students. An alternative viewpoint is that though the students unanimously supported a belief that gatekeeping is important, their belief system may not translate well to their first actual gatekeeping situation as a counselor educator. The study participants had no direct experience with the often painful situations faculty face when legal action or student grievances are directed against them.

The behind the curtain theme illuminated the lack of transparency in gatekeeping, in that students were surprised by the gatekeeping processes. The finding is puzzling because remediation and gatekeeping literature encourages transparency in identification of dispositions, remediation processes, and reasons students might be dismissed from any given academic program. Perhaps for legal or other reasons counselor education programs are somewhat opaque in their explanations of gatekeeping.

The results provide support for delivering content in gatekeeping through developmental and experiential approaches. Consistent with developmental theory (Piaget, 1977) and findings in doctoral instruction in clinical supervision instruction (Baker et al., 2002; Granello & Hazler, 1998), students began the process with concrete understandings and moved toward more complex interpretations. Also, mirroring other studies in doctoral pedagogy (Dollarhide et al., 2013; Granello & Hazler, 1998), students attributed learning to engagement in experiential activities, rarely referencing lectures or reading assignments except as sources of foundational knowledge.

Aligned with developmental theory (Piaget, 1977), we learned that experiential learning must be carefully cross-walked to parallel to the developmental level of the participants. Two of the six modules (Mentoring Students Through Monitoring Remediation and Gatekeeping Through a Systems Lens: Designing an Ecological Gatekeeping Map) contained experiential elements that in retrospect the authors believe were not well aligned with the developmental levels of the students. Regarding the remediation module, at the time of the study, the doctoral students were working to embrace the new roles of teacher, researcher, and clinical supervisor. Adding the difficult-to-define role of remediation mentor was perhaps experienced as role overload. On the ecological map, the authors hypothesized that the task was too complex, requiring more didactic instruction and experience with systems in organizations.

The finding that two experiential elements were perhaps not targeted at the designated developmental level was less critical than the underscoring of the importance of conducting research on pedagogy in doctoral-level courses. Until conducting the study, we were unaware that the two experiential units were problematic and would have argued that the ecological gatekeeping map was one of the strongest experiential components in the DEG Model.

Implications for Counselor Education
     The findings of the study led to insights that inform program development and pedagogy for counselor educators. The values branch of program evaluation (Abma & Widdershoven, 2008) advocates the use of qualitative analysis to develop deeper understandings of how knowledge is constructed.

The finding that doctoral students expressed more emotion in the immediate aftermath of experiential activities reinforces the importance of prompt attention to emotional processing after experiential components. The emotional–motivational theory on learning posits that anxiety negatively impacts concentration and desired outcome as well as reduces interest in engaging in future learning experiences in the content area. This relationship is well documented in research on math anxiety (Passolunghi et al., 2019). Anxiety was expressed in some student reflections, but not unexpectedly, as gatekeeping can be laden with conflict.

The results point to several practical pedagogical issues referred to in program evaluation theory by Stufflebeam (2003) as input factors. One such factor is that experiential pedagogy requires more instructional time than didactic instruction. The authors concluded that the importance of gatekeeping and the overall positive results justified the time investment but recognize the difficulties involved in implementing time-intensive experiential activities. The findings reflect another counselor education input issue, which is the importance of building strong relationships with administrators and the legal department in order to offer students the opportunity to gain perspectives on gatekeeping from stakeholders outside the core counseling faculty. The End of the Road: Gatekeeping and Heartbreaking Adversity module could not be implemented without strong relationships with administrators and legal services.

The unique contributions of this study for counselor educators include an underscoring of the importance of instructing doctoral students in gatekeeping and the power of using experiential strategies. The interview data showed that students initially had a concrete interpretation of gatekeeping, but through participation in the experiential modules, they reported more comprehensive understandings. The importance of matching the learning experience to the developmental level of the student has been previously well established in developmental theory, but through the study we gained the insight that doctoral instruction in gatekeeping should begin at a concrete developmental level. The doctoral students in our study may have been advanced in terms of clinical and research skills, but their initial understanding of gatekeeping was unidimensional.

The study also underscores the importance of helping students reflect and identify their individual belief systems and personal approaches to gatekeeping. Although legal services may recommend that faculty consistently speak in one voice on gatekeeping issues, an essential first step in eventually developing departmental consensus is transparency between individual faculty on their differing perspectives. Beyond the department level, this ongoing conversation is also foundational to growing the profession in our collective understanding of gatekeeping. The study highlights the importance of starting this process at the doctoral student level.

Limitations and Future Research

One limitation of the study is that qualitative research is not intended to be generalized. Therefore, it is unknown if the findings apply to doctoral students enrolled in other counselor education programs. Although there were advantages in utilizing a participant pool with different levels of engagement in the DEG Modules, a limitation associated with this research team decision was that participants who had only experienced early modules may have reflected different perspectives if they had been interviewed after participation in the final modules. Second interviews were not conducted. Another limitation is that the students, though not enrolled in courses from the lead author at the time of the study, may still have been influenced to offer a positive perspective on their learning experiences. Follow-up post-graduation interviews could be a useful mechanism to address this limitation.

A limitation inherent in the design of the DEG Model is that although the design was appropriate for the context of one CES doctoral program, it may not be applicable to the institutional environments of other CES doctoral programs. The context of a high research institution may differ from an institution with a stronger focus on teaching, which could influence student reactions to the DEG Model. A second limitation related to the model itself is that departmental agreement was necessary to infuse gatekeeping material into three courses with different instructors with differing personal values and beliefs on gatekeeping. In addition, agreement to include doctoral students in master’s remediation experiences and admissions interviews was necessary to implement the DEG Model. This level of faculty collaboration may not be possible in all doctoral programs.

More research on counselor education doctoral preparation is needed. The dearth of CES research on pedagogy for instructing doctoral students is apparent in content areas well beyond gatekeeping. Within pedagogy for doctoral student preparation in gatekeeping, research is needed on outcome measures for the attainment of gatekeeping competence. In addition, a greater understanding of the impact of the personal experiences of those doctoral students who were remediated during their master’s preparation on their perspectives as future gatekeepers would be useful to the profession. Also, research on the amount of instructional time needed to effectively teach gatekeeping to a level of minimum competence is needed.

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

References

*References marked with an asterisk indicate literature included in Table 1.

Abma, T. A., & Widdershoven, G. A. M. (2008). Evaluation and/as social relation. Evaluation, 14(2), 209–225. https://doi.org/10.1177/1356389007087540

Allen, R. F., & Pilnick, S. (1973). Confronting the shadow organization: How to detect and defeat negative norms. Organizational Dynamics, 1(4), 3–18. https://doi.org/10.1016/S0090-2616(73)80020-8

*American Counseling Association. (2014). ACA code of ethics. https://www.counseling.org/docs/ethics/2014-aca-code-of-ethics.pdf

*Association for Counselor Education and Supervision Taskforce on Best Practices in Clinical Supervision. (2011, April). Best practices in clinical supervision. https://acesonline.net/wp-content/uploads/2018/11/ACES-Best-Practices-in-Clinical-Supervision-2011.pdf

Baker, S. B., Exum, H. A., & Tyler, R. E. (2002). The developmental process of clinical supervisors in training: An investigation of the supervisor complexity model. Counselor Education and Supervision, 42(1), 15–30. https://doi.org/10.1002/j.1556-6978.2002.tb01300.x

Barrio Minton, C. A., Wachter Morris, C., & Bruner, S. L. (2018). Pedagogy in counselor education: 2011-2015 update. Counselor Education and Supervision, 57(3), 227–236. https://doi.org/10.1002/ceas.12112

*Bemak, F., Epp, L. R., & Keys, S. G. (1999). Impaired graduate students: A process model of graduate program monitoring and intervention. International Journal for the Advancement of Counselling, 21(1), 19–30.
https://doi.org/10.1023/A:1005387309472

Bernard, J. M. (1997). The discrimination model. In C. E. Watkins, Jr. (Ed.), Handbook of psychotherapy supervision (pp. 310–327). Wiley.

*Bernard, J. M. (2006). Tracing the development of clinical supervision. The Clinical Supervisor, 24(1–2), 3–21. https://doi.org/10.1300/j001v24n01_02

*Bhat, C. S. (2005). Enhancing counseling gatekeeping with performance appraisal protocols. International Journal for the Advancement of Counselling, 27(3), 399–411. https://doi.org/10.1007/s10447-005-8202-z

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals. David McKay Company.

Bodner, K. E. (2012). Ethical principles and standards that inform educational gatekeeping practices in psychology. Ethics & Behavior, 22(1), 60–74. https://doi.org/10.1080/10508422.2012.638827

Borders, L. D., Rainey, L. M., Crutchfield, L. B., & Martin, D. W. (1996). Impact of a counseling supervision course on doctoral students’ cognitions. Counselor Education and Supervision, 35(3), 204–217.
https://doi.org/10.1002/j.1556-6978.1996.tb00224.x

Borowy, T. D., & McGuire, J. M. (1983). Experiential versus didactic teaching: Changes in self-actualization. The Journal of Humanistic Education and Development, 21(4), 146–152.
https://doi.org/10.1002/j.2164-4683.1983.tb00227.x

*Brear, P., Dorrian, J., & Luscri, G. (2008). Preparing our future counseling professionals: Gatekeeping and the implications for research. Counselling and Psychotherapy Research, 8(2), 93–101.
https://doi.org/10.1080/14733140802007855

Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513–531. https://doi.org/10.1037/0003-066X.32.7.513

*Brown, M. (2013). A content analysis of problematic behaviors in counselor education programs. Counselor Education and Supervision, 52(3), 179–192. https://doi.org/10.1002/j.1556-6978.2013.00036.x

*Brown-Rice, K. (2012). Examining counselor educators’ use of impairment in gatekeeping terminology. In Ideas and research you can use: VISTAS 2012, 1. https://www.counseling.org/resources/library/VISTAS/vistas12/Article_21.pdf

*Case, R., Demetriou, A., Platsidou, M., & Kazi, S. (2001). Integrating concepts and tests of intelligence from the differential and developmental traditions. Intelligence, 29(4), 307–336.
https://doi.org/10.1016/S0160-2896(00)00057-x

Corbin, J., & Strauss, A. (2008). Basics of qualitative research (3rd ed.). SAGE.

Council for Accreditation of Counseling and Related Educational Programs. (2015). 2016 CACREP standards. http://www.cacrep.org/wp-content/uploads/2017/08/2016-Standards-with-citations.pdf

*Crawford, M., & Gilroy, P. (2013). Professional impairment and gatekeeping: A survey of master’s level training programs. The Journal of Counselor Preparation and Supervision, 5(1). https://doi.org/10.7729/51.0030

Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). SAGE.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE.

Dollarhide, C. T., Gibson, D. M., & Moss, J. M. (2013). Professional identity development of counselor education doctoral students. Counselor Education and Supervision, 52(2), 137–150.
https://doi.org/10.1002/j.1556-6978.2013.00034.x

*Dufrene, R. L., & Henderson, K. L. (2009). A framework for remediation plans for counseling trainees. In G. G. Walz, J. C. Bleuer, & R. K. Yep (Eds.), Compelling counseling images: VISTAS 2009 (pp. 149–159). American Counseling Association.

Eggen, P., & Kauchak, D. (2001). Educational psychology: Windows on classrooms (5th ed.). Merrill/Prentice Hall.

*Elpers, K., & FitzGerald, E. A. (2013). Issues and challenges in gatekeeping: A framework for implementation. Social Work Education, 32(3), 286–300. https://doi.org/10.1080/02615479.2012.665867

*Enochs, W. K., & Etzbach, C. A. (2004). Impaired student counselors: Ethical and legal considerations for the family. The Family Journal, 12(4), 396–400. https://doi.org/10.1177/1066480704267240

*Forrest, L., Elman, N., Gizara, S., & Vacha-Haase, T. (1999). Trainee impairment: A review of identification, remediation, dismissal, and legal issues. The Counseling Psychologist, 27(5), 627–686.
https://doi.org/10.1177/0011000099275001

*Forrest, L., Shen Miller, D. S., & Elman, N. S. (2008). Psychology trainees with competence problems: From individual to ecological conceptualizations. Training and Education in Professional Psychology, 2(4), 183–192. https://doi.org/10.1037/1931-3918.2.4.183

*Frame, M. W., & Stevens-Smith, P. (1995). Out of harm’s way: Enhancing monitoring and dismissal processes in counselor education programs. Counselor Education and Supervision, 35(2), 118–129.
https://doi.org/10.1002/j.1556-6978.1995.tb00216.x

Freeman, B. J., & Garner, C. M. (2020). Training video and tools: The Professional Disposition Competency Assessment System (Revised) [Unpublished admissions tool]. University of Nevada, Reno.

Furr, S. R., & Carroll, J. J. (2003). Critical incidents in student counselor development. Journal of Counseling & Development, 81(4), 483–489. https://doi.org/10.1002/j.1556-6678.2003.tb00275.x

Galizzi, M. (2014). Bringing Adam Smith’s pin factory to life: Field trips and discussions as forms of experiential learning. Journal of the Scholarship of Teaching and Learning, 14(5), 27–47.
https://doi.org/10.14434/josotlv14i5.12938

Garner, C., Freeman, B., Stewart, R., & Coll, K. (2020). Assessment of dispositions in program admissions: The Professional Disposition Competency Assessment—Revised Admission (PDCA-RA). The Professional Counselor, 10(3), 337–350. https://doi.org/10.15241/cg.10.3.337

Gaubatz, M. D., & Vera, E. M. (2002). Do formalized gatekeeping procedures increase programs’ follow-up with deficient trainees? Counselor Education and Supervision, 41(4), 294–305.
https://doi.org/10.1002/j.1556-6978.2002.tb01292.x

*Gazzola, N., De Stefano, J., Thériault, A., & Audet, C. T. (2013). Learning to be supervisors: A qualitative investigation of difficulties experienced by supervisors-in-training. The Clinical Supervisor, 32(1), 15–39. https://doi.org/10.1080/07325223.2013.778678

*Gizara, S. S., & Forrest, L. (2004). Supervisors’ experiences of trainee impairment and incompetence at APA-accredited internship sites. Professional Psychology: Research and Practice, 35(2), 131–140.
https://doi.org/10.1037/0735-7028.35.2.131

*Goodrich, K. M., & Shin, R. Q. (2013). A culturally responsive intervention for addressing problematic behaviors in counseling students. Counselor Education and Supervision, 52(1), 43–55.
https://doi.org/10.1002/j.1556-6978.2013.00027.x

Granello, D. H. (2000). Contextual teaching and learning in counselor education. Counselor Education and Supervision, 39(4), 270–283. https://doi.org/10.1002/j.1556-6978.2000.tb01237.x

Granello, D. H., & Hazler, R. J. (1998). A developmental rationale for curriculum order and teaching styles in counselor education programs. Counselor Education and Supervision, 38(2), 89–105.
https://doi.org/10.1002/j.1556-6978.1998.tb00561.x

Greene, J. P., Kisida, B., & Bowen, D. H. (2014). The educational value of field trips. Education Next, 14(1), 78–86. https://www.educationnext.org/the-educational-value-of-field-trips

*Henderson, K. L. (2010). The remediation of students in counseling graduate programs: Behavioral indicators, terminology, and interventions [Unpublished doctoral dissertation]. University of New Orleans.

*Homrich, A. M. (2009). Gatekeeping for personal and professional competence in graduate counseling programs. Counseling & Human Development, 41(7), 1–24.

Husserl, E. (2001). Logical investigations (2nd ed.). Routledge.

*Hutchens, N., Block, J., & Young, M. (2013). Counselor educators’ gatekeeping responsibilities and students’ first amendment rights. Counselor Education and Supervision, 52(2), 82–95.
https://doi.org/10.1002/j.1556-6978.2013.00030.x

*Jacobs, S. C., Huprich, S. K., Grus, C. L., Cage, E. A., Elman, N. S., Forrest, L. M., Schwartz-Mette, R., Shen-Miller, D., Van Sickle, K., & Kaslow, N. (2011). Trainees with professional competency problems: Preparing trainers for difficult but necessary conversations. Training and Education in Professional Psychology, 5(3), 175–184. https://doi.org/10.1037/a0024656

*Johnson, W. B., Elman, N. S., Forrest, L., Robiner, W. N., Rodolfa, E., & Schaffer, J. B. (2008). Addressing professional competence problems in trainees: Some ethical considerations. Professional Psychology: Research and Practice, 39(6), 589–599. https://doi.org/10.1037/a0014264

*Kerl, S. B., Garcia, J. L., McCullough, C. S., & Maxwell, M. E. (2002). Systematic evaluation of professional performance: Legally supported procedure and process. Counselor Education and Supervision, 41(4), 321–332. https://doi.org/10.1002/j.1556-6978.2002.tb01294.x

Kim, B. S. K., & Lyons, H. Z. (2003). Experiential activities and multicultural counseling competence training. Journal of Counseling & Development, 81(4), 400–408. https://doi.org/10.1002/j.1556-6678.2003.tb00266.x

Kolb, A. Y., & Kolb, D. A. (2009). The learning way: Meta-cognitive aspects of experiential learning. Simulation & Gaming, 40(3), 297–327. https://doi.org/10.1177/1046878108325713

*Kress, V. E., & Protivnak, J. J. (2009). Professional development plans to remedy problematic counseling student behaviors. Counselor Education and Supervision, 48(3), 154–166.
https://doi.org/10.1002/j.1556-6978.2009.tb00071.x

*Lamb, D. H., Presser, N. R., Pfost, K. S., Baum, M. C., Jackson, V. R., & Jarvis, P. A. (1987). Confronting professional impairment during the internship: Identification, due process, and remediation. Professional Psychology: Research and Practice, 18(6), 597–603. https://doi.org/10.1037/0735-7028.18.6.597

Loevinger, J. (1976). Ego development. Jossey-Bass.

*Lumadue, C. A., & Duffey, T. H. (1999). The role of graduate programs as gatekeepers: A model for evaluating student counselor competence. Counselor Education and Supervision, 39(2), 101–109.
https://doi.org/10.1002/j.1556-6978.1999.tb01221.x

*McAdams, C. R., III, Foster, V. A., & Ward, T. J. (2007). Remediation and dismissal policies in counselor education: Lessons learned from a challenge in federal court. Counselor Education and Supervision, 46(3), 212–229. https://doi.org/10.1002/j.1556-6978.2007.ub00026.x

*McCaughan, A. M., & Hill, N. R. (2015). The gatekeeping imperative in counselor education admission protocols: The criticality of personal qualities. International Journal for the Advancement of Counselling, 37, 28–40. https://doi.org/10.1007/s10447-014-9223-2

*McDaniel, K. Z. (2007). Using the right tool for the right situation: Tailoring remediation plans for problem trainees within accredited marriage and family therapy programs. [Unpublished doctoral dissertation]. Kansas State University.

Mertens, D. M., & Wilson, A. T. (2012). Program evaluation theory and practice (1st ed.) Guilford.

*Miller, C. M. (2010). Endorsement for licensure: Practices and challenges reported by counselor supervisors. [Unpublished doctoral dissertation]. University of Virginia.

Moules, N. J., McCaffrey, G., Field, J. C., & Laing, C. M. (2015). Conducting hermeneutic research: From philosophy to practice. Peter Lang Publishing.

Moustakas, C. (1994). Phenomenological research methods. SAGE.

NVivo qualitative data analysis. (2012). Pty Ltd. (Version 12) [Computer software]. QSR International.

Passolunghi, M. C., Cargnelutti, E., & Pellizzoni, S. (2019). The relation between cognitive and emotional factors and arithmetic problem-solving. Educational Studies in Mathematics, 100(3), 271–290.
https://doi.org/10.1007/s10649-018-9863-y

Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). SAGE.

Piaget, J. (1977). The development of thought: Equilibration of cognitive structures. Basil Blackwell.

*Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435–448. https://doi.org/10.1080/026029309286259

*Rossman, G. B., & Rallis, S. F. (1998). Learning in the field: An introduction to qualitative research (1st ed.). SAGE.

*Russell, C. S., DuPree, W. J., Beggs, M. A., Peterson, C. M., & Anderson, M. P. (2007). Responding to remediation and gatekeeping challenges in supervision. Journal of Marital and Family Therapy, 33(2), 227–244. https://doi.org/10.1111/j.1752-0606.2007.00018.x

*Russell, C. S., & Peterson, C. M. (2003). Student impairment and remediation in accredited marriage and family therapy programs. Journal of Marital and Family Therapy, 29(3), 329–337.
https://doi.org/10.1111/j.1752-0606.2003.tb01210.x

Schuermann, H., Avent Harris, J. R., & Lloyd-Hazlett, J. (2018). Academic role and perceptions of gatekeeping in counselor education. Counselor Education and Supervision, 57(1), 51–65.
https://doi.org/10.1002/ceas.12093

Shen-Miller, D. S., Schwartz-Mette, R., Van Sickle, K. S., Jacobs, S. C., Grus, C. L., Hunter, E. A., & Forrest, L.
(2015). Professional competence problems in training: A qualitative investigation of trainee perspectives. Training and Education in Professional Psychology, 9(2), 161–169. https://doi.org/10.1037/tep0000072

Shreeve, M. W. (2008). Beyond the didactic classroom: Educational models to encourage active student involvement in learning. The Journal of Chiropractic Education, 22(1), 23–28. https://doi.org/10.7899/1042-5055-22.1.23

Stufflebeam, D. L. (2003). The CIPP model for evaluation. In T. Kellaghan, D. Stufflebeam, & L. A. Wingate (Eds.), International handbook of educational evaluation (pp. 31–62). Springer.

*Swank, J. M. (2014). Assessing counseling competencies: A comparison of supervisors’ ratings and student supervisees’ self-ratings. Counseling Outcome Research and Evaluation, 5(1), 17–27.
https://doi.org/10.1177/2150137814529147

*Swank, J. M., Lambie, G. W., & Witta, E. L. (2012). An exploratory investigation of the Counseling Competencies Scale: A measure of counseling skills, dispositions, and behaviors. Counselor Education and Supervision, 51(3), 189–206. https://doi.org/10.1002/j.1556-6978.2012.00014.x

*Swank, J. M., & Smith-Adcock, S. (2013). Creative group strategies for interviewing applicants for counselor education programs. The Journal of Counselor Preparation and Supervision, 5(1). https://doi.org/10.7729/51.0039

*Tate, K. A., Bloom, M. L., Tassara, M. H., & Caperton, W. (2014). Counselor competence, performance assessment, and program evaluation: Using psychometric instruments. Measurement and Evaluation in Counseling and Development, 47(4), 291–306. https://doi.org/10.1177/0748175614538063

*Taub, D. J., Servaty-Seib, H. L., Wachter Morris, C. A., Prieto-Welch, S. L., & Werden, D. (2011). Developing skills in providing outreach programs: Construction and use of the POSE (Performance of Outreach Skills Evaluation) rubric. Counseling Outcome Research and Evaluation, 2(1), 59–72. https://doi.org/10.1177/2150137811401019

Tretinjak, C. A., & Riggs, E. M. (2008). Enhancement of geology content knowledge through field-based instruction for pre-service elementary teachers. Journal of Geoscience Education, 56(5), 422–433.
https://doi.org/10.5408/jge_nov2008_riggs_422

*Winograd, G., & Tryon, G. S. (2009). Counseling expectations among students in an opportunity program: Dispositional and cultural influences. Journal of Counseling & Development, 87(4), 438–448.
https://doi.org/10.1002/j.1556-6678.2009.tb00128.x

Brenda Freeman, PhD, NCC, LCPC, CPC, is a professor at the University of Nevada, Reno. Tricia Woodliff, PhD, NCC, ACS, CPC, is an assistant professor at the University of Nevada, Reno. Mona Martinez, PhD, CPC, is Downing Clinic Director at the University of Nevada, Reno. Correspondence may be addressed to Brenda Freeman, William Raggio Building Rm. 3007, University of Nevada, Reno/0281, Reno, NV 89557, brendafreeman@unr.edu.