Developing and Validating a Process Model of Counselor Burnout: A Serial Mediation Model

Donghun Lee, Sojeong Nam, Jeongwoon Jeong, GoEun Na, Jungeun Lee

Despite advanced definitions and continued research on counselor burnout, attempts to investigate an expanded structure of counselor burnout remain limited. Using a serial mediation, the current study conducted a path analysis of a hypothesized process model using the five dimensions of the Counselor Burnout Inventory. Our research findings support the hypothesized sequential process model of counselor burnout, confirming full mediating effects of Deterioration in Personal Life, Exhaustion, and Incompetence in a serial order on the relationship between Negative Work Environment and Devaluing Client. Suggestions for future research and practical implications for counselors, supervisors, and directors are discussed.

Keywords: counselor burnout, serial mediation, path analysis, process model, Counselor Burnout Inventory

     Burnout has received significant attention in counseling research because of the unique nature of counseling work (Bardhoshi et al., 2019; Bardhoshi & Um, 2021; Fye et al., 2020; J. J. Kim et al., 2018; Maslach & Leiter, 2016). Early studies on burnout focused on defining burnout as a phenomenon. Freudenberger (1975), recognized as one of the pioneers in examining the burnout phenomenon, emphasized a loss of motivation and emotional exhaustion, defining it as “failing, wearing out, or becoming exhausted through excessive demands on energy, strength, or resources” (p. 73). Osborn (2004) placed an emphasis on physical and psychological reactions to job stress, describing burnout as “the process of physical and emotional depletion resulting from conditions at work or, more concisely, prolonged job stress” (p. 319). Maslach and colleagues have attempted to find a broader definition of burnout in their studies (Maslach & Jackson, 1981a, 1981b; Maslach et al., 1997, 2001; Maslach & Leiter, 2016) by conceptualizing three core dimensions of burnout—emotional exhaustion, a sense of reduced personal accomplishment, and depersonalization—as a syndrome of individuals exposed to long-term emotional and interpersonal stressors related to their job.

Many researchers have investigated burnout phenomenon, particularly among counselors, over several decades (Emerson & Markos, 1996; Evans & Villavisanis, 1997; W. C. McCarthy & Frieze, 1999; Malach-Pines & Yafe-Yanai, 2001). These studies have manifested additional focus on counselors’ actual performance in working with clients by describing counselor burnout as a state in which counselors experience considerable difficulties in performing proper functions and providing effective counseling. Kesler (1990) emphasized counselors’ internal psychological process in her definition of counselor burnout, defining it as a decreased sense of personal accomplishment in which an individual blames themself for their emotional and physical exhaustion, career fatigues, cynical attitudes toward clients, withdrawal from clients, and chronic depression and/or increased anxiety.

Further efforts have been made to better understand counselor burnout by examining the relationships of the three core dimensions—emotional exhaustion, depersonalization, and reduced personal accomplishment. There is a common argument across burnout studies that emotional exhaustion is the central quality of burnout and the most obvious manifestation of the syndrome (Golembiewski & Munzenrider, 1988; R. T. Lee & Ashforth, 1996; Leiter & Maslach, 1988, 1999; Maslach, 1993, 1998). Researchers have articulated the relationships by positing that emotional exhaustion may first trigger depersonalization and depersonalization can then cause decreased personal accomplishment (Golembiewski & Munzenrider, 1988; R. T. Lee & Ashforth, 1996; Leiter & Maslach, 1988). Slightly different findings have suggested that exhaustion may simultaneously lead to depersonalization and reduced personal accomplishment (R. T. Lee & Ashforth, 1993; Maslach, 1993). Some research has provided distinctive relationships among the three dimensions, indicating that emotional exhaustion may result from increased depersonalization (Golembiewski et al., 1986; Taris et al., 2005) and decreased personal accomplishment (Golembiewski et al., 1986).

In response to the need to establish a broader definition of counselor burnout, S. M. Lee and colleagues (2007) expanded the three-dimension model, previously introduced by Maslach and colleagues (1997), to consider organizational and personal sources of burnout. They added two dimensions—negative work environment and deterioration in personal life—to the three core dimensions of burnout, which advanced the model to a five-dimensional burnout model (S. M. Lee et al., 2007). Using the five dimensions, S. M. Lee and colleagues introduced the Counselor Burnout Inventory (CBI). The CBI is the first scale aimed at measuring professional counselors’ burnout symptoms, integrating the expanded theoretical constructs of burnout with the five dimensions representative of the counseling profession (i.e., Exhaustion, Incompetence, Negative Work Environment, Devaluing Client, and Deterioration in Personal Life). Several studies that have explored the psychometric characteristics of the CBI with diverse counselor populations working in a wide range of settings have demonstrated the solid validity and reliability of the CBI (Bardhoshi et al., 2019; J. Lee et al., 2010; S. M. Lee et al., 2007). For example, Bardhoshi et al. (2019) conducted a meta-analysis of 12 studies that utilized the CBI to examine its psychometric characteristics. Their psychometric synthesis reported robust internal consistency, external validity, and structural validity of the five dimensions of burnout across diverse groups of professional counselors, supporting the CBI’s suitability as a tool for understanding the multidimensional burnout phenomenon.

The Current Study
     Despite such advanced definitions and continued research on counselor burnout, attempts to understand the expanded structure of counselor burnout using all five dimensions remain limited. The CBI has demonstrated its fitness in burnout research using the five dimensions, especially to explore the relationships among the dimensions and to understand the sequential process of them. Understanding this underlying sequential process will allow counseling professionals to detect burnout-related symptoms earlier and develop prevention and intervention plans accordingly (Gil-Monte et al., 1998;
R. T. Lee & Ashforth, 1993; Noh et al., 2013).

Therefore, the current study aimed to evaluate a hypothesized sequential process model of the five dimensions of counselor burnout syndrome using the CBI, which consists of (a) Negative Work Environment, (b) Deterioration in Personal Life, (c) Exhaustion, (d) Incompetence, and (e) Devaluing Client. Our overarching goal was to enrich the literature by providing a foundation on which to develop an integrative theoretical process model of counselor burnout. The following research questions guided the current study:

1) What are the relationships among the five dimensions of counselor burnout measured by the CBI?

2) Is the relation between Negative Work Environment and Devaluing Client mediated by Deterioration in Personal Life, Exhaustion, and Incompetence in a serial order?

Conceptual Framework
     The current study presents a hypothesized sequential process of counselor burnout using the five dimensions of burnout measured by the CBI. Below we provide the rationale for our proposed serial order of counselor burnout, which is: 1) Negative Work Environment, 2) Deterioration in Personal Life, 3) Exhaustion, 4) Incompetence, and 5) Devaluing Client.

The first dimension, Negative Work Environment, reflects counselors’ attitudes and feelings toward their work environments beyond personal and interpersonal issues (S. M. Lee et al., 2007). Previous studies identified organizational factors as an early indicator of burnout among counselors, such as excessive work demands, role ambiguity and conflict, lack of recognition, limited supervisor and colleague support, poor relationships at work, and unfair decision-making (Demerouti et al., 2001; N. Kim & Lambie, 2018; Leiter & Maslach, 1988; Maslach & Leiter, 2008; C. McCarthy et al., 2010; Walsh & Walsh, 2002). These factors were found to be correlated with feelings of burnout and were identified as predictive factors for counselor burnout (N. Kim & Lambie, 2018).

The second dimension of the CBI, Deterioration in Personal Life, recognized as another early indicator of burnout, significantly predicts a wide range of burnout syndromes. This dimension refers to the counselors’ failure to maintain well-being in their personal lives by spending insufficient time with family and friends and having poor boundaries between work and personal life. Deterioration in Personal Life was positively associated with the Exhaustion subscale of the Maslach Burnout Inventory (MBI), accounting for the high number of counselors’ levels of burnout (S. M. Lee et al., 2007). In terms of the sequential order between the two major indicators of counselor burnout, we posited that a negative work environment precedes deterioration in personal life. A negative work environment, with characteristics such as excessive workload, role ambiguity and conflict, and a lack of supervisor and colleague support, may restrict counselors’ personal lives by reducing personal time for their own wellness. When personal time conflicts with an unfavorable work environment and demand, counselors may not easily find a balance between work and life, thus experiencing reduced quality of life and emotional and physical exhaustion as consequences.

Exhaustion, the third dimension of burnout, represents counselors’ physical and emotional depletions that result from excessive workloads and conflictive relationships at work. Exhaustion is the central quality of burnout (Maslach & Leiter, 2008), accompanied by feelings of being drained and emotionally overextended (Maslach, 1998). As many researchers have argued that exhaustion precedes other dimensions of burnout (Leiter & Maslach, 1999; Maslach et al., 1997, 2001; Maslach & Leiter, 2016), the core idea penetrating throughout their arguments is that emotional exhaustion occurs first, followed by reduced personal accomplishment and depersonalization.

Therefore, we proposed that the fourth and fifth dimensions—Incompetence and Devaluing Client, respectively—may be consequences that stem from counselors’ emotional and physical exhaustion. In the CBI, Incompetence refers to a counselor’s internal feeling of incompetence while evaluating their effectiveness as a professional counselor, and it represents their belief that they are an incompetent counselor or that they are failing to make a positive change in their clients. Previous studies have provided evidence that emotional exhaustion increases professional incompetence or inefficacy (R. T. Lee & Ashforth, 1993; Park & Lee, 2013; van Dierendonck et al., 2001). R. T. Lee and Ashforth (1993) insisted that emotional exhaustion increases the possibility of reducing one’s personal accomplishment while also directly causing depersonalization. The last dimension, Devaluing Client, is a counselor’s attitude and perception of their relationship with clients. It describes counselors’ callous attitudes toward clients, such as little empathy for, or no concern about, the welfare of their clients. S. M. Lee et al. (2007) reported that the Devaluing Client dimension was positively correlated with the Depersonalization subscale of the MBI, defined as “a negative, cynical, or excessively detached response to other people” (Maslach, 1998, p. 69).

Regarding the sequence between Exhaustion and the two dimensions of consequence, we postulated that Devaluing Client is the final stage of the burnout developmental model, suggesting that exhaustion triggers feelings of incompetence first, which in turn results in counselors’ actual behavior of devaluing clients. Emotional and physical exhaustion is a major direct threat to counselors’ competencies in providing quality services to their clients (R. T. Lee & Ashforth, 1993; Park & Lee, 2013; van Dierendonck et al., 2001), while devaluing clients—treating clients as objects—can be viewed as an emotional coping strategy to deal with the frustration derived from emotional depletion (Gil-Monte et al., 1998). Emotional exhaustion may therefore increase counselors’ feelings of incompetence and then exacerbate their detachment from clients to the point where they become callous toward and no longer interested in their clients (R. T. Lee & Ashforth, 1993; Leiter & Maslach, 1988; Taris et al., 2005). Thus, we posited that devaluing clients is the final crucial manifestation of the phenomenon among counselors who have experienced prolonged burnout, leading many of them to consider leaving the counseling profession.

In summary, we proposed an integrative process model of counselor burnout that comprises the following stages in sequence: 1) Negative Work Environment, 2) Deterioration in Personal Life,
3) Exhaustion, 4) Incompetence, and 5) Devaluing Client. That is, professional counselors who work in a negative work environment for an extended period may start to experience a deterioration in their personal lives, which could lead counselors to emotional and physical exhaustion. Counselors exposed to prolonged exhaustion may also feel a lack of competence in counseling, which may make them prone to becoming callous toward their clients. Figure 1 depicts this serial process model of counselor burnout.

Figure 1
Saturated Model of Counselor Burnout Process

 

 

 

 

Note. NW = Negative Work Environment, DP = Deterioration in Personal Life, EX = Exhaustion, IC = Incompetence, DC = Devaluing Client.

Methods

Procedure
     Upon IRB approvals from two different institutions, mass email invitations were sent to professional counselors affiliated with professional counseling associations (i.e., the American Counseling Association [ACA] and the American School Counselor Association [ASCA]). The email invitations contained a description of the study, proof of IRB approval, informed consent, and a link to a self-reported survey. Prior to responding to the web-based survey, participants were asked to review their consent information. Those who agreed to take part in the survey were asked to respond to a demographic questionnaire, followed by the CBI (S. M. Lee et al., 2007). All measures and forms were coded with the participants’ identification numbers to protect their privacy. Email addresses submitted by those who wanted to be entered into a drawing for compensation were stored in a separate database from the survey responses. Of the 428 participants who completed the online survey, we eliminated 69 who ceased their participation or did not fully complete the survey. As a result, 359 were included in the final data analysis.

Participants
     A total of 359 professional counselors who were currently practicing and affiliated with one or more professional counseling-related association(s) participated in the current study. In terms of demographics, 281 participants self-identified as female (78.3%) and 76 as male (21.2%), in addition to two participants who did not want to respond (0.5%). With regard to racial/ethnic identity, the majority of the participants self-identified as White (n = 270, 75.2%). Thirty-three participants identified themselves as African American (9.2%), 20 as Asian/Asian American/Pacific Islander (5.6%), 19 as Hispanic (5.3%), and two as Native American (0.6%). The participants were employed in K–12 school (n = 123, 34.3%), outpatient (n = 76, 21.2%), private practice (n = 66, 18.4%), counselor education (n = 37, 10.3%), university counseling (n = 22, 6.1%), medical/psychiatric hospital (n = 11, 3.1%), or other (n = 24, 6.7%) settings. Years of experience ranged from 1 to 47 years, with a mean of 11.4 and standard deviation of 9.6. The participants displayed diverse specialties, including school counseling (42.9%), mental health counseling (42.9%), marriage and family therapy (4.6%), rehabilitation counseling (3.7%), and other disciplines (5.8%).

Measures
Counselor Burnout Inventory (CBI)
     The CBI (S. M. Lee et al., 2007), a 20-item self-report inventory used to assess professional counselors’ burnout, consists of five dimensions: Exhaustion (e.g., “I feel exhausted due to my work as a counselor”), Incompetence (e.g., “I am not confident in my counseling skills”), Negative Work Environment (e.g., “I feel frustrated with the system in my workplace”), Devaluing Client (e.g., “I am not interested in my clients and their problems”), and Deterioration in Personal Life (e.g., “My relationships with family members have been negatively impacted by my work as a counselor”). These five dimensions reflect the characteristics of feelings and behaviors that indicate various levels of burnout among counselors. The CBI asks participants to rate the relevance of the statements on a 5-point Likert scale, ranging from 1 (never true) to 5 (always true). S. M. Lee et al. (2007) reported that Cronbach’s alpha coefficients of internal consistency reliability were .80 for the Exhaustion subscale, .83 for the Negative Work Environment subscale, .83 for the Devaluing Client subscale, .81 for the Incompetence subscale, and .84 for the Deterioration in Personal Life subscale. The current study obtained Cronbach’s alpha coefficients of .89 for Exhaustion, .87 for Negative Work Environment, .77 for Devaluing Client, .78 for Incompetence, and .84 for Deterioration in Personal Life. 

Data Analysis
     We first explored descriptive statistics of demographic variables to understand the sample characteristics, including gender, race, age, employment status, work setting, specialty, and years of experience. Missing values were imputed using predictive mean matching, which estimates missing values by matching to the observed values in the sample (Rubin, 1986).

To examine relationships among the five dimensions, we conducted a correlation analysis. After identifying the correlation matrix, which confirmed significant relationships among the five dimensions, we tested the hypothesized process model with a serial mediation using a path analysis. The ratio of response-to-parameter was 18:1 for the data, which satisfies the minimum amount of data for conducting a path analysis (Kline, 2015). Because the variables did not follow normality according to the Shapiro-Wilk test, skewness and kurtosis, and plots, we used weighted least squares. We started with a saturated model in which all possible direct paths were identified (Figure 1). Using the trimming method introduced by Meyers and colleagues (2013), we successively removed the least statistically significant path from the previous model until we found a model with all significant paths. To assess the model goodness of fit, model fit indices, including root mean square error residual (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI) were evaluated. The RMSEA and SRMR values ≤ .06 indicate excellent fit, and CFI and TLI values ≥ .95 indicate excellent fit. All analyses were conducted using R Statistical Software (R Core Team, 2022).

Results

Pairwise Correlation Analysis
     Table 1 depicts the results of the pairwise correlation analysis. Significant positive correlations were found among the five dimensions of burnout. The relationship between Deterioration in Personal Life and Exhaustion was the largest, while that between Deterioration in Personal Life and Devaluing Client was the smallest. Also notable was that Devaluing Client, which is the dependent variable in the serial mediation model, displayed weak relationships with Negative Work Environment, Deterioration in Personal Life, and Exhaustion, but a moderate relationship with Incompetence.

Table 1
Descriptive Statistics and Correlation Matrix

Variable n M SD 1 2 3 4 5
1. Negative Work Environment 359 9.91 3.75
2. Deterioration in Personal Life 359 9.46 3.52 .35**
3. Exhaustion 359 11.84 3.54 .43** .58**
4. Incompetence 359 8.67 2.56 .23** .35** .33**
5. Devaluing Client 359 5.52 1.94 .23** .22** .27** .40**
**p < .01

 

Path Analysis
     The first model was identified drawing all possible direct paths among the variables (Table 2). Model fit indices were not calculated, as this is a saturated model. This model indicated that four of the 10 direct paths were not statistically significant, with p values ranging from .065 to .145.

Table 2
Results of a Path Analysis: Saturated Model

Endogenous Variables Exploratory Variables Estimate Standardized Estimate p SMC
Deterioration in Personal Life Negative Work Environment 0.321 0.049 .000 0.117
Exhaustion Negative Work Environment 0.248 0.043 .000 0.386
Deterioration in Personal Life 0.483 0.046 .000
Incompetence Negative Work Environment 0.056 0.038 .145 0.146
Deterioration in Personal Life 0.153 0.044 .000
Exhaustion 0.125 0.045 .006
Devaluing Client Negative Work Environment 0.050 0.029 .082 0.188
Deterioration in Personal Life −0.003 0.035 .930
Exhaustion 0.069 0.037 .065
Incompetence 0.254 0.037 .000

Note. SMC = squared multiple correlation.

Among them, the path connecting Deterioration in Personal Life to Devaluing Client (p = .930) was the least significant. According to the trimming process, this path was removed to identify modified Model 1. The modified Model 1 was found to include two nonsignificant paths, connecting Negative Work Environment to Incompetence (p = .145) and Negative Work Environment to Devaluing Clients
(p = .082). The path connecting Negative Work Environment to Incompetence was less significant, and it was removed from the modified Model 1 to fit modified Model 2. The modified Model 2 identified only one path that was not significant (p = .050), which connected Negative Work Environment to Devaluing Client. This path was eliminated to identify modified Model 3. The modified Model 3 finally rendered significant direct paths only with excellent model fit indices, and it was retained as the final model (Table 3). Model fit indices of all the models are described in Table 4.

Table 3
Results of a Path Analysis: Retained Model

Endogenous Variables Exploratory Variables Estimate Standardized
Estimate
p SMC
Deterioration in Personal Life Negative Work Environment 0.311 0.049 0.000 0.110
Exhaustion Negative Work Environment 0.255 0.042 0.000 0.398
Deterioration in Personal Life 0.491 0.046 0.000
Incompetence Deterioration in Personal Life 0.176 0.043 0.000 0.146
Exhaustion 0.136 0.044 0.002
Devaluing Client Exhaustion 0.082 0.031 0.008 0.193
Incompetence 0.273 0.033 0.000

 Note. SMC = squared multiple correlation.

In the retained model (Figure 2), the indirect effect involving all three mediators, Deterioration in Personal Life, Exhaustion, and Incompetence, were found to be significant in a serial chain. The total effect of Negative Work Environment on Devaluing Clients was significant (standardized estimate = .269, p = .000), but the direct effect of Negative Work Environment on Devaluing Clients was not significant (standardized estimate = .062, p = .402). The ratio of the total indirect effect to the total direct effect was .273, and the proportion of the total effect mediated was .125. Among the indirect effects involving one or two of the three mediators, only the paths through Deterioration in Personal Life and Incompetence and the path through Exhaustion and Incompetence were found to be significant (p < .05).

Figure 2
Retained Model of Counselor Burnout Process

 

 

 


Note
. NW = Negative Work Environment, DP = Deterioration in Personal Life, EX = Exhaustion, IC = Incompetence, DC = Devaluing Client.

Table 4
Goodness of Fit Indices

Model χ2 df GFI AGFI RMSEA SRMR CFI TLI
Saturated model 0.000 0.000 1.000 1.000 0.000 0.000 1.000 1.000
Modified model 1 0.008 1.000 1.000 1.000 0.000 0.001 1.000 1.071
Modified model 2 2.168 2.000 0.997 0.981 0.015 0.019 0.999 0.994
Modified model 3 (retained) 6.104 3.000 0.993 0.964 0.054 0.033 0.978 0.926

 Note. AGFI = adjusted goodness of fit index; CFI = comparative normed fit index; df = degree of freedom; GFI = goodness of fit index; RMSEA = root mean squared error of approximation; SRMR = standardized root mean square residual; TLI = Tucker-Lewis index.

Discussion

This study examined a process model of counselor burnout in the work context by analyzing our proposed serial mediation model. The findings support the hypothesized sequential process model of counselor burnout by confirming the full mediating effects of Deterioration in Personal Life, Exhaustion, and Incompetence in a serial order on the relationship between Negative Work Environment and Devaluing Client.

The final model describes the mechanism of counselor burnout, pertaining to how it starts and proceeds to the point where clients are affected. In the proposed model, Negative Work Environment, an external factor that counselors are usually not able to control, may affect counselors’ experiences not only at work but also in their personal lives. This is consistent with previous findings in which counselors exposed to unfavorable work environments for an extended period tended to have poor boundaries between work and life and thus failed to maintain well-being in their personal lives (Leiter & Durup, 1996; Puig et al., 2012). Limited ability to find a balance between work and personal life because of negative work-related factors such as an excessive workload may affect the overall quality of life, resulting in emotional and physical depletion among counselors. Accordingly, in the final model, Exhaustion was predicted by Negative Work Environment through Deterioration in Personal Life. Exhaustion then predicted counselors’ feelings of incompetence as hypothesized. Being emotionally and physically exhausted, counselors may experience a reduced sense of self-competence and self-view as professional counselors, which may also influence their actual performance. The final model depicted that the feelings of incompetence predicted Devaluing Client, indicating counselors were unable to emotionally connect with their clients and thus lost interest in their clients’ welfare. Our findings supported previous studies (Maslach & Leiter, 2016; Maslach et al., 2001; Park & Lee, 2013; Taris et al., 2005; van Dierendonck et al., 2001) that found causal relationships between exhaustion and reduced accomplishment, as well as between exhaustion and depersonalization. For example, several researchers have argued that exhaustion may decrease professionals’ self-efficacy in providing quality services to their clients and that it may also increase depersonalization, in which they feel indifferent toward their work and the people with whom they work (R. T. Lee & Ashforth, 1993; Maslach & Leiter, 2016; Maslach et al., 2001; Park & Lee, 2013; Taris et al., 2005; van Dierendonck et al., 2001).

Lastly, our serial process model confirmed Devaluing Client to be the final stage of counselor burnout, predicted by Negative Work Environment through Deterioration in Personal Life, Exhaustion, and Incompetence. According to Maslach (1998), devaluing clients (i.e., a cynical attitude and feelings toward work and clients) begins with the action of distancing oneself emotionally and cognitively from one’s work, which can be a way to cope with emotional exhaustion. In fact, emotional detachment from work can be viewed as somewhat functional and even a necessary action to take to maintain effectiveness as a professional (Gil-Monte et al., 1998; Golembiewski et al., 1986). Maintaining proper emotional distance from clients and creating a clear boundary from work may act as an effective coping strategy for dealing with emotional and physical exhaustion. However, emotional detachment from work, despite its virtue as a potential coping strategy and an ethical practice (Gil-Monte et al., 1998), can be aggravated by emotional exhaustion through the perceived lack of competence as professional counselors. Such a detached attitude may lead counselors to become callous toward clients and to contemplate leaving the counseling profession (Cook et al., 2021). As the process model indicates, a negative work environment could significantly affect counselors’ social, emotional, cognitive, and behavioral aspects in order, which may harm their clients and the profession.

In addition to the mechanism of counselor burnout involving all dimensions with three mediators, the final model also identified effects involving a part of the five dimensions (Figure 2). Among those, a significant finding relates to the one without Exhaustion. According to the model, without Exhaustion, Negative Work Environment may still lead counselors to devalue clients through the deterioration in their personal lives and the feeling of incompetence. This finding is significant given that a variety of burnout theories and research have posited exhaustion as a key concept of burnout, insisting that emotional and physical depletion of counselors would lead up to feelings of incompetence and devaluing clients (Leiter & Maslach, 1999; Maslach & Leiter, 2016; Maslach et al., 2001; Park & Lee, 2013). Distinguishably, our findings suggest that, even when counselors do not necessarily experience emotional and physical exhaustion or are not aware of such experiences, counselors who work in unfavorable work environments that negatively impact their personal lives for a long period (Puig et al., 2012) may feel unable to maintain their effectiveness as professionals (Bandura & National Institute of Mental Health, 1986; Hattie et al., 2004) and thus become callous toward their work and clients (Maslach & Leiter, 2016).

Implications
     To the best of our knowledge, the present study is the first attempt to acquire an in-depth understanding of a process model of counselor burnout using the five dimensions of burnout. The present study introduced a model that depicts the sequential process among the five dimensions of counselor burnout, indicating how counselor burnout may develop from their experiences at work as counselors to the point where they may harm their clients. Our research findings suggest important practical implications for counselors, clinical supervisors, and counseling center directors.

Implications for Counselors
     The findings first enrich counselors’ understanding of their experiences of burnout and allow them to reflect on how they can relate themselves to each stage of the model. Counselors may use this model to examine their environments and experiences and to engage in self-reflection, assessing whether they have any signs of burnout. A single experience that may not have always necessarily been considered an indication of being in a developmental phase of counselor burnout, such as a limited number of coworkers who can provide case consultation (i.e., negative work environment) or reduced quality time with family and friends because of limited free time (i.e., deterioration in personal life), can now be perceived as early signs of counselor burnout that require intervention. This means that they may still be in the early phases and have the potential to bounce back if they seek help and receive early intervention. Given the gravity of more severe symptoms of burnout (i.e., incompetence and devaluing clients), counselors may utilize this model to detect the early signs of counselor burnout and develop strategies, such as self-care or help-seeking plans, so they can avoid progressing to the later phases of counselor burnout. Failing to take immediate action and receive appropriate help can lead to a serious problem, resulting in not only violating ethical obligations given to all counselors but also potentially harming clients. Specifically, counselors exhibiting the symptom of devaluing clients but failing to take any action may violate the two core values of the counseling profession: “nonmaleficence” and “beneficence” (ACA, 2014). The ACA Code of Ethics (ACA, 2014) states that professional counselors should avoid causing harm to their clients and work for the good of the clients by promoting their mental health and well-being. By devaluing clients as a result of experiencing burnout, counselors would essentially devastate the therapeutic relationship with their clients (Garner, 2006), impair their own ability to promote clients’ well-being (i.e., violation of beneficence), and ultimately cause serious harm to the clients (i.e., violation of nonmaleficence).

Implications for Clinical Supervisors
     As counselors take primary responsibility for detecting the symptoms of burnout to maintain their optimal effectiveness, supervisors should continue to support them by guaranteeing adequate supervision and continuing education that provides an opportunity to discuss burnout experiences. Supervisors may set aside time during the supervision sessions for genuine discussions to help counselors better address their burnout and encourage them to regularly adopt the sequential model of the current study to assess their experience pertaining to the five dimensions. Continuing education may focus on the early warning signs of burnout, which in the current study were negative work environment and deterioration in personal life, so that supervisees can take immediate action when detecting the signs of burnout. Encouraging counselors to monitor their sudden changes and stressors at work and in their personal life and maintain a balance between them could help them seek support from supervisors and professional counseling. Relatedly, when counselors show the cognitive and behavioral patterns of devaluing clients, which is the last element in the sequential process of counselor burnout, supervisors should take immediate actions to protect both clients and counselors (ACA, 2014; Dang & Sangganjanavanich, 2015) because it may signal more serious levels of burnout. Therefore, supervisors need to maintain an open and inviting atmosphere to not only allow conversations when supervisees feel less empathetic toward their clients or disinterested in their clients’ lives, but also initiate further discussions for creating intervention strategies that are effective at mitigating impairment caused by burnout (Merriman, 2015). Supervisors should be mindful that some counselors may be reluctant to discuss their burnout symptoms with their supervisors because of fear of a negative evaluation. Having a conversation regarding their burnout in a more confidential relationship, such as counseling, would be more effective and safer for the counselors to evaluate their impairment accurately and take actions as necessary.

Implications for Counseling Center Directors
     According to our findings, without necessarily going through the stage of emotional and physical exhaustion, negative work environment can have tremendous negative impacts on counselors’ overall competencies. This finding stresses the role of directors of counseling centers to create a positive work environment for counselors. First, the directors may periodically examine counselors’ perceptions of their work environment to determine whether they feel frustrated with the working system or perceive any unfair treatment (e.g., excessive workload, limited resources, unfair decision-making). For instance, they could hold a monthly meeting to listen to counselors’ difficulties and discuss improvement opportunities in the system. Paying attention to counselors’ difficulties and feedback can be critical to not only making system improvements but also building healthy relationships among members of the environment. The directors may take a careful look at their counselors’ caseloads and maintain a reasonable counselor-to-client ratio to prevent burnout and create a working environment that allows for the best services for their clients.

Second, counseling center directors can prevent or reduce counselors’ burnout by providing professional development opportunities. Counselors have experienced rapid changes in terms of counseling modalities and strategies and may feel inadequately prepared to meet the unique needs of their clients, which can result in counselor burnout. Therefore, it is beneficial for counselors to engage in professional development activities, such as workshops, continuing education, and conferences, to expand their knowledge and skills. These types of opportunities can reduce counselors’ feelings of incompetence and prevent them from progressing to the last phase (devaluing clients), even if the counselors are in the later phases of burnout.

Limitations and Future Directions
     The current study provides an in-depth understanding of the expanded developmental model of counselor burnout and suggests significant implications for the counseling profession. Nevertheless, there are some limitations in the current study. First, although our research participants represented the overall characteristics of the counseling population fairly, future research should contain counselors from diverse backgrounds with regard to demographics, work settings, or specialties. Increased diversity could help us understand the unique burnout phenomenon among counselor populations working in various settings and with diverse clients.

Second, this study examined the sequential model of counselor burnout within the structure of the five subfactors of the CBI (S. M. Lee et al., 2007), including Negative Work Environment, Deterioration in Personal Life, Exhaustion, Incompetence, and Devaluing Client. Further investigation should involve external variables to explore what may contribute to the development of counselor burnout and how it may affect the counseling process and outcomes.

Lastly, a longitudinal study is necessary to capture the extended understanding of the sequential development model of counselor burnout, given that time may be a critical factor that influences burnout among professionals. By conducting a longitudinal study, counseling professionals will be able to detect the advancement of burnout and take immediate action to initiate prevention and intervention plans.

Conclusion

Professional counselors who work in a negative work environment for an extended period may start to experience a deterioration in their personal lives, which could lead counselors to emotional and physical exhaustion. Counselors exposed to prolonged exhaustion may also feel a lack of competence in counseling, which may make them prone to becoming callous toward their clients. To the best of our knowledge, the present study is the first attempt to acquire an expanded understanding of a process model of counselor burnout using the five dimensions of burnout. The research findings validated the aforementioned hypothesized process model of counselor burnout, suggesting how counselor burnout may develop from their experiences at work to the point where they may harm their clients. Counselors may utilize this model to detect the early signs of counselor burnout and to develop strategies, such as self-care or help-seeking plans, so they can avoid progressing to the later phases of counselor burnout. Failing to take immediate action and receive appropriate help can lead to a serious problem, resulting in not only violating ethical obligations given to all counselors but also potentially harming clients. Supervisors and counseling center directors may set aside time for genuine discussions to help counselors better address their burnout and encourage them to regularly adopt the sequential model of the current study to assess their experience pertaining to the five dimensions.

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

References

American Counseling Association. (2014). ACA code of ethics. https://www.counseling.org/resources/aca-code-of-ethics.pdf

Bandura, A., & National Institute of Mental Health. (1986). Social foundations of thought and action: A social cognitive theory. Prentice Hall.

Bardhoshi, G., Erford, B. T., & Jang, H. (2019). Psychometric synthesis of the Counselor Burnout Inventory. Journal of Counseling & Development, 97(2), 195–208. https://doi.org/10.1002/jcad.12250

Bardhoshi, G., & Um, B. (2021). The effects of job demands and resources on school counselor burnout: Self-efficacy as a mediator. Journal of Counseling & Development, 99(3), 289–301. https://doi.org/10.1002/jcad.12375

Cook, R. M., Fye, H. J., Jones, J. L., & Baltrinic, E. R. (2021). Self-reported symptoms of burnout in novice professional counselors: A content analysis. The Professional Counselor, 11(1), 31–45.
https://doi.org/10.15241/rmc.11.1.31

Dang, Y., & Sangganjanavanich, V. F. (2015). Promoting counselor professional and personal well-being through advocacy. Journal of Counselor Leadership and Advocacy, 2(1), 1–13.
https://doi.org/10.1080/2326716X.2015.1007179

Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. (2001). The job demands-resources model of burnout. Journal of Applied Psychology, 86(3), 499–512. https://doi.org/10.1037/0021-9010.86.3.499

Emerson, S., & Markos, P. A. (1996). Signs and symptoms of the impaired counselor. The Journal of Humanistic Education and Development, 34(3), 108–117. https://doi.org/10.1002/j.2164-4683.1996.tb00335.x

Evans, T. D., & Villavisanis, R. (1997). Encouragement exchange: Avoiding therapist burnout. The Family Journal: Counseling and Therapy for Couples and Families, 5(4), 342–345. https://doi.org/10.1177/1066480797054013

Freudenberger, H. J. (1975). The staff burn-out syndrome in alternative institutions. Psychotherapy: Theory, Research & Practice, 12(1), 73–82. https://doi.org/10.1037/h0086411

Fye, H. J., Cook, R. M., Baltrinic, E. R., & Baylin, A. (2020). Examining individual and organizational factors of school counselor burnout. The Professional Counselor, 10(2), 235–250. https://doi.org/10.15241/hjf.10.2.235

Garner, B. R. (2006). The impact of counselor burnout on therapeutic relationships: A multilevel analytic approach [Doctoral dissertation, Texas Christian University]. ProQuest Dissertations Publishing.

Gil-Monte, P. R., Peiro, J. M., & Valcárcel, P. (1998). A model of burnout process development: An alternative from appraisal models of stress. Comportamento Organization E Gestao, 4(1), 165–179.
https://www.researchgate.net/publication/263043320

Golembiewski, R. T., & Munzenrider, R. F. (1988). Phases of burnout: Developments in concepts and applications. Praeger.

Golembiewski, R. T., Munzenrider, R. F., & Stevenson, J. G. (1986). Stress in organizations: Toward a phase model of burnout. Praeger.

Hattie, J. A., Myers, J. E., & Sweeney, T. J. (2004). A factor structure of wellness: Theory, assessment, analysis, and practice. Journal of Counseling & Development, 82(3), 354–364.
https://doi.org/10.1002/j.1556-6678.2004.tb00321.x

Kesler, K. D. (1990). Burnout: A multimodal approach to assessment and resolution. Elementary School Guidance and Counseling, 24(4), 303–311.

Kim, J. J., Brookman-Frazee, L., Gellatly, R., Stadnick, N., Barnett, M. L., & Lau, A. S. (2018). Predictors of burnout among community therapists in the sustainment phase of a system-driven implementation of multiple evidence-based practices in children’s mental health. Professional Psychology: Research and Practice, 49(2), 132–141. https://doi.org/10.1037/pro0000182

Kim, N., & Lambie, G. W. (2018). Burnout and implications for professional school counselors. The Professional Counselor, 8(3), 277–294. https://doi.org/10.15241/nk.8.3.277

Kline, R. B. (2015). Principles and practice of structural equation modeling (4th ed.). Guilford.

Lee, J., Wallace, S., Puig, A., Choi, B. Y., Nam, S. K., & Lee, S. M. (2010). Factor structure of the Counselor Burnout Inventory in a sample of sexual offender and sexual abuse therapists. Measurement and Evaluation in Counseling and Development, 43(1), 16–30. https://doi.org/10.1177/0748175610362251

Lee, R. T., & Ashforth, B. E. (1993). A longitudinal study of burnout among supervisors and managers: Comparisons between the Leiter and Maslach (1988) and Golembiewski et al. (1986) models. Organizational Behavior and Human Decision Processes, 54(3), 369–398. https://doi.org/10.1006/obhd.1993.1016

Lee, R. T., & Ashforth, B. E. (1996). A meta-analytic examination of the correlates of the three dimensions of job burnout. Journal of Applied Psychology, 81(2), 123–133. https://doi.org/10.1037/0021-9010.81.2.123

Lee, S. M., Baker, C. R., Cho, S. H., Heckathorn, D. E., Holland, M. W., Newgent, R. A., Ogle, N. T., Powell, M. L., Quinn, J. J., Wallace, S. L., & Yu, K. (2007). Development and initial psychometrics of the Counselor Burnout Inventory. Measurement and Evaluation in Counseling and Development, 40(3), 142–154. https://doi.org/10.1080/07481756.2007.11909811

Leiter, M. P., & Durup, M. J. (1996). Work, home, and in-between: A longitudinal study of spillover. The Journal of Applied Behavioral Science, 32(1), 29–47. https://doi.org/10.1177/0021886396321002

Leiter, M. P., & Maslach, C. (1988). The impact of interpersonal environment on burnout and organizational commitment. Journal of Organizational Behavior, 9(4), 297–308. https://doi.org/10.1002/job.4030090402

Leiter, M. P., & Maslach, C. (1999). Six areas of worklife: A model of the organizational context of burnout. Journal of Health and Human Services Administration, 21(4), 472–489. https://www.researchgate.net/publication/12693291

Malach-Pines, A., & Yafe-Yanai, O. (2001). Unconscious determinants of career choice and burnout: Theoretical model and counseling strategy. Journal of Employment Counseling, 38(4), 170–184.
https://doi.org/10.1002/j.2161-1920.2001.tb00499.x

Maslach, C. (1993). Burnout: A multidimensional perspective. In W. B. Schaufeli, C. Maslach, & T. Marek (Eds.), Professional burnout: Recent developments in theory and research (pp. 19–32). Taylor and Francis.
https://www.researchgate.net/publication/263847970

Maslach, C. (1998). A multidimensional theory of burnout. In C. L. Cooper (Ed.), Theories of organizational stress (pp. 68–85). Oxford University Press. https://www.researchgate.net/publication/280939428

Maslach, C., & Jackson, S. E. (1981a). The Maslach Burnout Inventory. Consulting Psychologists Press.

Maslach, C., & Jackson, S. E. (1981b). The measurement of experienced burnout. Journal of Organizational Behavior, 2(2), 99–113. https://doi.org/10.1002/job.4030020205

Maslach, C., Jackson, S. E., & Leiter, M. P. (2018). The Maslach Burnout Inventory manual (4th ed.). Mind Garden. https://www.mindgarden.com/maslach-burnout-inventory-mbi/685-mbi-manual.html

Maslach, C., & Leiter, M. P. (2008). Early predictors of job burnout and engagement. Journal of Applied Psychology, 93(3), 498–512. https://doi.org/10.1037/0021-9010.93.3.498

Maslach, C., & Leiter, M. P. (2016). Understanding the burnout experience: Recent research and its implications for psychiatry. World Psychiatry, 15(2), 103–111. https://doi.org/10.1002/wps.20311

Maslach, C., Schaufeli, W. B., & Leiter, M. P. (2001). Job burnout. Annual Review of Psychology, 52, 397–422. https://doi.org/10.1146/annurev.psych.52.1.397

McCarthy, C., Van Horn Kerne, V., Calfa, N. A., Lambert, R. G., & Guzmán, M. (2010). An exploration of school counselors’ demands and resources: Relationship to stress, biographic, and caseload characteristics. Professional School Counseling, 13(3), 146–158. https://doi.org/10.1177/2156759X1001300302

McCarthy, W. C., & Frieze, I. H. (1999). Negative aspects of therapy: Client perceptions of therapists’ social influence, burnout, and quality of care. Journal of Social Issues, 55(1), 33–50.
https://doi.org/10.1111/0022-4537.00103

Merriman, J. (2015). Enhancing counselor supervision through compassion fatigue education. Journal of Counseling & Development, 93(3), 370–378. https://doi.org/10.1002/jcad.12035

Meyers, L. S., Gamst, G. C., & Guarino, A. J. (2013). Performing data analysis using IBM SPSS. Wiley.

Noh, H., Shin, H., & Lee, S. M. (2013). Developmental process of academic burnout among Korean middle school students. Learning and Individual Differences, 28, 82–89. https://doi.org/10.1016/j.lindif.2013.09.014

Osborn, C. J. (2004). Seven salutary suggestions for counselor stamina. Journal of Counseling & Development, 82(3), 319–328. https://doi.org/10.1002/j.1556-6678.2004.tb00317.x

Park, Y. M., & Lee, S. M. (2013). A longitudinal analysis of burnout in middle and high school Korean teachers. Stress and Health, 29(5), 427–431. https://doi.org/10.1002/smi.2477

Puig, A., Baggs, A., Mixon, K., Park, Y. M., Kim, B. Y., & Lee, S. M. (2012). Relationship between job burnout and personal wellness in mental health professionals. Journal of Employment Counseling, 49(3), 98–109.
https://doi.org/10.1002/j.2161-1920.2012.00010.x

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

Rubin, D. B. (1986). Statistical matching using file concatenation with adjusted weights and multiple imputations. Journal of Business & Economic Statistics, 4(1), 87–94. https://doi.org/10.2307/1391390

Taris, T. W., Le Blanc, P. M., Schaufeli, W. B., & Schreurs, P. J. G. (2005). Are there causal relationships between the dimensions of the Maslach Burnout Inventory? A review and two longitudinal tests. Work & Stress, 19(3), 238–255. https://doi.org/10.1080/02678370500270453

van Dierendonck, D., Schaufeli, W. B., & Buunk, B. P. (2001). Burnout and inequity among human service professionals: A longitudinal study. Journal of Occupational Health Psychology, 6(1), 43–52.
https://doi.org/10.1037/1076-8998.6.1.43

Walsh, B., & Walsh, S. (2002). Caseload factors and the psychological well-being of community mental health staff. Journal of Mental Health, 11(1), 67–78. https://doi.org/10.1080/096382301200041470

 

Donghun Lee, PhD, NCC, is an assistant professor at the University of Texas at San Antonio. Sojeong Nam, PhD, NCC, is an assistant professor at The University of New Mexico. Jeongwoon Jeong, PhD, NCC, is an assistant professor at The University of New Mexico. GoEun Na, PhD, NCC, is an assistant professor at The City University of New York at Hunter College. Jungeun Lee, PhD, LPC, LPC-S, is a clinical professor at the University of Houston. Correspondence may be addressed to Donghun Lee, 501 W. Cesar E. Chavez Blvd., Durango Building 4.304, San Antonio, TX 78207, donghun.lee@utsa.edu. 

Counseling and the Interstate Compact: Navigating Ethical Practice Across State Lines

Amanda DeDiego, Rakesh K. Maurya, James Rujimora, Lindsay Simineo, Greg Searls

In the wake of COVID-19, health care providers experienced an immense expansion of telehealth usage across fields. Despite the growth of telemental health offerings, issues of licensing portability continue to create barriers to broader access to mental health care. To address portability issues, the Counseling Compact creates an opportunity for counselors to have privileges to practice in states that have passed compact legislation. Considerations of ethical and legal aspects of counseling in multiple states are critical as counselors begin to apply for privileges to practice through the Counseling Compact. This article explores ethical and legal regulations relevant to telemental health practice in multiple states under the proposed compact system. An illustrative case example and flowcharts offer guidance for counselors planning to apply for Counseling Compact privileges and provide telemental health across multiple states.

Keywords: telehealth, legislation, Counseling Compact, portability, telemental health

 

     Maximizing the use of technology-assisted counseling techniques, telemental health (TMH) is a modality of service delivery that takes the best practices of traditional counseling and adapts practice for delivery via electronic means (National Institute of Mental Health, 2021). There are many terms to describe TMH, which include: distance counseling, technology-assisted services, e-therapy, and tele-counseling (Hilty et al., 2017). Counselors have previously been hesitant in adopting technology-supported counseling practice (Richards & Viganó, 2013). Since the onset of the COVID-19 pandemic, TMH use has grown exponentially in applications across various disciplines (Appleton et al., 2021).

Between March and August 2021, out of all telehealth outpatient visits across disciplines, 39% were primarily for a mental health or substance use diagnosis, up from 24% a year earlier, and 11% two years earlier (Lo et al., 2022). During the same period, 55% of clients in rural areas relied on TMH to access outpatient mental health and substance use services compared to 35% of clients in urban areas. Further, TMH has expanded the accessibility of mental health services to underserved populations, including people living in remote areas, and marginalized groups, including sexual minorities, ethnic and racial minorities, and people with disabilities (Hirko et al., 2020). With broader adoption of TMH, existing issues of licensure portability continue to represent barriers to service provision across state lines. Recent state and national legislation that addresses telehealth parity and clarifies language in the provision of telehealth services supports the continued expansion of TMH (Baumann et al., 2020). Thus, to address portability issues, a partnership of professional counseling organizations engaged in a decades-long effort to address barriers to portability, leading to the recent creation of the Counseling Compact.

With the Counseling Compact legislation having been successfully passed in far more than the required 10 states, the Counseling Compact Commission has convened to create bylaws and informational systems to manage opening applications for privilege to practice in early 2024 (American Counseling Association [ACA], 2021). As counselors venture into a new era with privileges to practice across state lines, consideration of ethical and legal provisions of practice are critical. This article seeks to offer guidance to counselors in how to practice ethically within the privileges of the Counseling Compact. Exploration of ethical guidelines and legal statutes governing practice, a case example of ethical practice, and tools for establishing an ethical process of practice are offered.

Legal and Ethical Considerations for TMH

Provision of TMH may include use of various technology-supported methods of distance counseling. The National Board for Certified Counselors (NBCC; 2016) differentiates between face-to-face counseling and distance professional services. Face-to-face services include synchronous interaction between individuals or groups of individuals using visual and auditory cues from the real world. Professional distance services utilize technology or other methods, such as telephones or computers, to deliver services like guidance, supervision, advice, or education.

NBCC (2016) also differentiates counseling services from supervision and consultation for counselors. Counseling represents a working partnership that enables various people, families, and groups to achieve their goals in terms of mental health, well-being, education, and employment. This type of professional relationship differs from supervision, in that supervision is a formal, hierarchical arrangement involving two or more professionals. Consultation is an intentional collaboration between two or more experts to improve the efficiency of professional services with respect to a particular person. NBCC also offers guidance in defining various modalities for the provision of TMH related to counseling, supervision, or consultation (see Table 1). TMH can enhance accessibility to mental health services (Maurya et al., 2020). Barriers to care via TMH include lack of access to video-sharing technologies (e.g., tablet, personal computer, laptop), reliable internet access, private space for using TMH, and adaptive equipment for people with disabilities, as well as a lack of overall digital literacy (Health Resources & Services Administration, 2022). However, with some shared resources and community partnerships, these barriers can be addressed, and TMH can offer broad access to mental health professionals with various expertise and specialty training to begin addressing health disparities (Abraham et al., 2021). Another barrier is the lack of mental health professionals offering TMH services, in part due to challenges in counselor licensing (Mascari & Webber, 2013). TMH barriers also include challenges in counselor training as well as technical and administrative support to implement TMH services with certain client populations (Shreck et al., 2020; Siegel et al., 2021).

Table 1
Common Modalities for Provision of Telemental Health

Modality Definition
Telephone-based Information is conveyed across synchronous distance interactions using audio techniques.
Email-based Asynchronous distance interaction in which information is transmitted via written text messages or email.
Chat-based Synchronous distance interaction in which information is received via written messages.
Video-based Synchronous distance interaction in which information is received by video and or audio mechanisms.
Social network Synchronous or asynchronous distance interaction in which information is exchanged via social networking mechanisms.

 

Navigating Laws and Ethics
     As part of responsible practice, counselors who engage in TMH need to consider ethical considerations and risks. The ACA Code of Ethics (2014) dedicates Section H to “Distance Counseling, Technology, and Social Media” (p. 17). This section was added in the 2014 iteration of the ethical code in recognition of the fact that TMH was a growing tool for the counseling profession (Dart et al., 2016). These ethical guidelines address competency, regulations, use of distance counseling tools, and online practice. According to the most recent counselor liability report from the Healthcare Providers Service Organization (2019), the most common reasons for licensing board complaints within the last 5 years are issues related to sexual misconduct, failure to maintain professional standards, and confidentiality breaches.

Mental health professionals adhere to federal and state laws regarding privacy and security of information stored electronically (Dart et al., 2016). Such federal and state laws have been enacted to protect the privacy and security of information, and counselors must adhere to them in order to avoid legal ramifications (Dart et al., 2016). The ACA Code of Ethics (2014) provides guidelines for counselors to consider when in ethical dilemmas. For example, the ACA Code of Ethics advises counselors to adhere to the laws and regulations of the practice location of the counselor and the client’s place of residence. When working within a counselor’s statutory legislation and regulation, the ACA Code of Ethics directs counselors to address conflicts related to laws and ethics through ethical decision-making models. There are several models for counselors to follow (Levitt et al., 2015; Remley & Herlihy, 2010; Sheperis et al., 2016). Counselors, then, should be familiar with these ethical decision-making models to address ethical dilemmas with consideration of legal statutes in their state of practice. The ACA Code of Ethics strongly aligns with the NBCC Policy Regarding the Provision of Distance Professional Services (2016) providing guidance for National Certified Counselors (NCCs).

Counselors need to ensure compliance with applicable state and federal law in the provision of TMH services. For instance, the Health Insurance Portability and Accountability Act (HIPAA) is a federal law that establishes legal rules in the security and privacy of medical records, and the electronic storage and transmission of protected health information (Dart et al., 2016). Counselors need to select HIPAA-compliant software and technologies to maintain the security, privacy, and confidentiality of electronic client information. For example, regarding videoconferencing, Skype is not HIPAA-compliant (Churcher, 2012), but VSee and several other vendors have the appropriate level of encryption to meet HIPAA standards for compliance (Dart et al., 2016). Once HIPAA-compliant platform usage is established, counselors need to implement and collect the following information related to clinical documentation: 1) verbal or written consent from client or representative of client in the case of minors or those declared legally unable to provide consent; 2) category of the office visit (e.g., audio with video or audio/telephone only based on acceptable practice in that state); 3) date of last visit or billable visit; 4) physical location of client; 5) counselor location; 6) names and roles of participants (including any potential third parties); and 7) length of visit (Smith et al., 2020). Finally, as part of HIPAA standards, a vendor must offer a Business Associate Agreement (BAA), which demonstrates that the software or tool aligns with HIPAA encryption and privacy standards in communication with clients, transmission of data, and storage of client data (U.S. Department of Health & Human Services [HHS], 2013).

During the COVID-19 federal state of emergency, the Office for Civil Rights, tasked with enforcement of HIPAA regulations related to telehealth, exercised discretion and declared the agency would not impose penalties on counselors for lack of compliance in the provision of telehealth, assuming the counselor demonstrates a good faith effort to adhere to standards (HHS, 2021). These guidelines related to the discretion of enforcement cited the use of teleconferencing or chat tools without the adequate level of encryption as allowable, but only during the declared state of emergency.

Ethical Considerations
     The scope of practice for counselors varies depending on state licensure laws. It is critical that counselors be familiar with and act in accordance with state licensing laws. For example, if a client is based in a specific physical location, then the counselor needs to adhere to the licensing regulations and scope of practice in that physical location. Some states require licensure where the counselor is located as well as where the client is located. However, there is a lack of specificity in state licensure requirements related to the demonstration of competence with telehealth-specific practices (Williams et al., 2021).

To be able to provide mental health counseling services, mental health counselors need to consider their scope of practice based on state licensure guidelines where the client is located. This scope of practice is defined by the licensing standards of each state. Beyond scope of practice, counselors also need to consider boundaries of competence. The ACA Code of Ethics (2014) is nonspecific about how counselors demonstrate competence, only stating that they should “practice within the boundaries of their competence, based on their education, training, supervised experience and state and national professional credentials” (p. 8). Because of the lack of specificity in state telehealth practices, unless state licensure guidelines explicitly prohibit or advocate for specific telehealth practices, counselors may need to clarify interpretation of statutes or rules with licensure boards to determine specific telehealth practices.

Inherent in a counselor’s responsibility is their ability to screen clients for the appropriateness of telehealth services (Sheperis & Smith, 2021). Counselors are advised to determine whether clients have characteristics that may render them inappropriate for telehealth services, and then to make appropriate referrals (Morland et al., 2015). Some clients may not be appropriate for telehealth because of their (a) inability to access specific technology, (b) rejection of technology during the informed consent process, (c) severe psychosis, (d) mood dysregulation, (e) suicidal or homicidal tendencies, (f) substance use disorder, or (g) cognitive or sensory impairment (Sheperis & Smith, 2021). Finally, counselors are advised to utilize age- and developmentally appropriate strategies for children, adolescents, and older adults (NBCC, 2016; Richardson et al., 2009).

Once service providers, such as counselors, have appropriately screened clients for service, then informed consent is the next step. When counselors provide technology-assisted services, they are tasked to make reasonable efforts to determine clients’ intellectual, emotional, physical, linguistic, and functional capabilities while also appropriately assessing the needs of the client (ACA, 2014). When working with children, counselors need to know the age of the child or adolescent and the client’s legal ability to provide consent (Kramer & Luxton, 2016). Age of consent laws vary between states, so counselors need to familiarize themselves with their specific state legislation. This information is critical for the informed consent process and determining emergency procedures in case of a crisis (Kramer & Luxton, 2016). Counselors then need to consider and complete the informed consent process acknowledging the practice of TMH services.

In the informed consent process, it is imperative that counselors disclose risks related to TMH such as accessibility to technology, technology failure, and data breaches (ACA, 2014). Counselors are required to provide information related to procedures, goals, treatment plans, risks, benefits, and costs of services as part of the informed consent process (Jacob et al., 2011). Other considerations counselors may want to include during the informed consent process include confidentiality and limits of TMH; emergency plans; documentation and storage of information; technological failures; contact between sessions, if any; and termination and referrals (Turvey et al., 2013).

Client Crisis Plans
     There are specific steps to ensure appropriate emergency management practices when working with clients via telehealth (Sheperis & Smith, 2021). For example, at intake, these are the steps counselors could take: 1) verify the client’s identity and contact information; 2) verify the current location of the client and their residential address; 3) inquire about other health care providers; 4) navigate conversation regarding contact during emergency and non-emergency situations; and 5) implement a safety plan, if needed (Sheperis & Smith, 2021; Shore et al., 2018). Moreover, counselors need to stay up to date with local state and federal requirements related to duty to warn and protection requirements (Kramer et al., 2015).

For clients and counselors operating in separate cities or states, it is necessary for counselors to gather local law enforcement and emergency service contact information and maintain a plan of action if needed (Shore et al., 2007). Counselors are also advised to plan for service interruptions if and when technical issues arise during a crisis situation (Kramer et al., 2015). Aside from emergency management practices, counselors who engage clients during a crisis still need to apply basic counseling techniques such as unconditional positive regard, congruence, and empathy (Litam & Hipolito-Delgado, 2021). Once a counselor establishes a client’s psychological safety, they can begin to work collaboratively with clients to reestablish safety and predictability; defuse emotions; validate experiences; create specific, objective, and measurable goals; and identify any resources and coping mechanisms (Litam & Hipolito-Delgado, 2021).

Licensure Portability

In the wake of the COVID-19 pandemic, as states of emergency were issued at the state and national levels, licensing requirements were waived for the sake of allowing medical professionals to offer continuity of care via telehealth (Slomski, 2020). These time-bound waivers of practice highlighted the need for licensure portability, especially for counselors, even though in many of these states the waivers were difficult to obtain and could be withdrawn at any point when the state of emergency was rescinded. The widened use of telehealth during the COVID-19 pandemic amplified the growing calls for long-term licensure portability options for counselors.

In the United States, counselors experience challenges in transferring licensure between states, as counseling licensure standards vary from state to state (Mascari & Webber, 2013). The profession of counseling, although a relatively new field as compared to other helping professions such as psychology and social work, has been working toward licensure portability over the past 30 years. Since its inception in 1986, the American Association of State Counseling Boards (AASCB) has been focused on advocacy efforts to establish consistency in counseling licensing standards and avenues for licensing portability across states (AASCB, 2022). To advance toward this goal, AASCB first partnered with organizations such as ACA, the Council for the Accreditation of Counseling and Related Educational Programs (CACREP), and NBCC. Together these groups established the professional identity of counselors through a unified definition of counseling as a profession (Kaplan & Kraus, 2018), as well as consistent training standards for professional counselors across the nation (Bobby, 2013).

In an effort to promote a unified counselor identity and facilitate licensure portability, the 20/20 initiative (Kaplan & Gladding, 2011) included an oversight committee comprised of stakeholders from various organizations to develop a consensus definition of the profession, address prominent issues facing the profession at the time, and develop principles to guide advocacy work in strengthening the counseling profession (Kaplan & Gladding, 2011; Kaplan & Kraus, 2018). Licensure portability was identified as one of these key issues critical for the future of the profession. This issue persisted, with various states assigning different licensure titles, guidelines, requirements, and continuing education standards. Common training standards across specialty areas through CACREP, which merged with the Council on Rehabilitation Education in 2017, promulgate widely used guidelines for counselor licensure (CACREP, 2017). There are various licensure portability models currently used in medical fields: (a) the nonprofit organization model, (b) the mutual recognition model, (c) the licensure language model, (d) the federal model, and (e) the national model (Bohecker & Eissenstat, 2019). In early efforts, Bloom et al. (1990) proposed model licensure language that could be used to establish national licensing standards, which was an effort toward portability under the licensure language model. AASCB previously tried to move toward a national portability system through the nonprofit organization model by establishing the National Credential Registry, which is a central repository for counselor education, supervision, exams, and other information relevant to state licensure (Bohecker & Eissenstat, 2019). However, recently the effort to establish a Counseling Compact for licensure portability under the mutual recognition model gained great momentum in the time of the COVID-19 pandemic (AASCB, 2022).

Licensing Compacts in Medicine and Allied Professions
     The National Center for Interstate Compacts (NCIC) provides technical assistance in developing and establishing interstate compact agreements. According to NCIC, interstate compact agreements are legal agreements between governments of more than one state to address common issues or achieve common goals. Counseling is not the first health profession to pursue a licensing compact. Interstate compacts for medicine and allied professions have been established (Litwak & Mayer, 2021). Prior to current efforts for the Counseling Compact, similar legislation introduced compacts for physicians (Adashi et al., 2021), registered nurses (Evans, 2015), physical therapists (Adrian, 2017), psychologists (Goodstein, 2012), speech pathologists (Morgan et al., 2022), and emergency medical personnel (Manz, 2015). Other efforts to pass licensing compacts are underway for social workers (Apgar, 2022) and nurse practitioners (Evans, 2015). These compact models include multistate licensing (MSL) or privilege-to-practice (PTP) structures. A single multistate license obtained through the MSL model would allow a practitioner to practice equally in all member states, as opposed to the PTP model in which a practitioner would be licensed in their designated home state and then allowed specific privileges to use that license in other places (Counseling Compact, n.d.).

MSL compacts include licensing effective in multiple states. The MSL model is used for the Nurse Licensure Compact (Interstate Commission of Nurse Licensure Compact Administrators, 2021; National Council of State Boards of Nursing, 2015). Nurses licensed within this compact system gain multistate licenses across all member states. The Nurse Licensure Compact legislation notes efforts to reduce redundancies in nursing licensure by using an MSL model. Draft legislation within the Nurse Licensure Compact MSL system defines a “multistate license” as a license awarded in a home state that also allows a nurse the ability to practice in all other member states under the said multistate license. This includes both in-person and remote practice. So, for example, a nurse in a compact state can be vetted and licensed through the central compact system, which allows traveling nurses to switch between placements rapidly without additional licensing required for compact states. On the other hand, non-compact states issue a “single state license” which does not allow practice across states.

The PTP licensing model is used by physical therapy and EMS professionals. PTP establishes an agreement between member states to grant legal authorization to permit counselors to practice (NCIC, 2020). Unlike the MSL structure, counseling licensure is still maintained by a single state, or “home state,” but member states allow privileges to practice with clients located in other states as part of the compact agreement. This licensing model includes the definition of a “single state license,” which indicates that licenses issued by the state do not by default allow practice in any other states but the home state (Interstate Commission of Nurse Licensure Compact Administrators, 2021). Further, definitions include “privilege to practice,” which allows legal authorization of practice in each designated remote state. The Counseling Compact uses this PTP model for portability of licensing privileges across member states (Counseling Compact, 2020). 

The Counseling Compact 

Development of the Counseling Compact began in 2019 as a solution to the challenges of licensure portability. Historically, navigating varying licensure standards across states represented a barrier to the portability of counseling professionals and access to services for the community (Mascari & Webber, 2013). To address these barriers, organizations including NBCC (2017), ACA (2018), American Mental Health Counselors Association (2021), and AASCB (2022) have worked to unify the profession, establish common minimum licensing standards across states, and create and promote the Counseling Compact. With the support of NCIC, draft legislation for a PTP compact was developed by the end of 2020 and followed by advocacy efforts to pass legislation in a minimum of 10 states to begin the process of establishing the Counseling Compact (ACA, 2021). As of October 2023, the Counseling Compact has been passed as law in a growing list of states, including Alabama, Arkansas, Colorado, Connecticut, Delaware, Florida, Georgia, Indiana, Iowa, Kansas, Kentucky, Louisiana, Maine, Maryland, Mississippi, Missouri, Montana, Nebraska, New Hampshire, North Carolina, North Dakota, Ohio, Oklahoma, Tennessee, Utah, Vermont, Virginia, Washington, West Virginia, and Wyoming, with legislation pending in several other states (Counseling Compact, 2022).

Compact Standards
     The model legislation for the Counseling Compact outlines the provisions of the PTP model used (Counseling Compact, 2020). Within these regulations, only independently licensed counselors can apply for compact privileges. Each state maintains its own licensing standards and processes separately. PTP applies equally to both in-person and TMH services. Within the regulations of the compact, each counselor establishes a home state in which they hold a primary license. Prior to the compact, a counselor would have to seek additional licensure in other states to provide service to clients. However, under the PTP Counseling Compact, counselors who hold an unencumbered license may apply for privileges to use their home license in another state without seeking an additional formal license.

Counselors may choose the states in which they apply for privileges to practice under the compact. Differing from the MSL model, under the Counseling Compact, counselors would need to apply for privileges in individual states where they wish to practice if these states have passed legislation to join the compact group (Counseling Compact, n.d.). This may involve passing jurisprudence exams for some states. However, licensing renewal and continuing education would only be required in accordance with the home state standards and process (Counseling Compact, 2020). According to the model legislation, each state is also able to set a fee for privileges to practice. A central Compact Commission oversees the process of privilege applications and exchange of information regarding ethical violations across privilege states.

As states have varying titles for professional counselors, the general requirements for compact eligibility include counselors who have taken and passed a national exam, have completed required supervision in accordance with their home state requirements, and hold a 60–semester-hour, or 90–quarter-hour, master’s degree. There is language in the model legislation specifying that counselors need to complete 60 semester hours, or 90 quarter hours, of graduate coursework in areas with a counseling focus to accommodate states that do not require a 60-hour master’s degree for licensure. The Counseling Compact (2020) does not include other professions (e.g., marriage and family therapists) and, for the purposes of defining applicable counseling license types, requires counselors be able to independently assess, diagnose, and treat clients. A key element of the Counseling Compact is that counselors are required to adhere to the individual state regulations and rules for each state where they exercise PTP. In the future, this may ultimately mean that counselors must simultaneously understand and navigate rules and regulations for potentially 20 different states as they practice using TMH across state lines. The following illustrative case example describes Sam, a licensed professional counselor, who requests privileges to practice online with a client in another state through the Counseling Compact.

Case Example

Ethical practice in multiple states entails more than just applying for privileges through the Compact Commission. The following case example illustrates how an independently licensed professional counselor would provide services in multiple states as part of the Counseling Compact. The compact provides avenues for the expansion of the availability of TMH services. However, counselors must mindfully apply ethical guidelines and adhere to state rules in using such privileges to practice, thus avoiding licensing complaints, liability, and client harm.

The Case of Sam
     Sam is an independently licensed professional counselor in their home state of Nebraska. Sam has the National Certified Counselor (NCC) and Board Certified-TeleMental Health Provider (BC-TMH) credentials. Nebraska just passed legislation and became part of the Counseling Compact. To practice as part of the Counseling Compact, Sam first confirms which states are members of the compact. Then Sam joins the compact through the Nebraska Board for Mental Health Practice. Sam has two potential clients for whom they would like to provide TMH services. These clients reside in Utah and Colorado. Sam verifies that both states have passed legislation to be part of the Counseling Compact. Sam applies for privileges in both Utah and Colorado. Sam is required to take a jurisprudence exam before being granted privileges in Colorado through the compact. Sam also may be required to pay fees for privileges in these states. After Sam is approved for privileges in Colorado and Utah through the Compact Commission, they are ready to practice via TMH in each state. Sam creates separate professional disclosure statements they will use for clients in Colorado and Utah. They create necessary forms and consider how they will verify the location and identity of the clients they will see via TMH. Sam reads and understands all rules and statutes for Utah and Colorado related to licensure. This includes understanding the scope of practice and any unique rules of conducting TMH in these states. Sam also makes sure their professional disclosure statements meet all requirements for Utah and Colorado.

As part of their professional disclosure, Sam creates a TMH guide for clients that includes concerns and risks about counseling online, with a troubleshooting guide if the internet is unstable. This disclosure provides tips for privacy during an online counseling session for the client. The disclosure also outlines the steps Sam will use to increase confidentiality, such as wearing headphones and conducting practice in a designated private space. Sam will also be using an online telehealth platform that provides a BAA and appropriate encryption for HIPAA compliance. This platform also allows for secure document signing and document transfer. Sam creates a protocol for TMH, which includes verifying client identity with a copy of photo identification provided as part of the intake process. Sam also plans to complete a safety plan with each new client in Utah and Colorado as part of the intake process. This safety plan will include a release of information to contact a local support person in case of an emergency and looking up the local law enforcement dispatch phone numbers for the client’s primary location in case of emergency. Sam also is sure to let all clients know about the 988 National Suicide Lifeline as part of this process.

Sam’s protocols also include asking the client to verify their location verbally at the beginning of each session and documenting this in their case notes. Sam also notes in their protocols they must have new forms completed should a client move their primary residence, verifying the client is still in a state where Sam has privileges. Once Sam has updated the appropriate forms and created their protocols, they begin to engage in services with their new clients using privileges from the Counseling Compact. After a few weeks, Sam gets a referral for a new client located in Florida. Because Florida has passed legislation to be part of the Counseling Compact, Sam repeats this process in working through the compact to gain privileges to practice in Florida and creates a new disclosure statement for this state.

Practicing With Compact Privileges
     To gain privileges to practice, according to Counseling Compact legislation (Counseling Compact, 2020), Sam would be responsible for submitting paperwork to their licensing board as well as paying any required fees to participate in the compact. Not doing so would be considered practicing without a license in those states. Therefore, in the case example, Sam did not automatically gain privileges to practice in all compact states as soon as Nebraska passed the Counseling Compact legislation. They had to apply for state privileges once they joined the compact through their home state of Nebraska (see Figure 1). Though counselors will not be required to meet any additional reciprocity requirements, they could be required to take the jurisprudence exam for specific states before they are able to provide services (Counseling Compact, 2020).

Figure 1
Flowchart for Seeking Compact Privileges in Another State

 

Beyond applying for privileges through the Compact Commission, Sam will need to consider other administrative aspects of TMH practice to engage in ethical practice (ACA, 2014, H.1.a.), which would support their ability to later work through ethical dilemmas and avoid disciplinary action with their licensing board. For each compact state, Sam will be responsible for reading, understanding, and abiding by all rules and statutes for the states in which they practice (ACA, 2014, H.1.b.). This means being responsible for the rules of not just their home state, but of all states in which they hold privileges. This may entail having various scopes of practice or rules in different states. Sam would need to review the rules for Utah, Colorado, and Florida to ensure their typical treatment modalities would be permitted under the scope of practice in each state. For example, in some states Sam may not be able to provide a client with a diagnosis according to their counseling scope of practice.

With each state having its own rules, statutes, and licensing board, it would be considered best practice for Sam to have a disclosure statement that is specific to each compact state. Sam will be responsible for having updated disclosure statements that align with the rules and statutes for each compact state licensing board to review as well as for their prospective clients within each compact state. For both the benefit of the client and the protection of Sam, Sam’s disclosure statements will include their Nebraska licensing information, information about the Counseling Compact as well as a definition of PTP, and information specific to the corresponding compact state licensing board if the client needs to file a complaint. For clients in Utah, complaints would be filed with the Utah Division of Occupational and Professional Licensing. For clients in Colorado, complaints would be filed with the Colorado Department of Regulatory Agencies. In Florida, complaints would be filed with the Florida Department of Health. Sam’s disclosure statements for each state would need to have this information listed. Complaints filed in privilege states would potentially result in revocation of PTP in said state, and disciplinary actions would be reported to the Compact Commission.

The Counseling Compact also does not include insurance billing privileges for each state, so Sam will need to explore the ability to join insurance panels or be approved to bill Medicaid in each state. They may also choose to only take out-of-pocket fees for clients in different states, in which case they would need to consider a means of securely collecting payments or working with a billing service.

TMH Practice
     When Sam is providing TMH services in the states of Colorado, Florida, and Utah via the Counseling Compact, they need to complete and obtain informed consent, which is a necessary standard of care (ACA, 2014, H.2.a.). When generating informed consent, Sam needs to gain consent (in writing) in real time and in accordance with the laws of all practice states, as some states have specific regulations (Kramer et al., 2015). Further, consent needs to be gained if the session needs to be recorded for any reason (e.g., consultation, education, legal). Then Sam needs to ensure their telehealth platform/software is secure, private, confidential, and in compliance with HIPAA, as it is important for them to use appropriate technologies, understand privacy requirements, and attend to any issues related to liability of technology use to ensure compliance with their scope of practice (Kramer et al., 2015).

Sam will need to consider a workflow of administrative protocols related to their TMH work (see Figure 2). This will address ethical issues that could arise when working with clients in multiple states. Sam has sought additional training with their BC-TMH credential to help with competency and considerations of ethical practice for TMH. Before engaging in TMH practice, Sam has prepared for the addition of these services by creating guidelines for clients to access online services (ACA, 2014, H.4.e.). Sam selected an online platform that includes secure video conferencing, case note storage, and file
transfer. Sam has arranged to use a HIPAA-compliant video platform with a BAA, and they have considered how to facilitate secure exchange of files with clients and how to obtain client signatures securely from a distance. Sam would also need to make this BAA available to clients upon request. If Sam opted to transfer documents virtually via email, the files would need to be encrypted and password protected, and Sam would need to make sure methods of communication meet state regulations regarding encryption. Discussing in detail with the client the most secure way to provide documentation in accordance with state statutes will be important (ACA, 2014, H.2.d.). Considering potential barriers of insurance billing for clients in other states, Sam has created a specific Good Faith Estimate of cost for each state with an out-of-pocket rate listed using a sliding scale to comply with the No Surprises Act (U.S. Department of Labor, 2022). Sam will also consider managing crises with clients in other states.

Figure 2
Workflow for Ethical Telehealth Services Across State Lines Under the Counseling Compact


 

Additional Considerations
     Although Sam has considered how to verify the identity and location of each client (ACA, 2014, H.3.), there is still the possibility a client reports to Sam they are in a compact state, when in fact they are not in a compact state either temporarily or permanently. This could lead to a formal complaint that Sam, unknowingly, was practicing in a non-compact state where they are not licensed. To prevent possible disciplinary action, Sam asks the client where they are located at the beginning of each session, even if they recognize the background of the client. Sam is sure to document the location of both themselves and the client in the clinical note for each session. Sam also makes sure to document where the client is living, working, or going to school. If possible, the client’s insurance policy or photo identification should corroborate their location. Sam will need to have this documentation in the event of a formal complaint. In this case, Sam could demonstrate due diligence in confirming the client’s location is within a compact state where Sam has privileges. Sam can then show the corresponding licensing boards and the Compact Commission they believed they were practicing within the compact to the best of their knowledge.

If a client moves, Sam will need to document the new mailing, work, or school addresses. This and any other corresponding information would lead Sam to believe the client is in the reported location which is in fact within a compact state where Sam holds privileges. Finally, it is important to note that if Sam or the client moves out of the compact state and into a state not part of the compact, services must immediately stop. Sam has included this in every disclosure that they offer a client (ACA, 2014, H.2.a.). Sam states in the disclosure that if either party relocates outside of a compact state, Sam is then responsible for finding the client possible referrals either in the client’s location or within the compact so the client can continue care. By discussing this at the beginning of the therapeutic relationship as part of informed consent, Sam makes the transition easier and more efficient for the client if a transfer of care needs to occur. In thoughtfully preparing to use privileges offered through the Counseling Compact, Sam has carefully ensured they are engaging in ethical and legal TMH practice from first contact with a client to termination of services.

Finally, it would be helpful for Sam to identify individuals with whom they can consult should ethical issues arise (ACA, 2014, I.2.c.). Ideally, these individuals would have good knowledge of TMH. Sam might also take advantage of consultation opportunities through their state licensing board and professional organizations. Sam would also identify an ethical decision-making model to use when ethical dilemmas arise to document how ethical decisions were made (ACA, 2014, I.1.b.).

Implications for Telehealth Practice via the Counseling Compact

The Compact Commission is in the process of setting up systems and processes for granting privileges for compact states. The application process for compact privileges is anticipated to open in 2024. Counselors who hope to participate in the Counseling Compact should first verify that the state in which they currently practice has passed legislation to become part of the compact. If that is not the case, there is an opportunity for advocacy with state legislatures to pass compact legislation to allow their state to join the Compact Commission. Counselors who are practicing in states that have already passed Counseling Compact legislation should review their TMH workflow and guidelines. There is an opportunity to establish administrative workflows and documentation, as well as review HIPAA compliance of all electronic systems being used for current practice.

Counselors should also review malpractice insurance policies to ensure TMH is covered by their current policy. Counselors may begin to research and review statutes and rules for states where they hope to gain privileges as part of the compact. They may also prepare for jurisprudence exams if required in states where they hope to have privileges. Counselors can also draft a professional disclosure statement and other necessary documents for TMH that can be adapted for different states.

Given the forthcoming revision of the ACA Code of Ethics, we propose that the H.1. Knowledge and Legal Considerations section be updated to incorporate additional guidelines for conducting ethical telehealth practice. Notably, these guidelines should emphasize the establishment of a crisis plan when rendering telehealth services, including a local law enforcement dispatch phone number and consent for disclosure for a designated local emergency contact. Counselors also have an ethical obligation to be familiar with local referral resources when working with clients in different states. Furthermore, the ACA Code of Ethics should underscore the necessity of a telehealth protocol or workflow as preparation for engaging in ethical telehealth practice.

Conclusion

The Counseling Compact creates new and exciting possibilities for counselors to have improved portability of licensure through practice privileges. The compact also addresses barriers to broader access and equity in TMH for various populations across the nation. However, before counselors enroll in the compact, there is a critical need to consider how to engage in TMH ethically when working with clients online in different states. The included guidelines and example workflow processes are important considerations for counselors preparing to apply for privileges within the Counseling Compact. These preparatory steps will help counselors to be prepared to apply for compact privileges when the portal becomes available.

 

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

 

References

Abraham, A., Jithesh, A., Doraiswamy, S., Al-Khawaga, N., Mamtani, R., & Cheema, S. (2021). Telemental health use in the COVID-19 pandemic: A scoping review and evidence gap mapping. Frontiers in Psychiatry, 12, 748069. https://doi.org/10.3389/fpsyt.2021.748069

Adashi, E. Y., Cohen, I. G., & McCormick, W. L. (2021). The Interstate Medical Licensure Compact: Attending to the underserved. JAMA, 325(16), 1607–1608. https://doi.org/10.1001/jama.2021.1085

Adrian, L. (2017). The Physical Therapy Compact: From development to implementation. International Journal of Telerehabilitation, 9(2), 59–62. https://doi.org/10.5195/ijt.2017.6237

American Association of State Counseling Boards. (2022). History of AASCB. https://wwwaascborg.wildapricot.org/History-of-AASCB

American Counseling Association. (2014). ACA code of ethics. https://www.counseling.org/resources/aca-code-of-ethics.pdf

American Counseling Association. (2018). ACA takes action toward interstate licensure portability. https://www.counseling.org/news/aca-blogs/aca-government-affairs-blog/aca-government-affairs-blog/2018/12/19/aca-takes-action-toward-interstate-licensure-portability

American Counseling Association. (2021). Counseling compact. https://www.counseling.org/government-affairs/counseling-compact

American Mental Health Counselors Association. (2021). Counselor licensure interstate portability endorsement and reciprocity plan.

Apgar, D. (2022). Social work licensure portability: A necessity in a post-COVID-19 world. Social Work, 67(4), 381–390. https://doi.org/10.1093/sw/swac031

Appleton, R., Williams, J., Vera San Juan, N., Needle, J. J., Schlief, M., Jordan, H., Sheridan Rains, L., Goulding, L., Badhan, M., Roxburgh, E., Barnett, P., Spyridonidis, S., Tomaskova, M., Mo, J., Harju-Seppänen, J., Haime, Z., Casetta, C., Papamichail, A., Lloyd-Evans, B., . . . Johnson, S. (2021). Implementation, adoption, and perceptions of telemental health during the COVID-19 pandemic: Systematic review. Journal of Medical Internet Research, 23(12), e31746. https://doi.org/10.2196/31746

Baumann, B. C., MacArthur, K. M., & Michalski, J. M. (2020). The importance of temporary telehealth parity laws to improve public health during COVID-19 and future pandemics. International Journal of Radiation Oncology, Biology, Physics, 108(2), 362–363. https://doi.org/10.1016/j.ijrobp.2020.05.039

Bloom, J., Gerstein, L. H., Tarvydas, V., Conaster, J., Davis, E., Kater, D., Sherrard, P., & Esposito, R. (1990). Model legislation for licensed professional counselors. Journal of Counseling & Development, 68(5), 511–523.
https://doi.org/10.1002/j.1556-6676.1990.tb01402.x

Bobby, C. L. (2013). The evolution of specialties in the CACREP standards: CACREP’s role in unifying the profession. Journal of Counseling & Development, 91(1), 35–43. https://doi.org/10.1002/j.1556-6676.2013.00068.x

Churcher, J. (2012). On: Skype and privacy. The International Journal of Psychoanalysis, 93(4), 1035–1037.
https://doi.org/10.1111/j.1745-8315.2012.00610.x

Council for the Accreditation of Counseling and Related Educational Programs. (2017). Fall 2017 CACREP connection. https://www.cacrep.org/newsletter/fall-2017-cacrep-connection/#accreditation

Counseling Compact. (n.d.). Interstate compacts vs. universal license recognition. https://counselingcompact.org/wp-content/uploads/2021/06/Compacts-Universal-Recognition-Explained-Final-Counseling-Compact.pdf

Counseling Compact. (2020). Counseling compact model legislation. https://counselingcompact.org/wp-content/uploads/2021/01/Final_Counseling_Compact_.pdf

Counseling Compact. (2022). Compact map. https://counselingcompact.org/map

Dart, E. H., Whipple, H. M., Pasqua, J. L., & Furlow, C. M. (2016). Legal, regulatory, and ethical issues in telehealth technology. In J. K. Luiselli & A. J. Fischer (Eds.), Computer-assisted and web-based innovations in psychology, special education, and health (pp. 339–363). Academic Press.

Evans, S. (2015). The Nurse Licensure Compact: A historical perspective. Journal of Nursing Regulation, 6(3), 11–16. https://doi.org/10.1016/s2155-8256(15)30778-x

Goodstein, L. D. (2012). The interstate delivery of psychological services: Opportunities and obstacles. Psychological Services, 9(3), 231–239. https://doi.org/10.1037/a0027821

Healthcare Providers Service Organization. (2019). Counselor liability claim report: 2nd edition. https://www.hpso.com/getmedia/ac13a1d8-3bdd-4117-ad36-8fba7c419153/counselor-claim-report-second-edition.pdf

Health Resources & Services Administration. (2022). Health equity in telehealth. https://telehealth.hhs.gov/providers/health-equity-in-telehealth

Hilty, D. M., Maheu, M. M., Drude, K. P., Hertlein, K. M., Wall, K., Long, R. P., & Luoma, T. L. (2017). Telebehavioral health, telemental health, e-therapy and e-health competencies: The need for an interprofessional framework. Journal of Technology in Behavioral Science, 2(3), 171–189.
https://doi.org/10.1007/s41347-017-0036-0

Hirko, K. A., Kerver, J. M., Ford, S., Szafranski, C., Beckett, J., Kitchen, C., & Wendling, A. L. (2020). Telehealth in response to the COVID-19 pandemic: Implications for rural health disparities. Journal of the American Medical Informatics Association, 27(11), 1816–1818. https://doi.org/10.1093/jamia/ocaa156

Interstate Commission of Nurse Licensure Compact Administrators. (2021). Final rules. https://www.ncsbn.org/public-files/FinalRulesadopted81120clean_ed.pdf

Jacob, S., Decker, D. M., & Hartshorne, T. S. (2011). Ethics and law for school psychologists (6th ed.). Wiley.

Kaplan, D. M., & Gladding, S. T. (2011). A vision for the future of counseling: The 20/20 Principles for Unifying and Strengthening the Profession. Journal of Counseling & Development, 89(3), 367–372.
https://doi.org/10.1002/j.1556-6678.2011.tb00101.x

Kaplan, D. M., & Kraus, K. L. (2018). Building blocks to portability: Culmination of the 20/20 initiative. Journal of Counseling & Development, 96(2), 223–228. https://doi.org/10.1002/jcad.12195

Kramer, G. M., Kinn, J. T., & Mishkind, M. C. (2015). Legal, regulatory, and risk management issues in the use of technology to deliver mental health care. Cognitive and Behavioral Practice, 22(3), 258–268.
https://doi.org/10.1016/j.cbpra.2014.04.008

Kramer, G. M., & Luxton, D. D. (2016). Telemental health for children and adolescents: An overview of legal, regulatory, and risk management issues. Journal of Child and Adolescent Psychopharmacology, 26(3), 198–203. https://doi.org/10.1089/cap.2015.0018

Levitt, D. H., Farry, T. J., & Mazzarella, J. R. (2015). Counselor ethical reasoning: Decision-making practice versus theory. Counseling and Values, 60(1), 84–99. https://doi.org/10.1002/j.2161-007X.2015.00062.x

Litam, S. D. A., & Hipolito-Delgado, C. P. (2021). When being “essential” illuminates disparities: Counseling clients affected by COVID-19. Journal of Counseling & Development, 99(1), 3–10. https://doi.org/10.1002/jcad.12349

Litwak, J. B., & Mayer, J. (2021). Developments in interstate compact law and practice 2020. Urban Lawyer, 51(1), 99–100. https://www.americanbar.org/groups/state_local_government/publications/urban_lawyer/2021/51-1/developments-interstate-compact-law-and-practice-2020

Lo, J., Rae, M., Amin, K., Cox, K., Panchal, N., & Miller, B. F. (2022). Telehealth has played an outsized role meeting mental health needs during the COVID-19 pandemic. Medical Benefits, 39(7), 8–9.

Manz, D. (2015). Legislation, regulation, and ordinance. In D. C. Cone, J. H. Brice, T. R. Delbridge, & J. B. Myers (Eds)., Emergency medical services: Clinical practice and systems oversight, clinical aspects of EMS (2nd ed., pp. 36–43). Wiley.

Mascari, J. B., & Webber, J. (2013). CACREP accreditation: A solution to license portability and counselor identity problems. Journal of Counseling & Development, 91(1), 15–25. https://doi.org/10.1002/j.1556-6676.2013.00066.x

Maurya, R. K., Bruce, M. A., & Therthani, S. (2020). Counselors’ perceptions of distance counseling: A national survey. Journal of Asia Pacific Counseling, 10(2), 1–22. https://doi.org/10.18401/2020.10.2.3

Morgan, S. D., Zeng, F.-G., & Clark, J. (2022). Adopting change and incorporating technological advancements in audiology education, research, and clinical practice. American Journal of Audiology, 31(3S), 1052–1058. https://doi.org/10.1044/2022_aja-21-00215

Morland, L. A., Poizner, J. M., Williams, K. E., Masino, T. T., & Thorp, S. R. (2015). Home-based clinical video teleconferencing care: Clinical considerations and future directions. International Review of Psychiatry, 27(6), 504–512. https://doi.org/10.3109/09540261.2015.1082986

National Board for Certified Counselors. (2016). National Board for Certified Counselors (NBCC) policy regarding the provision of distance professional services.

National Board for Certified Counselors. (2017). Joint statement on a national counselor licensure endorsement process. https://www.nbcc.org/assets/portability/portability-statement-endorsement-process.pdf

National Council of State Boards of Nursing. (2015). Nurse licensure compact.

National Institute of Mental Health. (2021). What is telemental health? (NIH Publication No. 21-MH-8155). U.S. Department of Health and Human Services, National Institutes of Health. https://www.nimh.nih.gov/sites/default/files/health/publications/what-is-telemental-health/what-is-telemental-health.pdf

Remley, T. P., Jr., & Herlihy, B. (2010). Ethical, legal, and professional issues in counseling (3rd ed.). Merrill.

Richards, D., & Viganó, N. (2013). Online counseling: A narrative and critical review of the literature. Journal of Clinical Psychology, 69(9), 994–1011. https://doi.org/10.1002/jclp.21974

Richardson, L. K., Frueh, B. C., Grubaugh, A. L., Egede, L., & Elhai, J. D. (2009). Current directions in videoconferencing tele-mental health research. Clinical Psychology: Science and Practice, 16(3), 323–338. https://doi.org/10.1111/j.1468-2850.2009.01170.x

Sheperis, D. S., Henning, S. L., & Kocet, M. M. (2016). Ethical decision making for the 21st century counselor. SAGE.

Sheperis, D. S., & Smith, A. (2021). Telehealth best practice: A call for standards of care. Journal of Technology in Counselor Education and Supervision, 1(1), 27–35. https://doi.org/10.22371/tces/0004

Shore, J. H., Hilty, D. M., & Yellowlees, P. (2007). Emergency management guidelines for telepsychiatry. General Hospital Psychiatry, 29(3), 199–206. https://doi.org/10.1016/j.genhosppsych.2007.01.013

Shore, J. H., Yellowlees, P., Caudill, R., Johnston, B., Turvey, C., Mishkind, M., Krupinski, E., Myers, K., Shore, P., Kaftarian, E., & Hilty, D. (2018). Best practices in videoconferencing-based telemental health April 2018. Telemedicine and e-Health, 24(11), 827–832. https://doi.org/10.1089/tmj.2018.0237

Shreck, E., Nehrig, N., Schneider, J. A., Palfrey, A., Buckley, J., Jordan, B., Ashkenazi, S., Wash, L., Baer, A. L., & Chen, C. K. (2020). Barriers and facilitators to implementing a U.S. Department of Veterans Affairs Telemental Health (TMH) program for rural veterans. Journal of Rural Mental Health, 44(1), 1–15.
https://doi.org/10.1037/rmh0000129

Siegel, A., Zuo, Y., Moghaddamcharkari, N., McIntyre, R. S., & Rosenblat, J. D. (2021). Barriers, benefits and interventions for improving the delivery of telemental health services during the coronavirus disease 2019 pandemic: A systematic review. Current Opinion in Psychiatry, 34(4), 434–443.
https://doi.org/10.1097/YCO.0000000000000714

Slomski, A. (2020). Telehealth success spurs a call for greater post–COVID-19 license portability. JAMA324(11), 1021–1022. https://doi.org/10.1001/jama.2020.9142

Smith, W. R., Atala, A. J., Terlecki, R. P., Kelly, E. E., & Matthews, C. A. (2020). Implementation guide for rapid integration of an outpatient telemedicine program during the COVID-19 pandemic. Journal of the American College of Surgeons, 231(2), 216–222. https://doi.org/10.1016/j.jamcollsurg.2020.04.030

Turvey, C., Coleman, M., Dennison, O., Drude, K., Goldenson, M., Hirsch, P., Jueneman, R., Kramer, G. M., Luxton, D. D., Maheu, M. M., Malik, T. S., Mishkind, M. C., Rabinowitz, T., Roberts, L. J., Sheeran, T., Shore, J. H., Shore, P., van Heeswyk, F., Wregglesworth, B., . . . Bernard, J. (2013). ATA practice guidelines for video-based online mental health services. Telemedicine and e-Health, 19(9), 722–730.
https://doi.org/10.1089/tmj.2013.9989

U.S. Department of Health & Human Services. (2013). Business associate contracts. https://www.hhs.gov/hipaa/for-professionals/covered-entities/sample-business-associate-agreement-provisions/index.html

U.S. Department of Health & Human Services. (2021). Notification of enforcement discretion for telehealth remote communications during the COVID-19 nationwide public health emergency. https://www.hhs.gov/hipaa/for-professionals/special-topics/emergency-preparedness/notification-enforcement-discretion-telehealth/index.html

U.S. Department of Labor. (2022). Requirements related to surprise billing: Final rules. https://www.dol.gov/sites/dolgov/files/EBSA/about-ebsa/our-activities/resource-center/faqs/requirements-related-to-surprise-billing-final-rules-2022.pdf

Williams, A. E., Weinzatl, O. L., & Varga, B. L. (2021). Examination of family counseling coursework and scope of practice for professional mental health counselors. The Family Journal, 29(1), 10–16.
https://doi.org/10.1177/1066480720978536

Amanda DeDiego, PhD, NCC, BC-TMH, LPC, is an associate professor at the University of Wyoming. Rakesh K. Maurya, PhD, is an assistant professor at the University of North Florida. James Rujimora, MEd, EdS, is a doctoral student at the University of Central Florida. Lindsay Simineo, MA, NCC, LPC, is Legislative Advocate for the Wyoming Counseling Association. Greg Searls is Executive Director of Professional Licensing Boards for the Wyoming Department of Administration & Information. Correspondence may be addressed to Amanda DeDiego, 125 College Dr, UU 431, Casper, WY 82601, adediego@uwyo.edu.

The Factor Structure of the Outcome Questionnaire-45.2 with Economically Vulnerable Adults

Dalena Dillman Taylor, Saundra M. Tabet, Megan A. Whitbeck, Ryan G. Carlson, Sejal Barden, Nicole Silverio

Individuals living in poverty have higher rates of mental health disorders compared to those not living in poverty. Measures are available to assess adults’ levels of psychological distress; however, there is limited support for instruments to be used with a diverse population. The purpose of our study was to examine the factor structure of Outcome Questionnaire-45.2 scores with an economically vulnerable sample of adults (N = 615), contributing to the evidence of validity of the measure’s scores in diverse mental health settings. Implications for professional counselors are considered, including clinical usage of the brief Outcome Questionnaire-16 and key critical items.

Keywords: poverty, psychological distress, factor structure, Outcome Questionnaire-45.2, validity

 

In the United States, it is estimated that 34 million adults live in poverty (i.e., income less than $12,880 per year), and poverty is a significant factor contributing to poor mental and physical health outcomes (Hodgkinson et al., 2017). Poverty, or economic vulnerability, refers to the extent to which individuals have difficulty living with their current level of income, increasing the risk for adverse social and economic consequences (Semega et al., 2021). Economically vulnerable adults often experience greater social inequality, lower educational attainment, less economic mobility (Stanford Center on Poverty and Inequality, 2015), and difficulty securing full-time employment (Dakin & Wampler, 2008), which leads to increased distress (Lam et al., 2019). Lower income levels are also associated with several mental health conditions (e.g., anxiety, depression, suicide attempts; Santiago et al., 2011). Further, Lam and colleagues (2019) found strong negative associations between income, socioeconomic status, and psychological distress.

To effectively support their clients, counselors must understand the unique context and financial stressors related to living in poverty. Incorporating poverty-sensitive measures into assessment and evaluation practices is essential to providing culturally responsive care that considers the systemic and environmental barriers of poverty (Clark et al., 2020). Implementing culturally responsive assessments ensures that counselors use outcome measures that are attuned to poverty-related experiences (Clark et al., 2020). Such measures can help counselors identify and prioritize treatment planning approaches and acknowledge the reality that economic disadvantages create for clients (Foss-Kelly et al., 2017). However, the availability of poverty-sensitive assessments is limited.

Measuring Psychological Distress in Adults Living in Poverty
     Because of the risk of mental health issues related to economic vulnerability, assessments with evidence of validity and reliability that measure psychological distress relative to income are warranted. Professional counselors can individualize their therapeutic approach to meet the needs of this population with the assistance of accurate assessments of related mental health conditions. Naher and colleagues (2020) noted the need for individual-level data as well as interventions specifically targeted to adults living in poverty. Although outcome assessments exist to measure psychological distress or severity of mental illness symptoms (e.g., Beck Depression Inventory [BDI], Beck et al., 1961; Generalized Anxiety Disorder Screener [GAD-7], Löwe et al., 2008; Patient Health Questionnaire-9 [PHQ-9], Kroenke et al., 2001), there is a lack of measures with evidence of validity and reliability with economically vulnerable adult populations. Therefore, our investigation examined the factor structure of the Outcome Questionnaire-45.2 (OQ-45.2; Lambert et al., 2004) with an economically vulnerable adult population, increasing the applicability of the measure in mental health settings.

Outcome Questionnaire-45.2

The OQ-45.2 (Lambert et al., 2004) is one of the most widely used outcome measures of psychological distress in applied mental health settings (Hatfield & Ogles, 2004). The OQ-45.2 assists professional counselors with monitoring client progress and can be administered multiple times throughout treatment, as it is sensitive to changes over time (Lambert et al., 1996). The OQ-45.2 has been implemented in outcome-based research with diverse populations such as university counseling center clients (Tabet et al., 2019), low-income couples (Carlson et al., 2017), and ethnic minority groups (Lambert et al., 2006). Lambert et al. (1996) reported strong test-retest reliability (r = .84) and internal consistency (α = .93) for the OQ-45.2, based on a sample of undergraduate students (n = 157) and a sample of individuals receiving Employee Assistance Program services (n = 289). However, researchers have yet to investigate the psychometric properties of the OQ-45.2 with an economically disadvantaged, diverse population.

Given the utility of the OQ-45.2 as a client-reported feedback measure, clinicians can use the OQ-45.2 in a variety of ways to evaluate client progress, including measuring changes in individual distress across the course of counseling and before and after specific treatment interventions, as well as to glean a baseline level of distress at the start of counseling (Lambert, 2017). For example, one study used the OQ-45.2 as a primary outcome measure for anxiety symptoms in clients engaging in cognitive behavioral therapy (Levy et al., 2020). The OQ-45.2 was administered at the beginning of each weekly counseling session and change scores were calculated between each session, which helped clinicians understand that about half of their sample reported clinically significant reductions in symptoms in just nine sessions (Levy et al., 2020). This example demonstrates how the OQ-45.2 can be implemented to monitor treatment outcomes and improve the duration and efficiency of counseling. A clinician can also use salient items as part of the intake clinical interview to encourage clients to elaborate on the specific symptoms they are experiencing, and how they may be impacting their functioning, across a variety of clinical settings (Espiridion et al., 2021; Lambert, 2017; Levy et al., 2020).

Factor Structure of OQ-45.2
     Researchers contested the factor structure proposed by Lambert et al. (2004), suggesting the need for further validation of the three-factor oblique measurement model and exploration of other possible factor structures (e.g., Kim et al., 2010; Mueller et al., 1998; Rice et al., 2014; Tabet et al., 2019). Mueller and colleagues (1998) examined three models: (a) a one-factor model, (b) a two-factor oblique model, and (c) a three-factor oblique model, none of which fit the data well. In addition, the factors in the three-factor model were highly correlated, ranging from .83 to .91, asserting that the subscales may not be statistically indistinguishable and the OQ-45.2 might be a unidimensional measure of global distress.

Kim and colleagues (2010) also explored three models to assess adequate fit of the data: (a) a one-factor model, (b) a three-factor model, and (c) a revised 22-item four-factor model. Indicating weak support for the OQ-45.2’s factorial validity across all models, researchers cautioned against widespread utilization in mental health and research settings, encouraging further psychometric exploration and validation of the OQ-45.2 (Kim et al., 2010).

Rice and colleagues (2014) found evidence to support a two-factor OQ-45.2 model that included (a) overall maladjustment and (b) substance use. Results indicated relatively good fit (comparative fit index [CFI] = .990, root-mean-square error of approximation [RMSEA] = .068) for a two-factor measure with 11 items, which demonstrated better model fit than the original three-factor model
(CFI = .840, RMSEA = .086 [90% confidence interval {CI} = .085, .087]). Overall, multiple researchers have demonstrated poor fit for the original factor structure of the OQ-45.2 (Kim et al., 2010; Mueller et al., 1998; Rice et al., 2014; Tabet et al., 2019), supporting the need for further validation for using the OQ-45.2 with samples of adults living in poverty.

This study’s primary aim is to examine the factor structure of the OQ-45.2 with an economically vulnerable sample to enhance the generalizability of the OQ-45.2 in mental health settings. Therefore, the following research questions guided our study:

RQ1. What is the factor structure of OQ-45.2 scores with a sample of adults living in poverty?
RQ2. What is the internal consistency reliability of the abbreviated 16-item OQ-45.2 scores with a sample of adults living in poverty?
RQ3. What is the test-retest reliability of the abbreviated 16-item OQ-45.2 scores with a sample of adults living in poverty?

Method 

Participants and Procedures
     Participants comprised a sub-sample from a grant-funded, community-based, relationship education program for individuals and couples at a university in the Southeastern United States. The project was funded through the U.S. Department of Health and Human Services, Administration for Children and Families, Office of Family Assistance (Grant #90FM0078). Study recruitment strategically involved passive and active recruitment strategies (Carlson et al., 2014) from various community locations that primarily serve low-income individuals and families (e.g., libraries, employment offices). Participants met inclusion criteria if they were at least 18 years old and interested in learning about healthy relationships. The relationship education intervention utilized was an evidence-based curriculum that taught individuals tools to improve their relationships in a small group setting (Prevention and Relationship Education Program [PREP]; Pearson et al., 2015).

We obtained ethical approval from the university’s IRB prior to data collection. Each person participated in a group intake session that consisted of a review of the informed consent; a battery of assessments, including the OQ-45.2; and a brief activity. Study participants (N = 615) included in this current analysis consented between July 2015 and June 2019.

Demographic Information
     We collected demographic data as part of this study, which included gender, age, ethnicity, income, educational level, working status, and marital status (see Table 1). The majority of participants fell below the poverty line when factoring in number of children and/or under- or unemployment. Therefore, our sample consisted of a diverse population, including variations in income, age, ethnicity, and race.

Table 1
Participant Demographic Characteristics

Descriptive Characteristic                                                                                            Total Sample (n, %)

Age

18–20 years

21–24 years

25–34 years

35–44 years

45–54 years

55–64 years

65 years or older

 

34 (5.5)

52 (8.5)

130 (21.1)

139 (22.6)

137 (22.3)

91 (14.8)

32 (5.2)

Gender (female) 498 (81.0)
Race
     American Indian or Alaska Native   18 (2.9)
     Asian   19 (3.1)
     Black or African American 176 (28.6)
     Native American or Pacific Islander     2 (0.3)
     White 248 (40.3)
     Other 144 (23.4)
Ethnicity

Hispanic or Latino

Not Hispanic or Latino

Income

 

258 (42.0)

356 (57.9)

    Less than $500 216 (35.1)
    $501–$1,000 108 (17.6)
    $1,001–$2,000

$2,001–3,000

    $3,001–$4,000

    $4,001–$5,000

    More than $5,000

124 (20.2)

81 (13.2)

28 (4.6)

18 (2.9)

18 (2.9)

Educational Level

No degree or diploma earned

    High school diploma

Some college but no degree completion

Associate degree

Bachelor’s degree

Master’s / advanced degree

 

24 (3.9)

18 (2.9)

75 (12.2)

66 (10.7)

134 (21.8)

77 (12.5)

Marital Status

Married

     Engaged

Divorced

Widowed

Never Married

 

93 (15.1)

11 (1.8)

164 (26.7)

24 (3.9)

270 (43.9)

Employment Status

Full-time employment

Part-time employment

Temporary, occasional, or seasonal, or odd jobs for pay

Not currently employed

     Employed, but number of hours change from week to week

     Selected multiple responses

Number of Children

0

1

2

3

4

5

6

 

227 (36.9)

83 (13.5)

41 (6.7)

207 (33.7)

29 (13.5)

6 (1.0)

 

148 (24.1)

60 (9.8)

44 (7.2)

17 (2.8)

6 (1.0)

4 (0.7)

1 (0.4)

 

Instrument
The Outcome Questionnaire-45.2
     The OQ-45.2 is a self-report questionnaire that captures individuals’ subjective functionality in various aspects of life that can lead to common mental health concerns (e.g., anxiety, depression, substance use). The current three-factor structure of the OQ-45.2 has 45 items rated on a 5-point Likert scale, with rankings of 0 (never), 1 (rarely), 2 (sometimes), 3 (frequently), and 4 (almost always; Lambert et al., 2004). Nine OQ-45.2 items are reverse scored, with total OQ-45.2 scores calculated by summing all 45 items with a range from 0 to 180. Clinically significant changes are represented in a change score of at least 14, whether positive or negative (i.e., increased or reduced distress).

The Symptom Distress subscale (25 items) evaluates anxiety, depression, and substance abuse symptoms, as these are the most diagnosed mental health concerns (Lambert et al., 1996). The Interpersonal Relations subscale (11 items) includes items that measure difficulties and satisfaction in relationships. The Social Role Performance subscale (nine items) assesses conflict, distress, and inadequacy related to employment, family roles, and leisure activities. The OQ-45.2 also includes four critical items (Items 8, 11, 32, and 44) targeting suicidal ideation, homicidal ideation, and substance use. The Cronbach’s alpha for the OQ-45.2 in the current study was calculated at .943.

Data Analysis
     We calculated descriptive statistics on the total sample population, including the mean, standard deviations, and frequencies. Subsequently, we conducted preliminary descriptive analyses to test for statistical assumptions that included missing data, collinearity issues, and multivariate normality (Byrne, 2016). In the first analysis, we used confirmatory factor analysis (CFA) to test the factor structure of the OQ-45.2 with this population (N = 615) and subsequently used exploratory factor analysis (EFA) to evaluate revised OQ models.

We conducted CFA utilizing the original three-factor oblique model (Lambert et al., 2004) as the a priori model to test the hypothesized structure of the latent variables. In addition, based on the results of the study, we tested a series of alternative structural models outlined by Bludworth and colleagues (2010). Given the non-normal distribution, we utilized MPlus (Version 8.4) with a robust maximum likelihood (MLR) parameter estimation (Satorra & Bentler, 1994). To address missing data, we employed a full information maximum likelihood (FIML) to approximate the population parameters and produce the estimates from the sample data (Enders, 2010). Results of the CFA were evaluated using several fit indices: (a) the chi-square test of model fit (χ2; nonsignificance at p > .05 indicate a good fit [Hu & Bentler, 1999]); (b) the CFI (values larger than .95 indicate a good fit [Bentler, 1990]); (c) TLI (values larger than .95 indicate a good fit [Tucker & Lewis, 1973]); (d) RMSEA with 90% CI (values between .05 and .08 indicate a good fit [Browne & Cudeck, 1993]); and (e) standardized root-mean-square residual (SRMR; values below .08 indicate good fit [Hu & Bentler, 1999]).

Following the CFA, we conducted EFA because of poor model fit across all models and several items with outer loadings of less than 0.5 (Tabachnick & Fidell, 2019). Kline (2016) recommended researchers should not be constrained by the original factor structure when CFA indicates low outer loadings and should consider conducting an EFA because the data may not fit the original number of factors suggested. Accordingly, we conducted an EFA to test the number of factors derived from the 45-item OQ-45.2 within our population. We exceeded the recommended ratio (i.e., 10:1) of participants to the number of items (12.6:1; Costello & Osborne, 2005; Hair et al., 2010; Mvududu & Sink, 2013). We conducted a principal axis factoring with Promax rotation to determine whether factors were correlated using SPSS version 25.0. We chose parallel analysis (Horn, 1965) using the 95th percentile to determine the number of factors to retain given that previous researchers have acknowledged parallel analysis to be a superior method to extract significant factors as compared to conventional statistical indices such as Cattell’s scree test (Henson & Roberts, 2006). We used stringent criterion when identifying loading and cross-loading items such as items that indicated high (i.e., equal to or exceeding 1.00) or low communality values (i.e., less than 0.40; Costello & Osborne, 2005) and items with substantive cross-loadings (< .30 between two factor loadings; Tabachnick & Fidell, 2019) were removed. To ensure the most parsimonious model, we removed items individually from Factor 1, which has the greatest number of items, to reduce the size of the model while still capturing the greatest variance explained by the items on that factor.

Results

We screened the data and checked for statistical assumptions prior to conducting factor analysis. Little’s Missing Completely at Random (MCAR) test (Little, 1988), a multivariate extension of a simple t-test, evaluated the mean differences of the 45 items to determine the pattern and missingness of data (Enders, 2010). Given the significant chi-square, data were not missing completely at random
(χ2 = 912.062, df = 769, p < .001). However, results indicated a very small percentage of values (< 1%) were missing from each variable; therefore, supporting data were missing at random (MAR; Osborne, 2013). When data are MAR, an FIML approach to replace missing values provides unbiased parameter estimates and improves the statistical power of analyses (Enders, 2010). The initial internal consistency reliability estimates (coefficient alpha) for scores on the original OQ-45.2 model were all in acceptable ranges except for Factor 3 (see Henson & Roberts, 2006): total α = .943, Symptom Distress α = .932
(k = 25 items), Interpersonal Relations α = .802 (k = 11 items), and Social Role Performance α = .683
(k = 9 items). We also conducted Bartlett’s test of sphericity (p < .001) and the Kaiser-Meyer-Olkin value (.950), indicating the data was suitable for conducting a factor analysis. We evaluated multivariate normality of the dataset with Mardia’s multivariate kurtosis coefficient. Mardia’s coefficient of multivariate kurtosis was .458; therefore, we deemed the data to be non-normally distributed
(Hu & Bentler, 1995).

Confirmatory Factor Analysis
     We tested the developer’s original OQ-45.2 three-factor oblique model, and because of the results subsequently tested a series of alternative structural models outlined by Bludworth and colleagues (2010). Specifically, the alternative structural models tested included: (a) a three-factor orthogonal model, (b) a one-factor model, (c) a four-factor hierarchical model, and (d) a four-factor bilevel model. Table 2 presents the fit indices results in the series of CFAs. The original three-factor oblique model allowed all three factors (Social Role Performance, Interpersonal Relations, and Symptom Distress) to correlate, but resulted in a poor fit: χ2 (942, N = 615) = 3.014, p < .001; CFI = .779; TLI = .768; RMSEA = .057, 90% CI [.055, .060]; SRMR = .063. We next uncorrelated the factors and tested a three-factor orthogonal model, which also presented a poor fit with worsened fit metrics: χ2 (945, N = 615) = 3.825, p < .001; CFI = .689; TLI = .674; RMSEA = .068, 90% CI [.065, .070]; SRMR = .202. Accordingly, because the factors demonstrated high intercorrelation (rs = .94, .93, .91) in the three-factor oblique model and lack of factorial validity based on the CFA results of both three-factor models, we suspected the OQ-45.2 to be a unidimensional, one-factor model. However, the CFA revealed a poor fit to the OQ-45.2 one-factor model: χ2 (945, N = 615) = 3.197,
p < .001; CFI = .758; TLI = .747; RMSEA = .060, 90% CI [.057, .062]; SRMR = .062.

Table 2
Goodness-of-Fit Indices for the Item-Level Models of the OQ-45.2

χ2 df p χ2/df CFI TLI RMSEA 90% CI SRMR
One-Factor 3021.300 945 .000 3.197 .758 .747 .060 [.057, .062] .062
Three-Factor (orthogonal) 3615.060 945 .000 3.825 .689 .674 .068 [.065, .070] .202
Three-Factor (oblique) 2839.335 942 .000 3.014 .779 .768 .057 [.055, .060] .063
Four-Factor (hierarchical) 2839.335 942 .000 3.014 .779 .768 .057 [.055, .060] .063
Four-Factor

(bilevel)

2363.263 900 .000 2.626 .829 .812 .051 [.049, .054] .054

Note. N = 615. χ2 = chi-square; df = degrees of freedom; χ2/df = relative chi-square; CFI = comparative fit index;
TLI = Tucker-Lewis Index; RMSEA = root-mean-square error of approximation; 90% CI = 90% confidence interval;
SRMR = standardized root-mean-square residual.

 

We proceeded to test the OQ-45.2 as a four-factor hierarchical model. In this multidimensional model, the three first-order factors (Social Role Performance, Interpersonal Relations, and Symptom Distress) became a linear combination to sum a second-order general factor (g-factor) of Psychological Distress (Eid et al., 2017). Results evidenced an unacceptable overall fit to the data: χ2 (942, N = 615) = 3.014, p < .001; CFI = .779; TLI = .768; RMSEA = .057, 90% CI [.055, .060]; SRMR = .063. Last, we examined a four-factor bilevel model. In this model, the g-factor of Psychological Distress has a direct effect on items, whereas, in the hierarchal model, it had an indirect effect on items. Therefore, the items in the four-factor bilevel model load onto both their intended factors (Social Role Performance, Interpersonal Relations, and Symptom Distress) and the g-factor (Psychological Distress). Nevertheless, although the four-factor bilevel was cumulatively the best fitting OQ-45.2 factorial model, the results still yielded a poor fit:
χ2 (900, N = 615) = 2.626, p < .001; CFI = .829; TLI = .812; RMSEA = .051, 90% CI [.049, .054]; SRMR = .054.

Overall, all models demonstrated a significant chi-square (p < .001); however, this result is common in larger sample sizes (N > 400; Kline, 2016). Because the chi-square statistic is sensitive to sample size and model complexity, researchers have recommended using other fit indices (e.g., RMSEA, CFI) to determine overall model fit (Tabachnick & Fidell, 2019). Nevertheless, the levels of the CFI values (ranging from .689 to .829) and TLI values (ranging from .674 to .812) were low, and far below the recommended referential cutoff (> .90; Tucker & Lewis, 1973). Although the models’ RMSEA values were within the recommended range of .05 to .08 (Browne & Cudeck, 1993), and the majority of SRMR values were below .08 (Hu & Bentler, 1999), these were the only fit indices that met acceptable cutoffs. We further examined outer loadings for the 45 items within the factorial models and identified that all models had outer loadings (ranging from 5 to 14 items) below the 0.5 cutoff (Tabachnick & Fidell, 2019). When CFA produces low factor loadings and poor fit indices, researchers should not be constrained to the original specified number of factors and should consider conducting an EFA (Kline, 2016). Hence, we elected to conduct an EFA to explore the factor structure with this population.

Exploratory Factor Analysis
     Results from the initial EFA using principal axis factoring with the 45 OQ items produced a solution that explained 55.564% of the total variance. After multiple iterations of item deletions, we concluded with a three-factor solution. We present the internal reliability estimates of two three-factor solutions: (a) a 16-item three-factor solution—the most parsimonious—and (b) an 18-item three-factor solution, including all critical items in Table 3. We present the first three-factor solution because it was derived using stringent criteria for creating the most parsimonious solution (Costello & Osborne, 2005; Henson & Roberts, 2006; Tabachnick & Fidell, 2019), whereas the second three-factor solution included conceptual judgment determining the inclusion of the critical items from the original OQ-45.2.

Table 3
Internal Consistency Estimates

Total Symptom Distress Interpersonal Relations Social Role Performance
Original OQ-45 .943 .932 .802 .683
Total Factor 1 Factor 2 Factor 3
16-Item Model .894 .864 .840 .710
18-Item Model .896 .857 .840 .700

 

Three-Factor Solution
     Results from the parallel analysis (Horn, 1965) indicated an initial four-factor solution. Through multiple iterations (n = 9) of examining factor loadings, removing items one at a time, and reexamining parallel analysis after each deletion, our results demonstrated that a three-factor solution was the most parsimonious. We removed a total of 29 items because of low communalities (< .5), low factor loadings (< .4), and substantive cross-loadings (> .3 between two factor loadings; Tabachnick & Fidell, 2019). Before accepting the removal of these items, we added each back to the model to determine its impact on the overall model. No items improved the model; therefore, we accepted the deletion of the 29 items. The final three-factor solution included 16 items with 57.99% of total variance explained, which indicates near acceptable variance in social science research, with 60% being acceptable (Hair et al., 2010). Factor 1 (seven items) explained 38.98% of the total variance; Factor 2 (six items) explained 11.37% of the total variance; and Factor 3 (three items) explained 7.64% of the total variance.

Three-Factor Solution With Critical Items
     After finalizing the model, we added Item 8 (“I have thoughts of ending my life”) and Item 44 (“I feel angry enough at work/school to do something I might regret”) into the final model for purposes of clinical utility. Both items resulted in low factor loading (< .4). Item 8 correlated with other items on Factor 3, and Item 44 correlated with other items on Factor 1. This final 18-item three-factor solution reduced the variance explained by the items on the factors by 3.45%, indicating a questionable fit for social sciences (54.54%; Hair et al., 2010). Factor 1 (eight items) explained 36.83% of the total variance; Factor 2 (six items) explained 10.82% of the total variance; and Factor 3 (four items) explained 6.90% of the total variance. Internal consistency estimates are presented in Table 3 for all three models: (a) the original OQ-45.2 (α = .943); (b) the 16-item, three-factor solution (α = .894); and (c) the 18-item, three-factor solution (α = .896).

Test-Retest Reliability
     To examine the stability of the new 16-item OQ scores over time, we assessed test-retest reliability over a 30-day interval using bivariate correlation (Pallant, 2016). Results yielded strong correlation coefficients between pre-OQ scores and post-OQ scores: (a) OQ Total Scores, r = .781, p < .001; (b) Factor 1, r = .782, p < .001; (c) Factor 2, r = .742, p < .001; and (d) Factor 3, r = .681, p < .001. The 18-item OQ scores also demonstrated significant support for test-retest reliability over a 30-day interval: (a) OQ Total Scores, r = .721, p < .001; (b) Factor 1, r = .658, p < .001; (c) Factor 2, r = .712, p < .001; and (d) Factor 3, r = .682, p < .001.

Discussion

     We found that the current factor structure of the OQ-45.2 poorly fits the sample population of economically vulnerable individuals. Our preliminary results support Rice and colleagues’ (2014) claim: because of the unique stressors economically vulnerable individuals face, the OQ-45.2 does not adequately capture their psychological distress. The lack of support for the OQ-45.2’s current structure (i.e., three-factor oblique) creates doubt clinically when assessing clients’ distress. Therefore, we explored alternative structural models proposed by Bludworth and colleagues (2010) using a CFA, and subsequently an EFA, to reexamine the factor structure of the OQ-45.2.

The EFA resulted in a 16-item, three-factor solution with our sample, indicating marginal support for the validity and reliability of the items for this brief model of the OQ, meaning that this model lacked reliability (i.e., ability to produce similar results consistently) and validity (i.e., ability to actually measure what it intends to measure: distress). In social science research, total variance explained of 60% is adequate (Hair et al., 2010); therefore, the three-factor model that approaches 60% could be acceptable, indicating that this model captures more than half or more than chance of the construct distress for this population. Still, additional research is needed to support the factor structure with a similar population of low-income, diverse individuals. Economically vulnerable individuals experience unique stressors (Karney & Bradbury, 2005), and brief assessments are best practices (Beidas et al., 2015). Therefore, we encourage other researchers to reexamine the use of this brief version of the OQ with a sample of economically vulnerable individuals or develop a new instrument that may more accurately capture psychological distress in economically disadvantaged individuals.

Also, the 16-item model results differ from the original OQ-45.2 in that we were unable to find support for the social role factor with our sample population. We hypothesize this finding is largely due to the economic stressors this population faces (e.g., unreliable transportation, food scarcity, housing needs). Anecdotally, some participants commented during the initial intake session that several items (e.g., specifically items on the social role factor relating to employment) were not relevant to their situation because of under- or unemployment. Further, reducing OQ-45.2 to a 16-item assessment may provide a more user-friendly version requiring less time for respondents and more efficient use of clinical time; however, without further research, the current authors are hesitant to support its clinical use with this population of economically vulnerable individuals.

Similar to previous researchers (e.g., Kim et al., 2010; Rice et al., 2014), we also found evidence for the need for a substance use factor (e.g., Factor 3) in the 18-item abbreviated model; however, this model deviated from the original OQ-45.2. The findings of this study support the need for professional counselors to assess substance use as part of psychological distress, whether it be implementing the
18-item version of the OQ or adding an additional assessment that has greater reliability and validity of its items with this population.

Implications
     We found initial, possible support for a brief version of the OQ-45.2 for economically vulnerable individuals. The abbreviated 16-item OQ assessment derived from this research requires less time to complete while capturing an individual’s distress on substance use, interpersonal relationships, and symptom distress. A brief instrument can provide professional counselors with a snapshot of the client’s concerns, which can assist in monitoring a client’s level of psychological distress throughout treatment. In clinical settings, counselors can utilize this instrument to briefly assess at intake the baseline distress of their clients and use it as a guide or conversation starter for discussing client distress. For example, a counselor may ask that the client complete the brief OQ-16 instrument with the intake paperwork. In review of all paperwork, the counselor may note to the client, “I noticed that you indicated high distress with interpersonal relationships. Is that a place you would like to begin, or do you have another place you want to begin?”

Further, we retained two critical items (i.e., Items 8 and 44) in the 18-item version of the OQ brief assessment, as psychological distress associated with economic vulnerability is linked to higher rates of suicide and homicide (Knifton & Inglis, 2020). Because of the clinical utility of this instrument, professional counselors may want to include those items to assess a client’s level of threat of harm to self or others. Dependent on the client’s answer to these critical items, professional counselors have a quick reference with which to intervene or focus the initial session to address safety. Therefore, the items of this assessment may possibly be used to start the initial dialogue regarding an individual’s psychological distress and/or suicidal intent; however, the assessment should not be used as the only tool or instrument to diagnose or treat psychological distress. We understand that these items can help professional counselors efficiently assess for suicidal or homicidal intent. Therefore, the counselor can opt to use the 16-item version and include an additional, more reliable assessment for measuring threat of harm to self and/or others. For example, counselors may opt to use an instrument such as the Ask Suicide-Screening Questions tool (Horowitz et al., 2012) to further evaluate suicidal intent.

In our experience, when following up with study participants based on a score higher than 1 on a scale of 1–5, many participants indicated they felt that way in the past but no longer feel that way now. In our use of the OQ-45.2, we find that participants tend to answer these questions based on their entire life versus the time frame indicated in the assessment instructions (the past week [7 days]). Therefore, professional counselors should be clear that respondents should answer based on the past week, rather than “ever experienced.” When offering the assessment to clients, we recommend that the counselor highlight the time frame in the instructions or clearly communicate that time frame to the client before they complete the instrument to gain the most accurate data.

Limitations and Suggestions for Future Research
     As with all research, results should be considered in light of limitations. The large study sample consisted of diverse individuals; however, the majority were women, and all individuals were from the southeast region of the United States, minimizing the generalizability of these findings. In addition, although findings indicate initial, possible support for a revised three-factor model consisting of 16 items, future studies are warranted to strengthen the validity of this abbreviated version of the OQ-45.2. We suggest that future researchers test the 16-item assessment through CFA with a similar population to confirm the current study’s findings. All respondents volunteered to participate in a 6-month study, which may indicate more motivation to improve or represent a population with distress responses different from those who were recruited but chose not to participate in the study. Additionally, study participants were actively recruited, and may have experienced less distress than a help-seeking sample.

The OQ is available in a Spanish translation; however, we only included people who completed the English OQ-45.2 version in the current study. Future analyses should examine the factor structure of the Spanish OQ-45.2 as well. Next, future research on the OQ should include the development and testing of new items. Lastly, future research should aim to validate the reduced 16-item and 18-item OQ scores on a new sample and seek to establish a new criterion for clinical significance. Professional counselors may also benefit from the creation of a specific instrument assessing distress related to the unique stressors that economically vulnerable clients face. Until further analyses are conducted with a new sample population to confirm the abbreviated models, we encourage professional counselors to implement the brief version tentatively and with caution, and to follow up with the client regarding high scores on critical items prior to making clinical judgments regarding reported subscale scores.

Conclusion

     Given the broad utility of the OQ-45.2 in research and mental health settings, researchers and professional counselors must understand the instrument’s structure for interpretation purposes and how the assessment should be adapted for various populations. Professional counselors can effectively support clients by assessing and recognizing how economic-related distress impacts their quality of life, which may directly relate to treatment outcomes. Findings from the current study add to previous literature that calls into question the original OQ-45.2 factor structure. Additionally, the current study’s findings support a revised 16-item, three-factor structure for economically vulnerable clients and we provide implications for use of this assessment in clinical practice. Future research should include a confirmatory analysis of the current findings.

Conflict of Interest and Funding Disclosure
This research was supported by a grant (90FM0078)
from the U.S. Department of Health and Human
Services (USDHHS), Administration for Children and
Families, Office of Family Assistance. Any opinions,
findings, conclusions, or recommendations are those
of the authors and do not necessarily reflect the views
of the USDHHS, Office of Family Assistance. The authors
reported no further funding or conflict of interest.

 

References

Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4(6), 561–571. https://doi.org/10.1001/archpsyc.1961.01710120031004

Beidas, R. S., Stewart, R. E., Walsh, L., Lucas, S., Downey, M. M., Jackson, K., Fernandez, T., & Mandell, D. S. (2015). Free, brief, and validated: Standardized instruments for low-resource mental health settings. Cognitive and Behavioral Practice, 22(1), 5–19. https://doi.org/10.1016/j.cbpra.2014.02.002

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.
https://doi.org/10.1037/0033-2909.107.2.238

Bludworth, J. L., Tracey, T. J. G., & Glidden-Tracey, C. (2010). The bilevel structure of the Outcome Questionnaire–45. Psychological Assessment, 22(2), 350–355. https://doi.org/10.1037/a0019187

Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen and J. S. Long (Eds.), Testing structural equation models. SAGE.

Byrne, B. M. (2016). Structural equation modeling with AMOS: Basic concepts, applications, and programming (3rd ed.). Routledge.

Carlson, R. G., Fripp, J., Munyon, M. D., Daire, A., Johnson, J. M., & DeLorenzi, L. (2014). Examining passive and active recruitment methods for low-income couples in relationship education. Marriage & Family Review, 50(1), 76–91. https://doi.org/10.1080/01494929.2013.851055

Carlson, R. G., Rappleyea, D. L., Daire, A. P., Harris, S. M., & Liu, X. (2017). The effectiveness of couple and individual relationship education: Distress as a moderator. Family Process, 56(1), 91–104.
https://doi.org/10.1111/famp.12172

Clark, M., Ausloos, C., Delaney, C., Waters, L., Salpietro, L., & Tippett, H. (2020). Best practices for counseling clients experiencing poverty: A grounded theory. Journal of Counseling & Development, 98(3), 283–294. https://doi.org/10.1002/jcad.12323

Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7), 1–9.
https://doi.org/10.7275/jyj1-4868

Dakin, J., & Wampler, R. (2008). Money doesn’t buy happiness, but it helps: Marital satisfaction, psychological distress, and demographic differences between low- and middle-income clinic couples. The American Journal of Family Therapy, 36(4), 300–311. https://doi.org/10.1080/01926180701647512

Eid, M., Geiser, C., Koch, T., & Heene, M. (2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22(3), 541–562. https://doi.org/10.1037/met0000083

Enders, C. K. (2010). Applied missing data analysis (1st ed.). Guilford.

Espiridion, E. D., Oladunjoye, A. O., Millsaps, U., & Yee, M. R. (2021). A retrospective review of the clinical significance of the Outcome Questionnaire (OQ) measure in patients at a psychiatric adult partial hospital program. Cureus, 13(3), e13830. https://doi.org/10.7759/cureus.13830

Foss-Kelly, L. L., Generali, M. M., & Kress, V. E. (2017). Counseling strategies for empowering people living in poverty: The I-CARE Model. Journal of Multicultural Counseling and Development, 45(3), 201–213.
https://doi.org/10.1002/jmcd.12074

Hair, J. F., Jr., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate data analysis (6th ed.). Pearson.

Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35(5), 485–491. https://doi.org/10.1037/0735-7028.35.5.485

Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416. https://doi.org/10.1177/0013164405282485

Hodgkinson, S., Godoy, L., Beers, L. S., & Lewin, A. (2017). Improving mental health access for low-income children and families in the primary care setting. Pediatrics, 139(1). https://doi.org/10.1542/peds.2015-1175

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447

Horowitz, L. M., Bridge, J. A., Teach, S. J., Ballard, E., Klima, J., Rosenstein, D. L., Wharff, E. A., Ginnis, K., Cannon, E., Joshi, P., & Pao, M. (2012). Ask Suicide-Screening Questions (ASQ): A brief instrument for the pediatric emergency department. Archives of Pediatrics & Adolescent Medicine, 166(12), 1170–1176.
https://doi.org/10.1001/archpediatrics.2012.1276

Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). SAGE.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.
https://doi.org/10.1080/10705519909540118

Karney, B. R., & Bradbury, T. N. (2005). Contextual influences on marriage: Implications for policy and intervention. Current Directions in Psychological Science, 14(4), 171–174.
https://doi.org/10.1111/j.0963-7214.2005.00358.x

Kim, S.-H., Beretvas, S. N., & Sherry, A. R. (2010). A validation of the factor structure of OQ-45 scores using factor mixture modeling. Measurement and Evaluation in Counseling and Development, 42(4), 275–295.
https://doi.org/10.1177/0748175609354616

Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford.

Knifton, L., & Inglis, G. (2020). Poverty and mental health: Policy, practice and research implications. BJPsych Bulletin, 44(5), 193–196. https://doi.org/10.1192/bjb.2020.78

Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x

Lam, J. R., Tyler, J., Scurrah, K. J., Reavley, N. J., & Dite, G. S. (2019). The association between socioeconomic status and psychological distress: A within and between twin study. Twin Research and Human Genetics, 22(5), 312–320. https://doi.org/10.1017/thg.2019.91

Lambert, M. J. (2017). Measuring clinical progress with the OQ-45 in a private practice setting. In S. Walfish, J. E. Barnett, & J. Zimmerman (Eds.), Handbook of private practice: Keys to success for mental health practitioners (pp. 78–93). Oxford University Press. https://doi.org/10.1093/med:psych/9780190272166.003.0007

Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Vermeersch, D. A., Clouse, G. C., & Yanchar, S. C. (1996). The reliability and validity of the Outcome Questionnaire. Clinical Psychology and Psychotherapy, 3(4), 249–258. https://doi.org/10.1002/(SICI)1099-0879(199612)3:4<249::AID-CPP106>3.0.CO;2-S

Lambert, M. J., Gregersen, A. T., & Burlingame, G. M. (2004). The Outcome Questionnaire-45. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults (3rd ed.; pp. 191–234). Routledge.

Lambert, M. J., Smart, D. W., Campbell, M. P., Hawkins, E. J., Harmon, C., & Slade, K. L. (2006). Psychotherapy outcome, as measured by the OQ-45, in African American, Asian/Pacific Islander, Latino/a, and Native American clients compared with matched Caucasian clients. Journal of College Student Psychotherapy, 20(4), 17–29. https://doi.org/10.1300/J035v20n04_03

Levy, H. C., Worden, B. L., Davies, C. D., Stevens, K., Katz, B. W., Mammo, L., Diefenbach, G. J., & Tolin, D. F. (2020). The dose-response curve in cognitive-behavioral therapy for anxiety disorders. Cognitive Behaviour Therapy, 49(6), 439–454. https://doi.org/10.1080/16506073.2020.1771413

Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.2307/2290157

Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., & Herzberg, P. Y. (2008). Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Medical Care, 46(3), 266–274. https://doi.org/10.1097/MLR.0b013e318160d093

Mueller, R. M., Lambert, M. J., & Burlingame, G. M. (1998). Construct validity of the Outcome Questionnaire: A confirmatory factor analysis. Journal of Personality Assessment, 70(2), 248–262.
https://doi.org/10.1207/s15327752jpa7002_5

Muthén, L. K., & Muthén, B. O. (2015). Mplus: Statistical analysis with latent variables. User’s guide. (7th ed.)
https://www.statmodel.com/download/usersguide/MplusUserGuideVer_7.pdf

Mvududu, N. H., & Sink, C. A. (2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4(2), 75–98. https://doi.org/10.1177/2150137813494766

Näher, A.-F., Rummel-Kluge, C., & Hegerl, U. (2020). Associations of suicide rates with socioeconomic status and social isolation: Findings from longitudinal register and census data. Frontiers in Psychiatry, 10(898), 1–9. https://doi.org/10.3389/fpsyt.2019.00898

Osborne, J. W. (2013). Best practices in data cleaning. SAGE.

Pearson, M., Stanley, S. M., & Rhoades, G. K. (2015). Within My Reach leader manual. PREP for Individuals, Inc.

Rice, K. G., Suh, H., & Ege, E. (2014). Further evaluation of the Outcome Questionnaire–45.2. Measurement and Evaluation in Counseling and Development, 47(2), 102–117. https://doi.org/10.1177/0748175614522268

Santiago, C. D., Wadsworth, M. E., & Stump, J. (2011). Socioeconomic status, neighborhood disadvantage, and poverty-related stress: Prospective effects on psychological syndromes among diverse low-income families. Journal of Economic Psychology, 32(2), 218–230. https://doi.org/10.1016/j.joep.2009.10.008

Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). SAGE.

Semega, J., Kollar, M., Shrider, E. A., & Creamer, J. F. (2021). Income and poverty in the United States: 2019. Current population reports. U.S. Census Bureau.
https://www.census.gov/content/dam/Census/library/publications/2020/demo/p60-270.pdf

Stanford Center on Poverty and Inequality. (2015). State of the states: The poverty and inequality report. Pathways. http://inequality.stanford.edu/sites/default/files/SOTU_2015.pdf

Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson.

Tabet, S. M., Lambie, G. W., Jahani, S., & Rasoolimanesh, S. (2019). The factor structure of the Outcome Questionnaire–45.2 scores using confirmatory tetrad analysis—partial least squares. Journal of Psychoeducational Assessment. https://doi.org/10.1177/0734282919842035

Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.

 

Dalena Dillman Taylor, PhD, LMHC, RPT-S, is an associate professor at the University of North Texas. Saundra M. Tabet, PhD, NCC, CCMHC, ACS, LMHC, is an assistant professor and CMHC Program Director at the University of San Diego. Megan A. Whitbeck, PhD, NCC, is an assistant professor at The University of Scranton. Ryan G. Carlson, PhD, is a professor at the University of South Carolina. Sejal Barden, PhD, is a professor at the University of Central Florida. Nicole Silverio is an assistant professor at the University of South Carolina. Correspondence may be addressed to Dalena Dillman Taylor, 1300 W. Highland St., Denton, TX 76201, Dalena.dillmantaylor@unt.edu.

The State of Integrated Primary and Behavioral Health Care Research in Counselor Education: A Review of Counseling Journals

Alexander M. Fields, Cara M. Thompson, Kara M. Schneider, Lucas M. Perez, Kaitlyn Reaves, Kathryn Linich, Dodie Limberg

The integration of behavioral health care within primary care settings, otherwise known as integrated care, has emerged as a treatment modality for counselors to reach a wide range of clients. However, previous counseling scholars have noted the lack of integrated care representation in counseling journals. In this scoping review, we identified 27 articles within counseling journals that provide integrated care implications. These articles appeared in 10 unique counseling journals, and the publication years ranged from 2004–2023. Articles were classified as: (a) conceptual, (b) empirical, or (c) meta-analyses and systematic reviews. The data extracted from the articles focused on the implications for integrated care training and practice for the next generation of counselors, evidence-based treatment approaches, and future research directions.

Keywords: integrated care, counseling journals, scoping review, implications, research

 

One in five U.S. adults are living with a mental illness or substance use disorder (e.g., major depressive disorder, generalized anxiety disorder, alcohol use disorder, nicotine use disorder) and individuals with a mental illness or substance use disorder are more likely to have a chronic health condition (Substance Abuse and Mental Health Services Administration [SAMHSA], 2021). Integrated primary and behavioral health, also termed integrated care (IC), has emerged as a noted treatment strategy to meet the holistic needs of individuals with comorbid mental and physical health symptoms. Although IC has been operationalized inconsistently by scholars, most definitions describe the integration and coordination of behavioral health services within primary care settings (Giese & Waugh, 2017). The SAMHSA-HRSA (Health Resources and Services Administration) Center for Integrated Health Solutions expanded upon this definition to outline IC on a continuum of health care service delivery (Heath et al., 2013). Heath and colleagues described the progressive movement toward IC as (a) collaborative care: providers from multiple health care professions collaborating on holistic health care treatment planning at a distance;
(b) co-located care: providers from multiple health care professions sharing basic system integration, such as sharing physical proximity and more frequent collaboration; and (c) IC: providers from multiple health care professions having systematic integration (i.e., sharing electronic medical records and office space) and a high level of collaboration resulting in a unified treatment approach. Thus, health care consumers are able to receive care for their behavioral and physical health at the same time and location when an IC approach is applied, which may reduce barriers (e.g., transportation, child care, time off work) and increase access to behavioral health care (Vogel et al., 2014).

Beyond support from SAMHSA and HRSA, the IC movement has been endorsed through government legislation. The Patient Protection and Affordable Care Act (2010) paved the way for agencies and health care systems demonstrating an IC approach to receive additional funding for health care providers, as well as increased reimbursements for the services they deliver. Furthermore, the federal government has recently pledged to double the funding support for IC to be more accessible in hospitals, substance abuse treatment facilities, family care practices, school systems, and other health care settings (The White House, 2022). This may be the result of IC showing efficacy in reducing mental health symptoms (Lenz et al., 2018), saving health care expenditures (Basu et al., 2017), and promoting overall life satisfaction (Gerrity, 2016). Compared to traditional (i.e., siloed) care, IC involves simultaneous treatment from physical and mental health providers, thus providing additional access to mental health screenings and services. For example, McCall et al. (2022) concluded that a mental health counselor in an IC setting may support treatment engagement and reduce health care costs for an individual with a substance use disorder when utilizing the screening, brief intervention, and referral to treatment (SBIRT) model. However, the IC paradigm is not a novel concept; Aitken and Curtis (2004) introduced IC to counseling journals by providing emerging evidence of IC support and advocating for health care settings to recognize counselors as an asset to IC teams and for counselors to be trained in IC.

Brubaker and La Guardia (2020) noted that the Council for Accreditation of Counseling and Related Educational Programs (Section 5, Standard C.3.d; CACREP; 2015) required IC education in counselor-in-training (CIT) development. Additionally, the 2024 CACREP Task Force has also included these standards for its proposed revisions (CACREP, 2022). HRSA has funded counselor education programs to train CITs during practicum and internship experiences, funding over 4,000 new school, addiction, or mental health counselors during 2014–2022 through the Behavioral Health Workforce and Education Training (BHWET) Program (HRSA, 2022). Although IC training, education, and practice is occurring within counselor education, IC literature remains scarce in counseling journals (Fields et al., 2023). The lack of representation presents an issue for appropriate training for CITs and future research directions, which leads to sustainability concerns. Specifically, Fields et al. (2023) reported that a lack of IC literature in counseling journals creates a weak foundation to advocate for counselors to be included in the IC movement. With the understanding that nearly half of U.S. adults with poor mental health receive their mental health care in a primary care setting (Petterson et al., 2014), counselors may increase their access to additional clients when they are invited to IC settings. Furthermore, it weakens counselors’ professional identity if counselors are not trained in a standardized approach. As such, this scoping review aims to amalgamate current IC literature within counseling journals and provide CITs, counselors, and counselor educators from diverse backgrounds with a resource to inform their education, practice, and scholarship. The guiding research question for this review is: What are the publication trends (i.e., publication years and journals), study characteristics and outcomes, implications, and recommendations for future research from IC literature within counseling journals?

Method

We conducted a scoping review to identify the publication trends, key characteristics of IC studies (i.e., type of article and study outcomes), and implications for future research of IC literature published in counseling journals (Munn et al., 2018). Our methodology followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews; Tricco et al., 2018) checklist to 1) establish eligibility, 2) identify sources of information, 3) conduct a screening process to select included articles, 4) identify and chart data items, 5) conduct a critical appraisal of included articles, and 6) synthesize results. We searched the following databases for eligible literature: (a) Alt HealthWatch, (b) APA PsycArticles, (c) APA PsycInfo, (d) Education Source, (e) EBSCOHost, (f) Health Source: Consumer Edition, (g) Health Source: Nursing/Academic Edition, (h) MEDLINE with Full Text, (i) Science Reference Center, (j) Social Sciences Full Text (H.W. Wilson), and (k) Social Work Abstracts. We used the search terms: “Integrat* care” OR “integrat* primary and behavioral healthcare” OR “integrat* primary and behavioral care” AND “counsel* education” OR “counsel*.” Additional criteria for this search were full-text, peer-reviewed journal articles, and an English translation.

Eligibility Criteria
     Eligibility criteria for articles included in this review are publication in a counseling journal, presentation of implications (i.e., recommendations for training and evidence-based counseling models or approaches) of IC practice for CITs and counselors through research methodology or conceptual themes, and discussion of future research on IC for counselor educators and counseling scholars through research methodology or conceptual themes. Eligible counseling journals included those published by divisions of the American Counseling Association (ACA), the American Mental Health Counselors Association (AMHCA), the American School Counselor Association (ASCA), the National Board for Certified Counselors (NBCC), and Chi Sigma Iota. Journals connected to international and regional divisions were also included. The initial database search resulted in 222 articles, which we reduced to 125 articles after removing duplicates. Another two articles were identified through additional sources. These additional sources included references identified through a review of an article and a social media post advertising an IC article. We reviewed titles and abstracts for inclusion criteria. This resulted in 28 articles that were fully reviewed. Research team members independently examined articles to summarize information relevant to the research question. During this process, articles were excluded if they did not provide future implications for IC in counseling or counselor education. Following this process, 27 articles were included. A visual representation of the eligibility and inclusion process can be found in Figure 1.

Data Extraction
     After consensus was reached on the final 27 articles, our research team assessed the available evidence and synthesized the results. The seven-member research team comprised four doctoral students in counselor education, an undergraduate student minoring in counselor education, a clinical assistant professor in a counselor education program, and an associate professor in a counselor education program. The initial data extraction process began with identifying journal representation and organizing articles based on similar characteristics. This resulted in classifying articles as either (a) conceptual, (b) empirical, or (c) meta-analyses and systematic reviews. Conceptual articles provided an overview of available literature and identified a current gap in IC understanding for counseling or counselor education. Articles classified as conceptual did not present original data or follow research methodology. Moreover, the conceptual models typically advocated for increased counseling representation in IC settings to reach traditionally underserved groups (e.g., LGBTQ+ clients, individuals from rural communities) or a replicable model of training grounded in empirical support to prepare CITs to work in IC settings. Data from these articles were presented in accordance with the authors’ population(s) of interest, the identified research gap, implications gathered from existing literature, and recommendations for future research. Empirical articles introduced a novel research question and presented results to address that question. Data from these articles were presented in accordance with the authors’ study classification (i.e., qualitative, quantitative, or mixed methods), research methodology, the number and profile of participants, research of interest, and results from their analyses. Lastly, meta-analyses and systematic reviews organized previous empirical studies and presented big picture results across multiple studies. Data from these articles were presented in accordance with the authors’ article classification (i.e., meta-analysis or systematic review), population of interest, number of included studies and number of total participants (if applicable), results, and implications for future research. Because of the broad scope and exploratory nature of this review, a quality assessment was not performed.

Figure 1
Integrated Care Literature in Counseling and Counselor Education Flow Chart

Note. This flow chart outlines the PRISMA-ScR (Tricco et al., 2018) search process.

 

Results

This scoping review resulted in a wide variety of articles in counseling journals that may inform the future of IC research in counseling and counselor education. Additionally, articles included in our review have ranging implications at the CIT, counselor, and client levels. The results section will begin with an overview of IC publication trends within counseling journals, detailing the publication range and specific journals. Next, results for this review were organized based on study outcomes and the classification of the article. The study outcomes sections will further detail included articles that are conceptual, empirical, or meta-analyses and systematic reviews.

Publication Trends
     Articles included in this review range in publication year from 2004–2023. Articles are represented in 10 unique journals. Specifically, the following journals are represented in this review: (a) Counseling Outcome Research and Evaluation (n = 2); (b) International Journal for the Advancement of Counselling (n = 2); (c) Journal of Addictions & Offender Counseling (n = 2); (d) Journal of College Counseling (n = 1); (e) Journal of Counseling & Development (n = 7); (f) Journal of Creativity in Mental Health (n = 1); (g) Journal of LGBTQ Issues in Counseling (n = 1); (h) Journal of Mental Health Counseling (n = 9); (i) The Family Journal (n = 1); and (j) The Professional Counselor (n = 1).

Study Outcomes
Conceptual Articles
     Our review included 11 conceptual articles (see Appendix A). Of these studies, five described IC as a treatment approach for underserved populations. In each of these articles, the authors described how IC provided a “one-stop-shop” treatment approach that provided increased access to a mental health provider in a traditional primary care setting, which reduced barriers to transportation, cost per service, and provider shortages. Six studies focused on current licensed counselors in primary care settings, counselor educators, CITs in a CACREP-accredited program, and counselors interested in IC. Common implications of these articles included advocacy, education, communication, networking, and teamwork.

Eight studies described how additional research could empirically investigate their IC model. The authors of these conceptual articles recommended continued investigation of the current medical model and national recognition of gaps of care for both the chronic pain and substance abuse population; integrating the interprofessional education collaborative (IPEC) into the curriculum of mental health counselors; interprofessional telehealth collaboration (IPTC) through cognitive behavioral therapy (CBT) for rural communities; treatments aligned with cultural tailoring; implementation of IC for those in the LGBTQ+ community; trauma-informed IC; and the role of counselors in an IC team treating obesity. The conceptual models reported in Table 1 highlight evidence-based approaches a counselor can apply in IC settings to assess for substance abuse and mental health disorders, brief interventions (e.g., CBT technique of challenging automatic thoughts, motivational interviewing) to encourage engagement in preventative health care, and trauma-informed practices (e.g., psychoeducation on trauma somatization). Moreover, counselors trained in the Multicultural and Social Justice Counseling Competencies (MSJCC; Ratts et al., 2016) can advocate for culturally tailored interventions to respect a client’s cultural identity.

Two studies highlighted different approaches to IC. Johnson and Mahan (2020) identified the IPTC model, which allows health professionals to use technology to increase access to services for rural communities. The IPTC model provides telehealth services to rural communities through an IC model to reduce negative social determinants of health, such as distance from a mental health provider. Specifically, Johnson and Mahan (2020) detailed their approach to working alongside primary care providers to deliver family counseling services and coordinate health care services to promote overall health and wellness for family systems. Goals of their family counseling sessions included increasing health literacy, enhancing a family’s coping strategies for medical conditions, and reducing family conflicts. The Chronic Care Model has been shown to improve the quality of care for clients with chronic medical conditions by increasing communication between health care professionals (Sheesley, 2016). Two articles also focused on the impact of two identified training programs. Johnson and Freeman (2014) identified the IPEC Expert Panel and their efforts to effectively train health professionals to collaborate. Lloyd-Hazlett et al. (2020) focused on the Program for the Integrated Training of Counselors in Behavioral Health (PITCH), which is a training program for master’s-level counseling students in a CACREP-accredited program aimed at training students to supply IC to rural, vulnerable, and underserved communities. These results are represented in Appendix A.

Empirical Articles
     Our review resulted in 13 empirical studies using the following designs: three mixed-methods designs, three quasi-experimental designs, two cross-sectional surveys, two pre-post designs, three phenomenological studies, and one exploratory cross-case synthesis. The studies were completed in a variety of settings, such as university clinics, trauma centers, and hospitals. Participant profiles varied across studies, with nine representing CITs or practicing counselors, three representing clients, and one representing both. In addition to counselors, studies with client-level data included service providers and undergraduate students from social work, speech–language pathology, dental hygiene, nursing, and physical therapy programs. Articles that reported client-level data tested an intervention (e.g., motivational interviewing in an IC setting for a substance use disorder), compared an IC approach to treatment as usual (TAU) in silos, or explored relationships between health care indicators and client engagement in a setting applying an IC modality. Furthermore, three studies in this article used Heath et al.’s (2013) conceptualization of IC, which was the most common model cited.

Most study outcomes were reported as positive benefits for IC. For CIT and counselor-level studies, six described a theme of increased ability and desirability to work with a collaborative approach on IC teams. Participants also commonly reported an increase in professional identity and self-efficacy. Participants in studies by Agaskar et al. (2021), Alvarez et al. (2014), and Lenz and Watson (2023) further demonstrated that working with underserved populations in IC settings increased their multicultural competence, specifically around areas of acceptance, advocacy, and awareness. A gap in IC awareness among service providers and organizational constraints were noted as potential barriers to IC care. Johnson et al. (2021) found interprofessional supervision as a potential barrier to remaining within a provider’s scope of practice, because a supervisor providing supervision to a supervisee from a different professional identity may not appropriately understand roles and responsibilities. Because of this, Johnson and colleagues noted implications for future research and graduate-level training in the classroom and field experience. All four of the studies completed with client-level data were quantitative, accounting for 2,378 client participants. Results of these studies suggested improvement in holistic client functioning (i.e., reduction in pathological symptoms and increase in preventative behaviors; Ulupinar et al., 2021), a decrease in crisis events (Schmit et al., 2018), and decrease in risky drinking behaviors for individuals receiving IC trauma care (Veach et al., 2018). The self-stigma of mental illness and of seeking help had an inverse relationship with mental health literacy among patients who received treatment in an IC setting (Crowe et al., 2017). These results are represented in Appendix B.

Meta-Analyses and Systematic Reviews
     Three articles in this review were meta-analyses or systematic reviews. Specifically, two articles were meta-analyses and one was a systematic review. Participants within these studies included adults with substance use disorders, mental health professionals receiving training to practice within IC, and individuals receiving mental health care in traditional primary care settings. All three articles described benefits of IC. Additionally, the authors differed on the number of studies and participants included in their analyses. Fields et al. (2023) completed a review of 18 articles that studied training interventions for mental health professionals to work on IC teams and concluded that training in IC promotes aspects of interprofessional collaboration, professional identity development, and self-efficacy. Balkin et al. (2019) concluded no statistical significance between IC treatment and TAU to decrease frequency of substance use. Balkin et al. also remarked that their study, including 1,545 participants, did not reach statistical power and results should be considered preliminary. Lenz et al. (2018) reported a decrease in mental health symptoms with a greater effect when a larger treatment team and number of behavioral health sessions are increased, compared to TAU. Lenz and colleagues generated their results from 14,764 participants. Lastly, Fields et al. (2023) and Lenz et al. (2018) both used Heath et al.’s (2013) model of IC for conceptualization. For all three of these studies, additional research is needed to understand IC at the client or consumer level, as well as how different variables affect the treatment process. These results are represented in Appendix C.

Discussion 

Implications for Counseling Practice
     The results of this scoping review have implications that may inform clinical practice for counselors and CITs. Most results suggested clinical benefits for individuals receiving counseling services through an IC setting. Clients or consumers that received IC treatment reported a reduction of mental health symptoms (Lenz et al., 2018; Ulupinar et al., 2021), mental health stigma (Crowe et al., 2018), and crisis events (Schmit et al., 2018). As almost half of individuals with poor mental health receive treatment in primary care settings (Petterson et al., 2014), integrating a counselor into a traditional primary care setting (e.g., hospital, community health care clinic) provides an additional treatment team member with specialized training to treat mental health concerns. Because of the potentially fast nature of IC settings, interested counselors are encouraged to review SAMHSA applications of SBIRT to facilitate brief meetings until more long-term services are provided. Furthermore, counselors may consider reviewing resources on evidence-based approaches, such as Ultra-Brief Cognitive Behavioral Interventions: A New Practice Model for Mental Health and Integrated Care (Sperry and Binensztok, 2019), and understanding common medical terminology, such as A Therapist’s Guide to Understanding Common Medical Conditions (Kolbasovsky, 2008).

Articles that were classified as conceptual also suggested that IC treatment has the potential to enhance service delivery for clients from diverse populations, such as LGBTQ+ and medically underserved communities (Kohn-Wood & Hooper, 2014; Moe et al., 2018). The primary rationale described by scholars is that an IC approach advocates for diverse populations to reduce social determinants of health, such as proximity barriers, communications barriers, and availability of culturally appropriate interventions. Counselors interested in working in an IC setting are strongly encouraged to review the MSJCC (Ratts et al., 2016) and be prepared to serve as an advocate for their client as they navigate the health care system. The Hays (1996) ADDRESSING model also provides counselors a conceptualization model for understanding power and privileges associated with cultural differences. Information drawn from an understanding of power and privileges may further assist the interdisciplinary team with delivering culturally appropriate care. However, Balkin et al. (2019) concluded that IC may not result in a decrease in frequency of substance misuse. As IC may not be the most ideal approach depending on the client’s presenting concern and therapeutic goals, counselors are ethically bound to continue ongoing assessments to collaborate with their client to determine the most appropriate treatment setting.

Implications for Counselor Education
     In addition to counseling practice, the results of our scoping review provide implications for counselor education and ongoing counselor development. First, counselors or CITs that have received training in IC have commonly reported an increase in their professional identity development, as practicing in IC settings creates an opportunity for counselors and CITs to differentiate counseling responsibilities from related health care professionals (Brubaker & La Guardia, 2020; Johnson et al., 2015). Counselor educators and supervisors are encouraged to consider how they can create opportunities to challenge their students or supervisees to understand their role in the health care landscape. For example, Johnson and Freeman (2014) described an interdisciplinary health care delivery course to train counselors alongside students from other disciplines (e.g., nursing, physical therapy), and counselor educators may consider how they can form partnerships across departments to provide these opportunities. Counselor or CIT participants also expressed an enhanced self-efficacy for clinical practice (Brubaker & La Guardia, 2020; Lenz & Watson, 2023). As trainings and field experience for IC practice typically involve experiential components, counselors and CITs are provided additional opportunities to practice their previous clinical trainings in IC settings. Farrell et al. (2009) provided an example of how counselor educators can use standardized patients (i.e., paid actors simulating a presenting concern) to role-play a client in a primary setting. In such situations, the CIT can practice a variety of brief assessments (e.g., substance abuse, suicide, depression screenings) and interventions (e.g., motivational interviewing techniques, such as building ambivalence) in an IC setting.

With the counseling profession’s emphasis on aspects of valuing cultural differences and social justice, counselor educators and supervisors may consider how they can train counselors and CITs to reduce social determinants of health through integrated and collaborative practices that promote affirmative and proximal care. Counselors or CITs that received training to work in IC settings often reported higher understanding of multicultural counseling (Agaskar et al., 2021; Lenz et al., 2018). Thus, counselor educators and supervisors can provide their counselors and CITs with challenges to incorporate aspects of the MSJCC (Ratts et al., 2016) when delivering interdisciplinary care. All trainings in our review were administered across multiple modalities (e.g., face-to-face, hybrid, virtual, asynchronous), which gives counselor educators flexibility in how they train counselors or CITs. The variety in training administration is a promising result, as the COVID-19 pandemic highlighted the need for flexible training options for counselors and CITs. In addition, counselors and CITs in rural communities often have infrequent access to training as compared to their non-rural colleagues, and thus flexibility may enhance the accessibility of IC training (Alvarez et al., 2014). Lastly, counselors and CITs being trained in IC modalities do not need to work in IC settings to use interprofessional skills developed through trainings. Heath et al. (2013) remarked that IC is not always a feasible option, but helping professionals can still apply collaborative approaches to enhance their client’s holistic outcomes. In other words, counselors or CITs may apply IC principles of preventative health care and interdisciplinary treatment plans by collaborating with other health care professionals at a distance. Glueck (2015) corroborated this notion and described a theme that counselors who have previously worked in IC settings believe they are able to provide more holistic care because they are better equipped to collaborate with health care professionals from multiple disciplines. However, these counselors also reported that they would have been more prepared to work in IC if they received training at some point in their career.

Limitations and Recommendations for Future Research
     The methodology of a scoping review has noted limitations. Because of the nature of a scoping review, the data extraction process and results section are broad (Munn et al., 2018). Articles were not systematically evaluated to assess study quality, and the reader is encouraged to review a specific study before interpreting the results. In addition to study quality, scoping reviews include articles from a variety of article classifications, so the results and implications should be considered exploratory. Thus, we caution how readers draw conclusions from results presented in the included articles. Second, the search terms and inclusion criteria may have resulted in limitations. This search focused on IC; therefore, concepts such as interprofessional collaboration and interprofessional education may have been excluded. These concepts are discussed in the Heath et al. (2013) model, but they do not directly result in IC practice. Counseling and counselor education were also search terms, which may have excluded articles written by counseling scholars in journals outside of counseling and counselor education journals. Third, this review resulted in four studies that empirically investigated IC at the client level. With limited data at the client level, there are funding and advocacy sustainability concerns for IC within counseling and counselor education. Lastly, nine studies specifically provided implications for marginalized populations and multicultural competency development through an IC lens. Although Kohn-Wood and Hooper (2014) and Vogel et al. (2014) concluded that IC is a modality that advocates for the treatment of marginalized populations that have traditionally received services at unequal rates to their White, cisgender counterparts, this topic has limited representation in counseling IC literature. As discussed by Fields et al. (2023), this review demonstrates the need for understanding how the counseling professional identity rooted in social justice and advocacy may contribute to the advancement of IC services.

In light of our limitations, this review resulted in recommendations for future research directions. Conceptual articles included in this review synthesized literature on the importance of CITs and counselors understanding applications of IC, as well as potential treatment approaches to treat a variety of marginalized communities and clinical practices. Our research team recommends that counseling scholars reviewing the included conceptual articles consider how they can use the implications and future research directions to inform future research studies. These articles can also serve as support for counseling scholars who are applying for internal and external funding. Furthermore, the empirical studies, systematic reviews, and meta-analyses included in our review present data that can inform future research. For example, Balkin et al. (2019) and Veach et al. (2018) concluded contrasting results about IC in reducing substance abuse behaviors. Future research studies can continue researching substance misuse within IC settings to better understand evidence-based approaches to treat these populations. Twenty-one articles included recommendations for continued research at the client or consumer level, specifically for clients from marginalized communities. Counseling scholars are encouraged to stay up to date with program evaluation scholarship and implement a variety of methodical procedures to document the impact of IC on clients. Lastly, counseling scholars must advocate for continued IC literature publication within counseling and counselor education journals.

Conclusion

Our scoping review identified IC literature within counseling journals. Specifically, this review followed PRISMA-ScR protocols (Tricco et al., 2018) and identified 27 articles across 10 unique counseling journals. Most articles were within national flagship journals (such as those of ACA and AMHCA) and publication years ranged from 2004–2023. The articles in this review were organized according to their classification, and were described as either conceptual, empirical, or meta-analyses and systematic reviews. Implications for CITs, counselors, and clients were represented across each classification. Overall, IC implications from each article were positive for training and practice perceptions for CITs and counselors, as well as clinical outcomes for clients. Moving forward, authors unanimously encouraged counselor educators and counseling scholars to continue studying IC. Future scholarship would benefit from a deeper understanding of client-level implications, with an emphasis on how IC can benefit marginalized communities.

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

 

References

Note. Studies with an asterisk (*) are included in the scoping review.

*Agaskar, V. R., Lin, Y.-W. D., & Wambu, G. W. (2021). Outcomes of “integrated behavioral health” training: A pilot study. International Journal for the Advancement of Counselling, 43, 386–405.
https://doi.org/10.1007/s10447-021-09435-z

*Aitken, J. B., & Curtis, R. (2004). Integrated health care: Improving client care while providing opportunities for mental health counselors. Journal of Mental Health Counseling, 26(4), 321–331.
https://doi.org/10.17744/mehc.26.4.tp35axhd0q07r3jk

*Alvarez, K., Marroquin, Y., Sandoval, L., & Carlson, C. (2014). Integrated health care best practices and culturally and linguistically competent care: Practitioner perspectives. Journal of Mental Health Counseling, 36(2), 99–114. https://doi.org/10.17744/mehc.36.2.480168pxn63g8vkg

*Balkin, R. S., Lenz, A. S., Dell’Aquila, J., Gregory, H. M., Rines, M. N., & Swinford, K. E. (2019). Meta-analysis of integrated primary and behavioral health care interventions for treating substance use among adults. Journal of Addictions & Offender Counseling, 40(2), 84–95. https://doi.org/10.1002/jaoc.12067

Basu, S., Landon, B. E., Williams, J. W., Jr., Bitton, A., Song, Z., & Phillips, R. S. (2017). Behavioral health integration into primary care: A microsimulation of financial implications for practices. Journal of General Internal Medicine, 32(12), 1330–1341. https://doi.org/10.1007/s11606-017-4177-9

*Brubaker, M. D., & La Guardia, A. C. (2020). Mixed-design training outcomes for fellows serving at-risk youth within integrated care settings. Journal of Counseling & Development, 98(4), 446–457.
https://doi.org/10.1002/jcad.12346

Council for Accreditation of Counseling and Related Educational Programs. (2015). 2016 CACREP standards. http://www.cacrep.org/wp-content/uploads/2018/05/2016-Standards-with-Glossary-5.3.2018.pdf

Council for Accreditation of Counseling and Related Educational Programs. (2022). 2024 CACREP standards, draft three. https://www.cacrep.org/wp-content/uploads/2022/04/Draft-3-2024-CACREP-Standards.pdf

*Crowe, A., Mullen, P. R., & Littlewood, K. (2018). Self-stigma, mental health literacy, and health outcomes in integrated care. Journal of Counseling & Development, 96(3), 267–277. https://doi.org/10.1002/jcad.12201

Farrell, M. H., Kuruvilla, P., Eskra, K. L., Christopher, S. A., & Brienza, R. S. (2009). A method to quantify and compare clinicians’ assessments of patient understanding during counseling of standardized patients. Patient Education and Counseling, 77(1), 128–135. https://doi.org/10.1016/j.pec.2009.03.013

​​Fields, A. M., Linich, K., Thompson, C. M., Saunders, M., Gonzales, S. K., & Limberg, D. (2023). A systematic review of training strategies to prepare counselors for integrated primary and behavioral healthcare. Counseling Outcome Research and Evaluation, 14(1), 1–14. https://doi.org/10.1080/21501378.2022.2069555

Gerrity, M. (2016). Evolving models of behavioral health integration: Evidence update 2010–2015. Milbank Memorial Fund. https://www.milbank.org/wp-content/uploads/2016/05/Evolving-Models-of-BHI.pdf

Giese, A. A., & Waugh, M. (2017). Conceptual framework for integrated care: Multiple models to achieve integrated aims. In R. E. Feinstein, J. V. Connelly, & M. S. Feinstein (Eds.), Integrating behavioral health and primary care (pp. 3–16). Oxford University Press.

*Glueck, B. P. (2015). Roles, attitudes, and training needs of behavioral health clinicians in integrated primary care. Journal of Mental Health Counseling, 37(2), 175–188. https://doi.org/10.17744/mehc.37.2.p84818638n07447r

Hays, P. A. (1996). Addressing the complexities of culture and gender in counseling. Journal of Counseling & Development, 74(4), 332–338. https://doi.org/10.1002/j.1556-6676.1996.tb01876.x

Health Resources and Services Administration. (2022). Health professions training programs. https://data.hrsa.gov/topics/health-workforce/training-programs

Heath, B., Wise-Romero, P., & Reynolds, K. A. (2013). Standard framework for levels of integrated healthcare. SAMHSA-HRSA Center for Integrated Health Solutions. https://www.thenationalcouncil.org/wp-content/uploads/2020/01/CIHS_Framework_Final_charts.pdf?daf=375ateTbd56

*Jacobson, T., & Hatchett, G. (2014). Counseling chemically dependent chronic pain patients in an integrated care setting. Journal of Addictions & Offender Counseling, 35(1), 57–61.
https://doi.org/10.1002/j.2161-1874.2014.00024.x

*Johnson, K. F., Blake, J., & Ramsey, H. E. (2021). Professional counselors’ experiences on interprofessional teams in hospital settings. Journal of Counseling & Development, 99(4), 406–417. https://doi.org/10.1002/jcad.12393

*Johnson, K. F., & Freeman, K. L. (2014). Integrating interprofessional education and collaboration competencies (IPEC) into mental health counselor education. Journal of Mental Health Counseling, 36(4), 328–344.
https://doi.org/10.17744/mehc.36.4.g47567602327j510

*Johnson, K. F., Haney, T., & Rutledge, C. (2015). Educating counselors to practice interprofessionally through creative classroom experiences. Journal of Creativity in Mental Health, 10(4), 488–506.
https://doi.org/10.1080/15401383.2015.1044683

*Johnson, K. F., & Mahan, L. B. (2020). Interprofessional collaboration and telehealth: Useful strategies for family counselors in rural and underserved areas. The Family Journal, 28(3), 215–224.
https://doi.org/10.1177/1066480720934378

*Kohn-Wood, L., & Hooper, L. (2014). Cultural competency, culturally tailored care, and the primary care setting: Possible solutions to reduce racial/ethnic disparities in mental health care. Journal of Mental Health Counseling, 36(2), 173–188. https://doi.org/10.17744/mehc.36.2.d73h217l81tg6uv3

Kolbasovsky, A. (2008). A therapist’s guide to understanding common medical conditions: Addressing a client’s mental and physical health. W. W. Norton.

*Lenz, A. S., Dell’Aquila, J., & Balkin, R. S. (2018). Effectiveness of integrated primary and behavioral healthcare. Journal of Mental Health Counseling40(3), 249–265. https://doi.org/10.17744/mehc.40.3.06

*Lenz, A. S., & Watson, J. C. (2023). A mixed methods evaluation of an integrated primary and behavioral health training program for counseling students. Counseling Outcome Research and Evaluation, 14(1), 28–42.
https://doi.org/10.1080/21501378.2022.2063713

*Lloyd-Hazlett, J., Knight, C., Ogbeide, S., Trepal, H., & Blessing, N. (2020). Strengthening the behavioral health workforce: Spotlight on PITCH. The Professional Counselor, 10(3), 306–317.
https://doi.org/10.15241/jlh.10.3.306

McCall, M. H., Wester, K. L., Bray, J. W., Hanchate, A. D., Veach, L. J., Smart, B. D., & Wachter Morris, C. (2022). SBIRT administered by mental health counselors for hospitalized adults with substance misuse or disordered use: Evaluating hospital utilization and costs. Journal of Substance Abuse Treatment, 132, 108510. https://doi.org/10.1016/j.jsat.2021.108510

*Moe, J., Johnson, K., Park, K., & Finnerty, P. (2018). Integrated behavioral health and counseling gender and sexual minority populations. Journal of LGBT Issues in Counseling, 12(4), 215–229.
https://doi.org/10.1080/15538605.2018.1526156

Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 1–7. https://doi.org/10.1186/s12874-018-0611-x

Patient Protection and Affordable Care Act of 2010, Pub. L. No. 111-148, 124 Stat. 119 (2010).
https://www.congress.gov/111/plaws/publ148/PLAW-111publ148.pdf

Petterson, S., Miller, B. F., Payne-Murphy, J. C., & Phillips, R. L., Jr. (2014). Mental health treatment in the primary care setting: Patterns and pathways. Families, Systems, & Health, 32(2), 157–166.
https://doi.org/10.1037/fsh0000036

Ratts, M. J., Singh, A. A., Nassar-McMillan, S., Butler, S. K., & McCullough, J. R. (2016). Multicultural and social justice counseling competencies: Guidelines for the counseling profession. Journal of Multicultural Counseling and Development, 44(1), 28–48. https://doi.org/10.1002/jmcd.12035

*Regal, R. A., Wheeler, N. J., Daire, A. P., & Spears, N. (2020). Childhood sexual abuse survivors undergoing cancer treatment: A case for trauma-informed integrated care. Journal of Mental Health Counseling, 42(1), 15–31. https://doi.org/10.17744/mehc.42.1.02

*Schmit, M. K., Watson, J. C., & Fernandez, M. A. (2018). Examining the effectiveness of integrated behavioral and primary health care treatment. Journal of Counseling & Development, 96(1), 3–14.
https://doi.org/10.1002/jcad.12173

*Sheesley, A. P. (2016). Counselors within the chronic care model: Supporting weight management. Journal of Counseling & Development, 94(2), 234–245. https://doi.org/10.1002/jcad.12079

Sperry, L., & Binensztok, V. (2019). Ultra-brief cognitive behavioral interventions: A new practice model for mental health and integrated care. Routledge.

Substance Abuse and Mental Health Services Administration. (2021). Key substance use and mental health indicators in the United States: Results from the 2020 National Survey on Drug Use and Health. https://www.samhsa.gov/data/sites/default/files/reports/rpt35325/NSDUHFFRPDFWHTMLFiles2020/2020NSDUHFFR1PDFW102121.pdf

The White House. (2022, March 1). Fact sheet: President Biden to announce strategy to address our national mental health crisis, as part of unity agenda in his first State of the Union. [Press release]. https://www.whitehouse.gov/briefing-room/statements-releases/2022/03/01/fact-sheet-president-biden-to-announce-strategy-to-address-our-national-mental-health-crisis-as-part-of-unity-agenda-in-his-first-state-of-the-union

Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., . . . Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850

*Tucker, C., Sloan, S. K., Vance, M., & Brownson, C. (2008). Integrated care in college health: A case study. Journal of College Counseling, 11(2), 173–183. https://doi.org/10.1002/j.2161-1882.2008.tb00033.x

*Ulupinar, D., Zalaquett, C., Kim, S. R., & Kulikowich, J. M. (2021). Performance of mental health counselors in integrated primary and behavioral health care. Journal of Counseling & Development, 99(1), 37–46.
https://doi.org/10.1002/jcad.12352

*Veach, L. J., Moro, R. R., Miller, P., Reboussin, B. A., Ivers, N. N., Rogers, J. L., & O’Brien, M. C. (2018). Alcohol counseling in hospital trauma: Examining two brief interventions. Journal of Counseling & Development, 96(3), 243–253. https://doi.org/10.1002/jcad.12199

*Vereen, L. G., Yates, C., Hudock, D., Hill, N. R., Jemmett, M., O’Donnell, J., & Knudson, S. (2018). The phenomena of collaborative practice: The impact of interprofessional education. International Journal for the Advancement of Counselling, 40(4), 427–442. https://doi.org/10.1007/s10447-018-9335-1

*Vogel, M., Malcore, S., Illes, R., & Kirkpatrick, H. (2014). Integrated primary care: Why you should care and how to get started. Journal of Mental Health Counseling, 36(2), 130–144.
https://doi.org/10.17744/mehc.36.2.5312041n10767k51

*Wood, A. W., Zeligman, M., Collins, B., Foulk, M., & Gonzalez-Voller, J. (2020). Health orientation and fear of cancer: Implications for counseling and integrated care. Journal of Mental Health Counseling, 42(3), 265–279. https://doi.org/10.17744/mehc.42.3.06

 

Appendix A

Conceptual Articles

Author(s) Population(s) of Interest Research Gap Identified Implications and Future Directions
Aitken & Curtis, 2004 Counselor educators
and counselors
Lack of IC literature in counselor education journals Increased training for counselors to work competently in IC. Increased advocacy efforts to be on insurance panels. Build relationships with other health care professionals. More literature is needed in counselor education journals.
Jacobson & Hatchett, 2014 Clients who are chemically dependent with chronic pain Lack of literature for clients who are chemically dependent with chronic pain Clients that have co-occurring chemical dependence and chronic pain have reported benefits when their symptoms are treated by mental and physical health providers. Additional research is needed to understand treatment strategy effectiveness.
Johnson & Freeman, 2014 Health care undergraduate and graduate students (including CITs)
learning IC strategies
Lack of literature documenting IC training across multiple disciplines, specifically including CITs Provides a framework for IC training across multiple disciplines in accordance with SAMHSA IC competency standards. Additional research is needed to understand the effectiveness for each discipline and as a whole.
Johnson & Mahan, 2020 Family counselors
in rural and
underserved areas
Family counselors leading connection between rural families and other providers
of health care services
Emphasis on interprofessional collaboration (IPC) and use of telehealth options where family counselors use systemic training to advocate for rural, marginalized families, as well as network and connect families to health care providers when family members have unmet medical health needs or need specialized mental health care treatment. Additional research is needed to understand this phenomenon.
Kohn-Wood & Hooper, 2014 Mental health professionals working
in primary care settings
How culturally tailoring evidence-based treatment models can reduce mental health disparities Cultural tailoring of treatments should be a primary factor that is evaluated in future research studies. Future researchers should consult existing literature on culturally tailoring treatment to increase engagement and improve outcomes for diverse groups.
Lloyd-Hazlett et al., 2020 CITs Need for a replicable model
to train CITs in IC
The Program for the Integrated Training of Counselors in Behavioral Health (PITCH) model creates community partnerships, introduces CITs to applications of IC, and awards CITs a graduate certificate. Additional research is needed to demonstrate sustainability.
Moe et al., 2018 LGBTQ+ clients Lack of LGBTQ+ literature pertaining to IC CITs, counselors, and other health care professionals working with LGBTQ clients may benefit from additional training and supervision in collaborative care and IC. Additional research is needed to understand the impact IC has with the LGBTQ+ population.
Regal et al., 2020 Clients with cancer who are survivors of childhood sexual abuse Lack of trauma-informed
care literature pertaining
to IC, specifically for individuals with adverse childhood experiences (ACEs)
IC offers opportunities for appropriate assessments to identify ACEs for holistic care, as represented in the case study. Additional research is needed to understand universal screening for ACEs and the integration of trauma-informed practices within traditional primary care settings.
Sheesley, 2016 Counselor educators, counselors, and primary care settings Elaborate on the role of mental health counselors within the Chronic Care Model (CCM) Counselors influencing the future of obesity treatment within the CCM. Additional research is needed to understand evidence-based practices for counselors within the CCM for the treatment of obesity.
Tucker et al., 2008 An international student’s experience receiving IC on a college campus The effect of an IC program and mindfulness-based cognitive therapy (MCBT) approach As reported by the multidisciplinary team, clients using medication and individual and group therapy improved from the first time they had met. The authors emphasized the use of MCBT in treatment. Additional research is needed for IC on college campuses.
Vogel et al., 2014 Counselors considering IPC Access issues, adherence, and the effectiveness of IPC with particular attention to culturally diverse groups Increased training in evidence-based culturally tailored practices. Increased education for counselors regarding IPC to help determine if primary care is a good fit. Additional research is needed on various aspects of successful IPC execution.

 

Appendix B

Empirical Articles

Author(s) Methodology N and
Participant
Profile
Research of Interest Results
Agaskar
et al., 2021
Mixed methods; quantitative: single-group design; qualitative: thematic analysis 12 CITs The effect of an IPC and evidence-based practices curriculum to enhance students’ ability to work with at-risk youth in IC settings CITs reported an increase in multicultural competence and ability to work on IC teams, utilize evidence-based practices, and implement suicide interventions.
Alvarez
et al., 2014
Qualitative; exploratory
cross-case synthesis
8 service providers in an IC setting The experiences of IC service providers working with culturally and linguistically diverse populations Three themes emerged: (a) patient-centered care benefits underserved populations, (b) desirability of a multidisciplinary team, and
(c) importance of the organization to change with circumstances.
Brubaker & La Guardia, 2020 Quantitative;
single case and quasi-experimental
11 CITs The effect of an IC training intervention, Serving At-Risk Youth Fellowship Experience for Counselors (SAFE-C) CITs reported an increase in understanding professional identity, self-efficacy, and interprofessional socialization.
Crowe
et al., 2017
Quantitative;
Cross-sectional survey design
102 clients from an IC medical facility To examine the relationship between mental health self-stigmas, mental health literacy, and health care outcomes Self-stigma of mental illness and self-stigma of seeking help had an inverse relationship with mental health literacy.
Glueck 2015 Qualitative; phenomenological 10 mental health professionals working in IC settings Roles and attitudes of mental health professionals working in IC and perceived training needs Mental health professionals reported that they were involved in brief interventions and assessments, administrative work, and consultation and that additional graduate training is needed in classroom and field experiences.
Johnson
et al., 2015
Mixed methods; qualitative: the pre- and post-survey design; qualitative: thematic analysis 22 CITs, as well as dental hygiene, nursing, and
physical therapy students
CITs’ attitudes toward interprofessional learning and collaboration following an interdisciplinary course on IPC Perceptions about learning together and collaboration improved, negative professional identity scores decreased, and higher reports of positive professional identity.
Johnson
et al., 2021
Qualitative; phenomenology 11 counselors in hospital setting Experiences of counselors working on interprofessional teams (IPTs) in a hospital setting Four themes emerged:
(a) counselors rely on common factors and foundational principles; (b) counselors must have interprofessional supervision; (c) counselors must remember their scope of practice; and (d) counselors must adhere to ethical codes and advocacy standards.
Lenz & Watson, 2023 Mixed-methods; quantitative: non-experimental pre- and post-test; qualitative: thematic analysis 45 CITs The impact an IC training program has on CITs’ self-efficacy, interprofessional socialization, and multicultural competence, as well as barriers to student growth Increase in self-efficacy, interprofessional socialization, and aspects of multicultural competence. Most reported barriers were IC awareness and organizational constraints.
Schmit
et al., 2018
Quantitative;
quasi-experimental
196 clients; 98 received IC and 98 received treatment as usual (TAU) The effect of IC for individuals with severe mental illness compared
to TAU
Group that received the IC intervention demonstrated an improvement in overall functioning, including a
decrease in crisis events.
Ulupinar et al., 2021 Quantitative;
quasi-experimental
1,747 clients and 10 counselors To examine the therapeutic outcomes and client dropout rates of adults experiencing mental disorders in an IC center The addition of counselors resulted in a decrease in client symptom reports.
Veach
et al., 2018
Quantitative; pre- and post-test survey 333 clients in a trauma-based IC center A brief IC counseling intervention for risky alcohol behavior The IC counseling intervention resulted in reduced risky alcohol behaviors.
Vereen
et al., 2018
Qualitative; phenomenological inquiry 13 graduate students; five CITs  and eight speech– language pathologists The effect of interprofessional education (IPE) on the development of collaborative practice for both CITs and speech– language pathologists-in-training Five themes emerged:

(a) benefits of IPE,
(b) expectations of collaborative practice, (c) benefits of experienced IC providers,
(d) challenges of IC practice, and
(e) optimization of IC practice.

Wood
et al., 2020
Quantitative;
cross-sectional survey design
155 undergraduate students studying psychology and aspects of counseling How factors related to prevention and wellness relate to topics that counselors are adept at addressing, such as optimism, social support, and resilience Results indicated that health anxiety was positively correlated with fear of cancer, but that psychosocial variables either had no relationship or were not significant moderators between health anxiety and fear of cancer.

 

Appendix C

Meta-Analyses and Systematic Reviews

Author(s) Article Classification Population of Interest Number
of Included Studies and Participants
Results and Implications
Balkin
et al., 2019
Meta-analysis Adults with substance use disorders 8 studies with 1,545 participants;
722 received IC and 823
received alternative
Effects of IC were small with this sample (i.e., small effect in decrease in substance use).
Authors recommended additional research to understand substance use disorders within an IC context and variables beyond use of substances.
Fields
et al., 2023
Systematic review Mental health professionals and mental health professionals-in-training receiving education on IC 18 studies Four themes emerged:
(a) HRSA-funded studies,
(b) trainee skill development, (c) enhancement of
self-efficacy, and
(d) increased understanding of interprofessional collaboration. Authors recommended more studies focusing on client-level data and more multicultural competencies.
Lenz
et al., 2018
Meta-analysis Individuals receiving mental health care in traditional primary care settings 36 studies with 14,764 participants Effects of IC, as compared to alternative treatments, resulted in a decrease in mental health symptoms. A greater effect is shown with a larger treatment team and number of behavioral health sessions.

 

Alexander M. Fields, PhD, is an assistant professor at the University of Nebraska at Omaha. Cara M. Thompson, PhD, is an assistant professor at the University of North Carolina at Pembroke. Kara M. Schneider, MS, is a doctoral candidate at the University of South Carolina. Lucas M. Perez, MA, is a doctoral candidate at the University of South Carolina. Kaitlyn Reaves, BS, is a doctoral student at Adler University. Kathryn Linich, PhD, is a clinical assistant professor at Duquesne University. Dodie Limberg, PhD, is an associate professor at the University of South Carolina. Correspondence may be addressed to Alexander M. Fields, University of Nebraska at Omaha, College of Education, Health, and Human Services, Department of Counseling, Omaha, NE 68182, alexanderfields@unomaha.edu.

Supporting Early College High School Students: The Effects of Cognitive Behavioral Therapy for Perfectionism on Perfectionism, Negative Affectivity, and Social–Emotional Well-Being

Arielle Bendit, Melissa Mariani, Paul Peluso, Elisa Calabrese

Meeting the unique needs of high-achieving students is particularly challenging for professional counselors. Currently, there is a paucity of research that prioritizes the needs and social–emotional development of high-achieving students in accelerated curricula. This study examined the effectiveness of a modified version of the cognitive behavioral therapy for perfectionism (CBT-P) small group intervention on the perfectionism, negative affectivity, and social–emotional well-being of high-achieving students. A series of ANCOVA analyses revealed a statistically significant difference in negative affectivity and self-oriented perfectionism between treatment (n = 20) and comparison (n = 21) groups. No statistically significant difference was found regarding participants’ socially prescribed perfectionism or social–emotional well-being. The modified CBT-P yielded large effects (ηp2 = .22) in reducing negative affectivity and medium effects (ηp2 = .11) in reducing self-oriented perfectionism. These findings further support this population’s need for specialized school-based interventions and illuminate future research directions.

Keywords: high-achieving students, perfectionism, negative affectivity, social–emotional well-being, CBT-P

For decades, positive stereotyping surrounding the high-achieving student population has continued to promote the misconception that high-achieving students do not face problems or social–emotional challenges (Colangelo & Wood, 2015; Peterson, 2009). Yet, a growing body of literature has led researchers to conclude that high-achieving students are not immune to mental health concerns (Kennedy & Farley, 2018; Suldo et al., 2018). In fact, high-achieving students are more likely to struggle with perfectionism, internalize expectations and problems, and maintain a façade to avoid expressing their needs (Peterson, 2009). Left undetected, mental health problems may impact academic and social–emotional success (Luthar et al., 2020).

With mental health problems increasing among adolescents, there is a growing interest in expanding strategies and early intervention programs to improve mental health outcomes (Divin et al., 2018). The school system plays an essential role in addressing adolescents’ social–emotional and academic needs, with many schools adopting multi-tiered systems of support (MTSS) to implement and evaluate schoolwide interventions (O’Brennan et al., 2019). Prevention and intervention programs being utilized within MTSS need to be tailored to meet the student populations’ needs (Dai et al., 2015). Tailoring programs is essential when considering interventions to support high-achieving students, as they typically perform well enough academically in accelerated courses to maintain enrollment, but their emotional health problems may go undetected by counselors and educators (Suldo et al., 2018). Additionally, many of the programs available to support high-achieving students’ needs are the same as those offered to general education students (O’Brennan et al., 2019), and there is little empirical evidence linking these programs or practices to positive outcomes for this population (Colangelo & Wood, 2015).

Early college high schools are an innovative way for high school students to earn both high school degrees and college degrees simultaneously. These schools are on the rise (Song et al., 2021) and meet many high-achieving students’ academic and social–emotional needs (Dai et al., 2015). However, the literature on early college high schools also highlights themes of increased academic pressure, stress, adjustment concerns, and anxiety (Dai et al., 2015; Peters & Mann, 2009). Unfortunately, the needs of high-achieving students in accelerated curricula are often not prioritized in research and there is a lack of attention in counselor education programs given to the social–emotional development of the high-achieving population (O’Brennan et al., 2019; Suldo et al., 2018). Therefore, further research is needed to demonstrate effective interventions targeted to meet high-achieving students’ unique emotional health needs.

High-Achieving Students and Mental Health
     The term high-achieving refers to students who “exhibit outstanding intellectual ability, or promise, and are capable of extraordinary performance and accomplishment” (McClain & Pfeiffer, 2012, p. 59). Unfortunately, the high-achieving population has been surrounded by positive stereotyping for years (Peterson & Lorimer, 2011), leaving them vulnerable to social isolation, stigmatization, and psychological distress. Researchers studying high-achieving students specifically have identified many concerns, including anxiety, perfectionism, suicidal ideation, bullying, academic underachievement, poor coping skills, and trauma (Peterson, 2009; Tang & Fisher, 2012). There are a number of situations, compounded by the additional factor of high ability, that put high-achieving students at a more unique risk for developing mental health problems (Colangelo & Wood, 2015; Cross & Cross, 2015). For example, high-achieving students may face challenges related to asynchronous development, which is when cognitive development outpaces the physical and social–emotional domains (Papadopolous, 2020). This disparity may cause issues in how high-achieving students experience and relate to the world (Colangelo & Wood, 2015), which can lead to social anxiety and peer rejection (Cross & Cross, 2015). Further, increased levels of stress stemming from high-ability characteristics such as over-commitment, fear of making mistakes, and high expectations are common in this population (Cross & Cross, 2015).

High-Achieving Students and Perfectionism
     Perfectionism is a personality disposition characterized by the act of striving for precision and having exceedingly high standards of performance accompanied by relentless self-criticism while in pursuit of those standards (Frost et al., 1990; Tang & Fisher, 2012). Several researchers have proposed that there are two higher-order dimensions of perfectionism: perfectionistic strivings and perfectionistic concerns (Leone & Wade, 2018; Strickler et al., 2019). Perfectionistic strivings are high personal standards and the belief that being perfect is important to oneself (Hewitt & Flett, 1991), whereas perfectionistic concerns include the fear of making mistakes, expressing self-doubt about one’s abilities, and perceiving that others expect perfection of oneself (Frost et al., 1990; Mofield & Parker Peters, 2015).

Perfectionism is one of the most frequently cited traits in high-achieving individuals (G. A. Horowitz et al., 2012; Papadopoulos, 2020) and one of the most common concerns expressed by their parents (Stricker et al., 2019). Damian and colleagues’ (2017) study was one of the first to provide evidence that high academic ability is a common factor in developing perfectionism. For many high-achieving students, pursuing perfection is a way to preserve positive self-worth, reduce shame, or gain a sense of control (Flett et al., 2002). Further research exploring perfectionism dimensions in high-achieving students found that perfectionistic strivings are associated with more positive outcomes such as happiness, increased self-esteem, life satisfaction (Chan, 2011), and greater academic self-concept (Dixon et al., 2004). In contrast, perfectionistic concerns are associated with more negative outcomes such as poor coping skills, psychological health concerns, and underachievement (Dixon et al., 2004; Stricker et al., 2019).

Universal Complete Mental Health Screening
     Comprehensive school counseling programs are intended to integrate data-driven MTSS to address academic and behavioral concerns (Belser et al., 2016). Within MTSS, all students receive universal core instruction (Tier 1). These universal programs provide approximately 80% of students with what they need to be academically, socially, and emotionally successful. Common Tier 1 supports and programming include reward systems, explicit social–emotional learning lessons, daily mindfulness activities, and modeled schoolwide expectations. With that, approximately 20% of students who receive core instruction may need additional Tier 2 (small group or mentoring) or Tier 3 (individual counseling or community referral) supports to be successful (O’Brennan et al., 2016).

MTSS efforts are critical in the early identification and prevention of long-term mental and behavioral health problems (von der Embse, 2018). As such, universal screening data is used to inform which students are in need of additional services beyond Tier 1 (Moore et al., 2019). Additionally, universal screening data helps to align counseling services and supports to the appropriate intensity needed at each tier to meet students’ social–emotional needs (Belser et al., 2016). With advancements from the positive psychology movement, universal mental health screeners have adopted a more strengths-based approach, termed complete mental health screening, to include both psychological distress and strength indicators (Furlong et al., 2018; Moore et al., 2019).

Using this approach, students are classified into one of four different mental health groups, according to their severity of psychological distress and their positive strength indicators (Suldo & Shaffer, 2008). The complete mental health group (high strengths, low distress) is typically the largest mental health group to emerge across samples (Suldo et al., 2016) and refers to individuals with optimal wellness (Suldo & Shaffer, 2008). Students with complete mental health are often successful in terms of academic skills and emotional engagement and are likely to only need Tier 1 support (Moore et al., 2019). The symptomatic group has elevated distress and strengths (Moore et al., 2019). Although research on this group is inconsistent, it suggests that the presence of positive strength indicators may act as a protective factor despite the presence of psychological distress (Suldo et al., 2016). The troubled group experiences high distress and low levels of strengths and often has the worst outcomes of the four groups (Moore et al., 2019). Finally, the vulnerable group is those who report low levels of distress and low levels of strengths (Moore et al., 2019). Students in this group are typically excluded from intervention services using traditional screening methods because of their lack of psychopathology (Suldo & Shaffer, 2008). Therefore, incorporating both psychopathology and strengths-based measures provides a more comprehensive conceptualization of students’ functioning (Furlong et al., 2018) and is more accurate in identifying students at each level of MTSS (Suldo & Shaffer, 2008).

Despite favorable findings, complete mental health screening is not widely used in school-based assessment practices (Suldo & Shaffer, 2008); in fact, most schools do not use any sort of universal screening tool (Wood & McDaniel, 2009). In contrast to screenings, many schools rely on office discipline referrals to determine whether at-risk students are in need of supplemental support services (Bruhn et al., 2014). However, discipline referrals only identify students who “act out” or struggle academically, and they do not capture students with internalized problems (Bruhn et al., 2014). Therefore, given the prevalence of internalizing concerns such as anxiety and depression among high-achieving students (Sanzone & Perez, 2019), coupled with their reluctance to seek help (Peterson, 2009), early college high school programs should use complete mental health screenings to identify and support at-risk, high-achieving students.

The Cognitive Behavior Therapy for Perfectionism Small Group Intervention
     Perfectionism has emerged as a critical vulnerability factor for a variety of maladjustments and mental health disorders (Flett & Hewitt, 2014; Speirs Neumeister, 2018). Flett and Hewitt (2014) found that interventions with an explicit focus on decreasing perfectionism were more effective in reducing negative aspects of perfectionism than general prevention programs. Further, Feiss and colleagues (2019) conducted a meta-analysis to evaluate the efficacy of school-based programming aimed at reducing internalizing mental health problems in adolescents and found that targeted programming was more effective than universal programming. Overall, interventions for Tier 2 and Tier 3 can provide additional support for at-risk students, who are more likely to access mental health treatment when referred to school-based services (Biolcati et al., 2017). When delivered in a group format, identified students are likely to find peers they can relate to and connect with (J. L. Horowitz et al., 2007).

Shafran and colleagues (2002) proposed a cognitive behavioral conceptualization of perfectionism that helps to guide counselors in assessment and treatment. Specific structured protocols were outlined by the authors as an extension of the protocols developed in the treatment studies by Steele et al. (2013) and Handley et al. (2015). Despite having structured sample protocols, all cognitive behavior therapy for perfectionism (CBT-P) treatment strategies should be used flexibly and be focused on individualized formulation (Egan & Shafran, 2018). A series of clinical trials in different settings and formats evaluated the treatment efficacy of CBT-P (Egan & Shafran, 2018; Egan et al., 2014; Handley et al., 2015). Handley and colleagues (2015) conducted the first randomized control trial to determine the efficacy of CBT-P in a group format. Participants had a range of disorders, including anxiety, depression, obsessive-compulsive disorder, and eating concerns. Those in the treatment group, who received the CBT-P group protocol, demonstrated significant large effect size (Cohen’s d = 1.2) reductions in disorders compared to those in the control group. However, only a few studies have examined the efficacy of CBT-P in adolescents, and no studies have examined CBT-P on high-achieving students within a school-based setting.

Purpose of the Present Study
     The primary purpose of this quasi-experimental, pre-post design study was to determine the effectiveness of a modified version of the CBT-P small group intervention on perfectionism, negative affectivity, and social–emotional well-being in grade 9 to 12 early college high school students. The following research questions were addressed:

RQ1. Is there a decrease in the levels of perfectionism in grade 9 to 12 early college high school students after participating in the modified CBT-P group counseling intervention?
RQ2. Is there a decrease in negative affectivity (anxiety, depression, stress) in grade 9 to 12 early college high school students after participating in the modified CBT-P group counseling intervention?
RQ3. Is there an increase in the social–emotional well-being of grade 9 to 12 early college high school students after participating in the modified CBT-P group counseling intervention?

Method

Participants and Selection
     The study was approved by both the school district and the researchers’ university IRB. The participants represented in this study were grade 9 to 12 high school students enrolled in one diverse early college high school program in the Southeastern United States. This school is the only public, accelerated pre-collegiate program with all of its students working toward a high school diploma and a cost-free bachelor’s degree simultaneously. By grade 10, students take a minimum of 12 college course credits with the general undergraduate population at a large Southeastern public university. Admission into the program is highly selective, with an 18% acceptance rate. As a result, students enrolled in the program are uniquely advanced, with exceedingly high academic achievement and initiative. As of August 2021, the student body (668 students) demographic breakdown included 35% White non-Hispanic, 27% Hispanic, 17% Asian or Pacific Islander, 13% Black non-Hispanic, and 7% Multiracial students. Across the student body, 58% identified as female and 42% identified as male.

Participants were selected from the sample population through the use of universal complete mental health screeners. Students with parental consent were instructed, as a part of their school’s comprehensive counseling program, to complete the mental health screeners in the third week of the school semester. One group of students participated in the fall 2021 semester and another group participated in the spring 2022 semester. The administered instruments were completed via computer-based administration and took approximately 10 to 15 minutes to complete. The school counseling team then sorted the students into four distinct mental health groups based on negative affectivity scores, assessed by the Depression Anxiety Stress Scale (DASS-21), and social–emotional well-being scores, assessed by the Social Emotional Health Survey-Secondary (SEHS-S). T-scores determined the criteria for high versus low scores. Table 1 includes the breakdown of the results.

Table 1
Groups Yielded From a Dual-Factor Model of Mental Health

Semester Negative Affectivity      Low SEWB Average to High SEWB
Low  II. Vulnerable

91

51

 I. Complete Mental Health

336

197

 

Fall 2021

Spring 2022

High  IV. Troubled

14

13

III. Symptomatic but Content

92

48

Note. SEWB = social–emotional well-being.

In the fall 2021 semester, 192 (36%) out of 533 students were categorized into the high-needs groups (symptomatic, vulnerable, troubled) and identified as needing supplemental intervention. In the spring 2022 semester, 287 students completed the screener, and 90 students (31%) were identified as needing supplemental intervention. After reviewing the results, the researchers prioritized students in the high-need groups with the first opportunity to participate in the study. Students identified in the complete mental health group and considered to have optimal wellness were not prioritized for the study; however, they were still eligible to participate. The CBT-P program and study procedures were described in the informed consent letter to parents and students. Refer to Table 2 for the demographic breakdown for the participants (N = 46) in both the treatment (n = 23) and comparison (n = 23) groups.

Table 2
Treatment and Comparison Group Percentages for Demographics

Demographics Treatment

(n = 23)

Comparison

(n = 23)

Ethnicity White 39% 43%
Hispanic 28% 17%
Black 4% 17%
Asian or Pacific Islander 13% 13%
Multiracial 28% 9%
Gender Male 13% 13%
Female 87% 87%
Grade Level Grade 9 35% 22%
Grade 10 43% 17%
Grade 11 17% 35%
Grade 12 4% 26%
Universal Screener Group Complete 52% 39%
Symptomatic 9% 30%
Vulnerable 35% 22%
Troubled 4% 9%

 Note. n = number of students.

Instrumentation
Depression Anxiety Stress Scale-21
     The Depression Anxiety Stress Scale-21 (DASS-21) is a self-report measure designed to assess internalizing symptoms (Lovibond & Lovibond, 1995). Respondents use a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me most of the time) to rate the extent to which each item applied to them over the past week. The 21-item measure consists of three 7-item subscales (Depression, Anxiety, Stress). A total score is created by adding each of the items, with higher scores indicating higher levels of severity (Lovibond & Lovibond, 1995). Among adolescent samples, each subscale had good internal consistency scores (Depression, α = .97 to .88; Anxiety, α = .92 to .79; Stress, α = .95 to .81; total α = .93), and strong convergent validity (from .68 to .79; Antony et al., 1998). The DASS-21 demonstrated strong internal reliability (α = .93) for this study. For the purpose of this study, the
DASS-21 was utilized to assess negative affectivity. 

Social Emotional Health Survey-Secondary
     The Social Emotional Health Survey-Secondary (SEHS-S) is a 36-item self-report survey designed to assess positive psychological dispositions among adolescents ages 13 to 18 (Furlong et al., 2013). The measure has 12 subscales of positive social–emotional health constructs that create four general traits: (a) emotional competence, (b) engaged living, (c) belief in self, and (d) belief in others. These four traits are combined to make up an overall strength score. Participants rate themselves using a 4-point scale (1 = not at all true, 2 = a little true, 3 = pretty much true, 4 = very much true). Participants’ social–emotional health scores are grouped along a continuum of low, low average, average, high average, and high scores based on a validated national sample of students (Furlong et al., 2018). Initial validation reported high internal reliability scores, including emotional competence (α = .78), engaged living (α = .87), belief in self (α = .76), and belief in others (α = .81; Furlong et al., 2013). Calculated reliability for this study was high (α = .94). The measure also demonstrated strong psychometric properties across validation studies with diverse samples (Lee et al., 2016; You et al., 2014).

Child Adolescent Perfectionism Scale
     The Child Adolescent Perfectionism Scale (CAPS) is one of the most widely used multidimensional measures of perfectionism in children and adolescents (Flett et al., 2016). The CAPS is a 22-item measure designed to assess two subscales: Socially Prescribed Perfectionism (i.e., the perception or belief that others demand perfection from the self) and Self-Oriented Perfectionism (i.e., exceedingly high personal standards). The 22-item measure uses a 5-point scale (1 = falsenot at all true of me, and 5 = very true of me), with higher scores indicating greater levels of perfectionism. The CAPS demonstrated good internal reliability, reporting Cronbach’s alpha levels of .86 and .85 for Socially Prescribed Perfectionism and Self-Oriented Perfectionism, respectively. Calculated reliability for this study was .80 for the Socially Prescribed subscale and .76 for the Self-Oriented subscale.

Procedures
     The study utilized a quasi-experimental, non-equivalent groups research design with pretest and posttest. Participants were identified from the sample population using universal screening measures and then assigned to either the treatment group or comparison group using matching procedures based on demographic data and consideration of student availability and intervention group times. Matching aimed to reduce bias by selecting subsets of the treatment and comparison groups with similar observed covariate distributions (Stuart & Rubin, 2007). Participants in the treatment group received the modified CBT-P small group intervention, while participants in the comparison group did not receive the CBT-P small group intervention. Participants in the comparison group were eligible to receive the CBT-P small group intervention the following semester, once the study was completed.

Participating master’s-level counselors-in-training (CITs), currently fulfilling their internship requirements through the university’s counseling and psychological services, received the CBT-P guide and a 4-hour training on the CBT-P protocols and conceptual framework (Egan et al., 2014). CITs were eligible to participate in the study if they met the following criteria: (a) they had at least 1 year of previous counseling experience; (b) they had weekly individual supervision; (c) they were in good academic standing with their training program; and (d) they attended all CBT-P training. To ensure treatment fidelity, the CITs received 1-hour weekly group supervision for consultation and support and completed a weekly electronic report. The report collected information on the date of each weekly session, participant attendance, start and end times, unanticipated problems or issues, perceived effectiveness, and a completed checklist of session content.

After receiving the training and program materials, CITs co-facilitated weekly CBT-P small group counseling sessions with the treatment groups. Session protocols developed and evaluated by Handley et al. (2015) and Steele et al. (2013) suggested eight 2-hour group sessions; however, to increase feasibility and fidelity, session protocols were pared down to eight 1-hour group sessions to meet the students’ scheduling needs within their school setting. Arielle Bendit, the first author, modified the treatment in accordance with the flexible nature of the CBT-P guidelines (Egan et al., 2014). Table 3 outlines the session modules and components.

Table 3
Sessions and Session Components of CBT-P Group Treatment Protocol

Session Session Components  
1. What Is Perfectionism? What are the core features of perfectionism?

Why does perfectionism persist?

Pros and cons of perfectionism

2. Self-Monitoring Domains of perfectionism

Tips for effective self-monitoring

3. Surveys and Behavioral Experiments Develop more functional belief system

Introduce behavioral experiments

4. New Ways of Thinking All-or-nothing thinking

Moving toward flexibility

5. Broadening Attention, Cognitive
Distortions, Diaries
Challenge distortions

Increased awareness through use of thought diary

6. Procrastination and Problem Solving Ways to challenge and overcome procrastination

Time management and pleasant activities

7. Values and Reducing Self-Criticism How to respond to self-criticism

Increasing self-compassion

8. Expanding Self–Evaluation, Goals,
and Relapse Prevention
Encourage realistic and flexible goals

How to deal with setbacks

 Note. Based on protocol developed and evaluated by Steele et al. (2013) and Handley et al. (2015).

Round 1
     In August 2021, five CITs completed the CBT-P training. Participating students then completed the CAPS pretest. Bendit then assigned participants (n = 27) to their condition groups, with 14 participants assigned to the treatment group and 13 participants assigned to the comparison group. The CBT-P treatment groups started in October 2021 and ended in December 2021, with all participants again completing the CAPS posttest. Final posttest data were collected in January 2022 when participants completed their second complete universal mental health screener (DASS-21 and SEHS-S).

Round 2
     Participants (n = 19) completed the CAPS pretest in January 2022 and then were assigned to the treatment group (n = 9) or comparison group (n = 10). The treatment groups started in February 2022 and met weekly for 8 weeks. The last group session occurred in April 2022, and all participants completed the CAPS posttest. Participants completed the final posttest data (DASS-21 and SEHS-S) in May 2022 as a part of the school’s counseling program.

Preliminary Analysis
     Researchers use an analysis of covariance (ANCOVA) when they want to compare two or more groups on one dependent variable at the same time (Heppner & Heppner, 2004). To determine if it is appropriate to use a one-way ANCOVA, the data needs to meet the necessary statistical assumptions. First, Bendit and Paul Peluso determined that the dependent and covariate variables were all measured on a continuous scale and the independent variable consisted of two or more independent groups. Next, Bendit and Peluso determined that independence of observation was met, with the treatment and comparison groups having different participants in each group with no participant being in more than one group. There were two outliers detected (DASS-21 pre and CAPS pre), but we opted to keep these values included. Next, we used Shapiro-Wilk tests to determine that the data met normal distribution. Homogeneity of variances was assessed by a visual inspection of a scatterplot, and there was a linear relationship between pre and post for each intervention type.

Data Analysis
     To address the research questions in this study, an alpha level of .05 was set and a series of one-way ANCOVA was used to analyze the effect of the independent variable (CBT-P group; Shafran et al., 2002) on the dependent variables (perfectionism, negative affectivity, social–emotional well-being) between the treatment and comparison groups. Results were reported using the F statistic and associated p-value (alpha .05), indicating statistical significance (Heppner & Heppner, 2004). Effect size was also calculated using the partial eta-square statistic (ηp2), with benchmarks set forth to determine small (.01), medium (.06), and large (.14) effect size strength (Maher et al., 2013). We did not include participants’ data in the analysis if they did not attend a minimum of four group sessions or if survey data was missing at posttest.

Results

The treatment and comparison groups’ means, standard deviations, and change scores for the study variables at pretest and posttest are provided in Table 4. To control for differences prior to treatment, the participants’ pretest scores were used as covariates and group as a factor. The first research question explored the impact of the CBT-P group intervention on levels of perfectionism. Results from the one-way ANCOVA revealed a statistically significant difference [F(1, 38) = 4.94, p = .03; ηp2 of .11] in Self-Oriented Perfectionism between treatment and comparison groups and no statistically significant difference [F(1, 38) = .04, p = .83; ηp2 of .00] in Socially Prescribed Perfectionism between treatment and comparison groups. This significant medium effect finding revealed that using the modified CBT-P small group intervention (Shafran et al., 2002) can positively influence levels of self-oriented perfectionism among early college high school students. Participants in the treatment group (M = 39.15, SD = 5.78) reported lower scores for Self-Oriented Perfectionism at posttest as opposed to participants in the comparison group (M = 43.20, SD = 6.50), a mean difference of −4.05. The second research question assessed if the students who received the CBT-P group intervention would show a decrease in negative affectivity compared to students in the comparison group who did not receive the intervention. Results from the ANCOVA revealed a statistically significant difference [F(1, 37) = 10.35, p = .003; ηp2 of .22, a large effect] between treatment and comparison groups. Notably, the post hoc test revealed that total negative affectivity was statistically significantly higher in the comparison group (M = 65.90, SD = 34.11) compared to the treatment group (M = 33.80, SD = 21.38), a mean difference of 32.10. The final research question tested the hypothesis that students in the treatment group who received the CBT-P group intervention would show an increase in social–emotional well-being compared to students in the comparison group. ANCOVA results revealed no statistically significant differences [F(1, 37) = .007, p = .94; ηp2 of .00, no effect] between the treatment and comparison groups scores from pretest to posttest.

Table 4
Treatment and Comparison Group Means, Standard Deviations, and Change Scores for the Study Variables

Measure Condition (n) Pretest

M(SD)

Posttest

M(SD)

M +/−
DASS-21 Treatment (20) 42.80

(25.50)

33.80

(21.38)

−9.00
Comparison (20) 55.40

(30.12)

65.90

(34.11)

+10.50
SEHS-S Treatment (20) 102.80

(17.34)

105.80

(16.19)

+3.00
Comparison (20) 103.70

(15.45)

106.65

(13.76)

+2.95
CAPS
          SOP Treatment (20) 47.45

(7.88)

39.15

(5.78)

−8.30
Comparison (21) 47.30

(7.83)

43.20

(6.50)

−4.10
          SPP

 

Treatment (20) 31.75

(9.27)

29.40

(5.78)

−2.35
Comparison (21) 31.25

(8.47)

29.85

(7.93)

−1.40

Note. DASS-21 = Depression Anxiety Stress Scale; SEHS-S = Social Emotional Health Survey Secondary; CAPS = Child Adolescent Perfectionism Scale; SOP = Self-Oriented Perfectionism Subscale; SPP = Socially Prescribed Perfectionism Subscale; n = number, M = mean; SD = standard deviation; +/− = mean change score.

Discussion

Findings from this study were consistent with previous studies that demonstrated the successful impact of CBT-P group therapy on perfectionism, anxiety, stress, and depression (Handley et al., 2015;
Steele et al., 2013). This study also addressed Egan and colleagues’ (2014) recommendation to investigate the effects of CBT-P within other mental health settings by exploring the effects within an early college high school. With high academic achievement being a common factor in the development of perfectionism (Damian et al., 2017), reducing self-oriented perfectionism is crucial within this population, as students with higher levels of self-oriented perfectionism are less likely to seek formal support and more likely to suffer in silence (Zeifman et al., 2015).

Despite findings that did not support statistically significant differences in socially prescribed perfectionism between treatment and comparison groups, there was a decrease in socially oriented perfectionism for both groups. Additionally, both groups noted lower levels of socially prescribed perfectionism compared to self-oriented perfectionism. Socially prescribed perfectionism is most consistently associated with mental health stigma and concerns around attending counseling (Dang et al., 2020). Therefore, the student’s right to ultimately determine their initial and ongoing participation in the study may indicate they were more comfortable and not as concerned about participating in group counseling services even before the intervention started. Therefore, participants’ lower levels of socially prescribed perfectionism may have skewed the lack of statistically significant differences found between groups and minimal changes between pretest and posttest scores.

Collectively, these findings support the argument that perfectionism is a transdiagnostic process. A transdiagnostic process is “an aspect of cognition or behavior that may contribute to the maintenance of a psychological disorder” (Egan et al., 2014, p. 40). Studies have recognized levels of perfectionism are elevated across disorders (Egan et al., 2014; Handley et al., 2014) and as a maintaining factor across disorders (Egan et al., 2011). Within this study, the finding of a significant large effect size in terms of reducing negative affectivity is fascinating, as the CBT-P intervention does not target these symptoms directly. Overall, this study supports Egan and colleagues’ (2011) claim that CBT-P significantly decreases a wide range of psychological symptoms through its focus on perfectionism (Egan et al., 2011). However, it is important to note that other therapeutic factors of the modified CBT-P small group intervention may have contributed to an overall decrease in negative affectivity. Finally, this study also supports Feiss and colleagues’ (2019) assertion that targeted programming is more effective than universal programming in addressing internalizing symptoms such as anxiety, depression, and stress.

Implications for Practice and Future Research
     The findings of this study extend what is known about the underlying factors that impact high-achieving students’ overall well-being and unique needs. The study results offer support for CBT-P as an effective intervention for decreasing negative affectivity and aspects of perfectionism in high-achieving students. However, further research is needed to identify other beneficial, targeted interventions to help support this population (Colangelo & Wood, 2015). Additionally, counselor education programs need more education and training when working with the high-achieving population in accelerated programs (O’Brennan et al., 2019), as adolescents present with the highest prevalence of mental health problems (Corry & Leavey, 2017).

School counselors and other school-based professional counselors are well-positioned to address these problems through intervention. Unfortunately, increased national attention on student academic achievement has challenged school counselors to provide interventions that promote student academic success (Collins, 2014), and the duality of school counselors’ roles in addressing both mental health and academic needs serves as a barrier to providing students with adequate mental health support (Lambie et al., 2019). Yet, school counselors must first address student mental health in order to improve student achievement (Collins, 2014), as students’ unmet mental health needs pose barriers to learning (American School Counselor Association, 2020). Findings from this study could help school counselors recognize the value of collaborating with training programs and utilizing CITs to meet students’ mental health needs, particularly given high school counselor-to-student ratios. Utilizing additional personnel, such as CITs, can bolster the number of services offered and students served.

Kennedy and Farley (2018) have noted the importance of tailoring treatment approaches to best meet high-achieving students’ needs by acknowledging a students’ giftedness as a part of their identity. Further, it is important to consider how high-achieving students’ abilities may interact with other high achievement factors, such as perfection. The literature on perfectionism demonstrates a strong association with dichotomous thinking patterns, underachievement, and anxiety (Kennedy & Farley, 2018). The modified CBT-P small group intervention (Shafran et al, 2002) specifically addressed these associations and underlying concerns through various treatment strategies. This study verified the importance of understanding contextual factors (such as accelerated curricula) and underlying concerns (such as perfectionism) in high-achieving students when formulating theories and designing and implementing targeted therapeutic support. Therefore, future studies should consider and assess other factors of high achievement when delivering tailored interventions to high-achieving students in early college programs.

Future research should also explore the impacts of universal classroom-based CBT-P interventions, both as stand-alone programs and in conjunction with small-group counseling interventions. A classroom approach has the potential to reach even more high-achieving students, specifically those who are more prone not to seek help or self-disclose mental health concerns (Flett & Hewitt, 2014; Leone & Wade, 2018). Further, research should continue to explore the impact of delivery modality (i.e., in-person or online) of the modified CBT-P small group intervention (Shafran et al., 2002) on early college high school students. Despite the efficacy of online therapy being well established in the treatment of mental health concerns, some studies suggest that students would prefer face-to-face support over online therapy (Horgan & Sweeney, 2010; Sweeney et al., 2019). Although some students prefer in-person groups, because of COVID-19 and IRB barriers, attending an in-person group was not an option for this study.

Limitations
     As with any study, this one had limitations. For one, the generalizability of the findings are limited, as the study only included one early college high school in the Southeastern United States. Second, a non-equivalent comparison group design was used because randomization was not possible within the school setting. So future studies might consider a true random assignment of students to the group conditions to control for confounding variables and increase internal validity. Third, the study was also limited to half a school year for both rounds and did not examine the impact of the modified CBT-P intervention over time. Future studies should explore the impact of CBT-P over time with a larger sample size.

Another limitation of the study was the small sample size (N = 41). The original sample size in the study (N = 46) was above the adequate sample size (N = 42) according to the a priori G-power analysis calculation. However, barriers with recruitment and retention existed, resulting in a final total sample size of N = 41. Missing data can make it more challenging to carry out a true intention-to-treat analysis. Attrition most notably occurred within the treatment group after attending at least one CBT-P small group. Pfeiffer (2021) identified two necessary components that are critical for therapy success for the high achieving: (a) establishing and maintaining a strong therapeutic relationship; and (b) a deep understanding of working with gifted children. Certified school counselors may have more knowledge and training and be better equipped than CITs to develop strong therapeutic relationships within this student population. Therefore, using certified school counselors who are already embedded within the school and have the necessary training could improve participant recruitment and retention.

Also, Bendit was a part of the comprehensive school counseling team and therefore had access to student information, which may have contributed to potential selection bias. However, to adjust for selection bias, Bendit used rigorous criteria to avoid confounding results and matched participants in the study groups as closely as possible. A final limitation surrounded the measures used. The instruments were student self-report and, therefore, are subject to self-report bias. Self-report bias is particularly problematic with high-achieving students, who are more likely to minimize problems (Luthar et al., 2020) and be reluctant to ask for help (Peterson, 2009). Further, the practice effect of taking a pretest may influence the outcome of posttests, which could influence participants to be more responsive to the intervention. Additional research should address other data sources in addition to self-report measures, such as attendance, grades, and parent report. Despite these limitations, the data provides promising preliminary evidence for the effectiveness of the modified CBT-P group intervention for targeting students’ levels of self-oriented perfectionism and negative affectivity.

Conclusion

Results of this study supported the prediction that students who received the modified CBT-P small group intervention (Shafran et al., 2002) would show significant differences in negative affectivity and self-oriented perfectionism compared to their counterparts who did not receive the intervention. Overall, this study can help to inform comprehensive school counseling programs and demonstrate the importance of implementing more targeted identification interventions for high-achieving student populations that meet their unique needs.

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

References

American School Counselor Association. (2020). The school counselor and student mental health. https://www.schoolcounselor.org/Standards-Positions/Position-Statements/ASCA-Position-Statements/The-School-Counselor-and-Student-Mental-Health

Antony, M. M., Beiling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998). Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychological Assessment, 10(2), 176–181. https://doi.org/10.1037/1040-3590.10.2.176

Belser, C. T., Shillingford, M. A., & Joe, J. R. (2016). The ASCA model and a multi-tiered system of supports: A framework to support students of color with problem behavior. The Professional Counselor, 6(3), 251–262. https://doi.org/10.15241/cb.6.3.251

Biolcati, R., Palareti, L., & Mameli, C. (2017). What adolescents seeking help teach us about a school-based counseling service. Child and Adolescent Social Work Journal, 35(1), 45–56.
https://doi.org/10.1007/S10560-017-0503-7

Bruhn, A. L., Woods-Groves, S., & Huddle, S. (2014). A preliminary investigation of emotional and behavioral screening practices in K–12 schools. Education and Treatment of Children, 37(4), 611–634.
https://doi.org/10.1353/etc.2014.0039

Chan, D. W. (2011). Perfectionism among Chinese gifted and nongifted students in Hong Kong: The use of the Revised Almost Perfect Scale. Journal for the Education of the Gifted, 34(1), 68–98.
https://doi.org/10.1177/016235321003400104

Colangelo, N., & Wood, S. M. (2015). Counseling the gifted: Past, present, and future directions. Journal of Counseling & Development, 93(2), 133–142. https://doi.org/10.1002/j.1556-6676.2015.00189.x

Collins, T. P. (2014). Addressing mental health needs in our schools: Supporting the role of school counselors.
The Professional Counselor, 4(5), 413–416. https://doi.org/10.15241/tpc.4.5.413

Corry, D. A. S., & Leavey, G. (2017). Adolescent trust and primary care: Help-seeking for emotional and psychological difficulties. Journal of Adolescence, 54(1), 1–8. https://doi.org/10.1016/j.adolescence.2016.11.003

Cross, J. R., & Cross, T. L. (2015). Clinical and mental health issues in counseling the gifted individual. Journal of Counseling & Development, 93(2), 163–172. https://doi.org/10.1002/j.1556-6676.2015.00192.x

Dai, D. Y., Steenbergen-Hu, S., & Zhou, Y. (2015). Cope and grow: A grounded theory approach to early college entrants’ lived experiences and changes in a STEM program. Gifted Child Quarterly, 59(2), 75–90.
https://doi.org/10.1177/0016986214568719

Damian, L. E., Stoeber, J., Negru-Subtirica, O., & Băban, A. (2017). On the development of perfectionism: The longitudinal role of academic achievement and academic efficacy. Journal of Personality, 85(4), 565–577. https://doi.org/10.1111/jopy.12261

Dang, S. S., Quesnel, D. A., Hewitt, P. L., Flett, G. L., & Deng, X. (2020). Perfectionistic traits and self-presentation are associated with negative attitudes and concerns about seeking professional psychological help. Clinical Psychology & Psychotherapy, 27(5), 621–629. https://doi.org/10.1002/cpp.2450

Divin, N., Harper, P., Curran, E., Corry, D., & Leavey, G. (2018). Help-seeking measures and their use in adolescents: A systematic review. Adolescent Research Review, 3(1), 113–122.
https://doi.org/10.1007/s40894-017-0078-8

Dixon, F. A., Lapsley, D. K., & Hanchon, T. A. (2004). An empirical typology of perfectionism in gifted adolescents. Gifted Child Quarterly, 48(2), 95–106. https://doi.org/10.1177/001698620404800203

Egan, S. J., & Shafran, R. (2018). Cognitive-behavioral treatment for perfectionism. In J. Stoeber (Ed.), The psychology of perfectionism: Theory, research, applications (pp. 284–305). Routledge.

Egan, S. J., Wade, T. D., & Shafran, R. (2011). Perfectionism as a transdiagnostic process. Clinical Psychology Review, 31(2), 203–212. https://doi.org/10.1016/j.cpr.2010.04.009

Egan, S. J., Wade, T. D., Shafran, R., & Antony, M. M. (2014). Cognitive-behavioral treatment of perfectionism. Guilford.

Feiss, R., Dolinger, S. B., Merritt, M., Reiche, E., Martin, K., Yanes, J. A., Thomas, C. M., & Pangelinan, M. (2019). A systematic review and meta-analysis of school-based stress, anxiety, and depression prevention programs for adolescents. Journal of Youth and Adolescents, 48(9), 1668–1685. https://doi.org/10.1007/s10964-019-01085-0

Flett, G. L., & Hewitt, P. L. (2014). A proposed framework for preventing perfectionism and promoting resilience and mental health among vulnerable children and adolescents. Psychology in Schools, 51(9), 899–912. https://doi.org/10.1002/pits.21792

Flett, G. L., Hewitt, P. L., Besser, A., Su, C., Vaillancourt, T., Boucher, D., Munroe, Y., Davidson, L. A., & Gale, O. (2016). The Child–Adolescent Perfectionism Scale: Development, psychometric properties, and associations with stress, distress, and psychiatric symptoms. Journal of Psychoeducational Assessment, 34(7), 634–652. https://doi.org/10.1177/0734282916651381

Flett, G. L., Hewitt, P. L., Oliver, J. M., & Macdonald, S. (2002). Perfectionism in children and their parents: A developmental analysis. In G. L. Flett & P. L. Hewitt (Eds.), Perfectionism: Theory, research, and treatment (pp. 89–132). American Psychological Association.

Frost, R. O., Marten, P., Lahart, C., & Rosenblate, R. (1990). The dimensions of perfectionism. Cognitive Therapy and Research, 14(5), 449–468. https://doi.org/10.1007/BF01172967

Furlong, M. J., Dowdy, E., & Nylund-Gibson, K. (2018). Social Emotional Health Survey-Secondary—2015 edition. University of California Santa Barbara, International Center for School Based Youth Development.

Furlong, M. J., You, S., Renshaw, T. L., O’Malley, M. D., & Rebelez, J. (2013). Preliminary development of the Positive Experiences at School Scale for elementary school children. Child Indicators Research, 6(1), 753–775. https://doi.org/10.1007/s12187-013-9193-7

Handley, A. K., Egan, S. J., Kane, R. T., & Rees, C. S. (2014). The relationship between perfectionism, pathological worry and generalised anxiety disorder. BMC Psychiatry, 14, 1–8. https://doi.org/10.1186/1471-244X-14-98

Handley, A. K., Egan, S. J., Kane, R. T., & Rees, C. S. (2015). A randomised controlled trial of group cognitive behavioural therapy for perfectionism. Behaviour Research and Therapy, 68(1), 37–47.
https://doi.org/10.1016/j.brat.2015.02.006

Heppner, P. P., & Heppner, M. J. (2004). Writing and publishing your thesis, dissertation, and research: A guide for students in the helping profession. Cengage.

Hewitt, P. L., & Flett, G. L. (1991). Perfectionism in the self and social contexts: Conceptualization, assessment, and association with psychopathology. Journal of Personality and Social Psychology, 60(3), 456–470.
https://doi.org/10.1037/0022-3514.60.3.456

Horgan, Á., & Sweeney, J. (2010). Young students’ use of the internet for mental health information and support. Journal of Psychiatric and Mental Health Nursing, 17(2), 117–123.
https://doi.org/10.1111/j.1365-2850.2009.01497.x

Horowitz, J. L., Garber, J., Ciesla, J. A., Young, J. F., & Mufson, L. (2007). Prevention of depressive symptoms in adolescents: A randomized trial of cognitive-behavioral and interpersonal prevention programs. Journal of Consulting and Clinical Psychology, 75(5), 693–706. https://doi.org/10.1037/0022-006X.75.5.693

Tang, J. J., & Fisher, P. A. (2012). High achieving students and their experience of the pursuit of academic excellence. ISEP International Symposium, 475–499.

Kennedy, K., & Farley, J. (2018). Counseling gifted students: School-based considerations and strategies. International Electronic Journal of Elementary Education, 10(3), 361–367.
https://doi.org/10.26822/iejee.2018336194

Lambert, M. J. (2005). Early response in psychotherapy: Further evidence for the importance of common factors rather than “placebo effects.” Journal of Clinical Psychology, 61(7), 855–869. https://doi.org/10.1002/jclp.20130

Lambie, G. W., Stickl Haugen, J., Borland, J. R., & Campbell, L. O. (2019). Who took “counseling” out of the role of professional school counselors in the United States? Journal of School-Based Counseling Policy and Evaluation, 1(3), 51–61. https://doi.org/10.25774/7kjb-bt85

Lee, S.-Y., You, S., & Furlong, M. J. (2016). Validation of the Social Emotional Health Survey–Secondary for Korean school students. Child Indicators Research, 9(1), 73–92. https://doi.org/10.1007/s12187-014-9294-y

Leone, E. M., & Wade, T. D. (2018). Measuring perfectionism in children: A systematic review of the mental health literature. European Child and Adolescent Psychiatry, 27(5), 553–567. https://doi.org/10.1007/s00787-017-1078-8

Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335–343. https://doi.org/10.1016/0005-7967(94)00075-u

Luthar, S. S., Kumar, N. L., & Zillmer, N. (2020). Teachers’ responsibilities for students’ mental health: Challenges in high achieving schools. International Journal of School and Educational Psychology, 8(2), 119–130.
https://doi.org/10.1080/21683603.2019.1694112

Maher, J. M., Markey, J. C., & Ebert-May, D. (2013). The other half of the story: Effect size analysis in quantitative research. Life Sciences Education, 12(3), 345–351. https://doi.org/10.1187/cbe.13-04-0082

McClain, M.-C., & Pfeiffer, S. (2012). Identification of gifted students in the United States today: A look at state definitions, policies, and practices. Journal of Applied School Psychology, 28(1), 59–88.
https://doi.org/10.1080/15377903.2012.643757

Mofield, E. L., & Parker Peters, M. (2015). Multidimensional perfectionism within gifted suburban adolescents: An exploration of typology and comparison of samples. Roeper Review, 37(2), 97–109.
https://doi.org/10.1080/02783193.2015.1008663

Moore, S. A., Mayworm, A. M., Stein, R., Sharkey, J. D., & Dowdy, E. (2019). Languishing students: Linking complete mental health screening in schools to Tier 2 intervention. Journal of Applied School Psychology, 35(3), 257–289. https://doi.org/10.1080/15377903.2019.1577780

O’Brennan, L. M., Suldo, S. M., Shaunessy-Dedrick, E., Dedrick, R. F., Parker, J. S., Lee, J. S, Ferron, J. M., & Hanks, C. (2019). Supports for youth in accelerated high school curricula: An initial study of applicability and acceptability of a motivational interviewing intervention. Gifted Child Quarterly, 64(1), 19–40. https://doi.org/10.1177/0016986219886933

Papadopoulos, D. (2020). Psychological framework for gifted children’s cognitive and socio-emotional development: A review of the research literature and implications. Journal for the Education of Gifted Young Scientists, 8(1), 305–323. https://doi.org/10.17478/jegys.6666308

Peters, S. J., & Mann, R. L. (2009). Getting ahead: Current secondary and postsecondary acceleration options for high-ability students in Indiana. Journal of Advanced Academics, 20(4), 630–657.
https://doi.org/10.1177/1932202X0902000404

Peterson, J. S. (2009). Myth 17: Gifted and talented individuals do not have unique social and emotional needs. Gifted Child Quarterly, 53(4), 280–282. https://doi.org/10.1177/0016986209346946

Peterson, J. S., & Lorimer, M. (2011). Student response to a small-group affective curriculum in a school for gifted children. Gifted Child Quarterly, 55(3), 167–180. https://doi.org/10.1177/0016986211412770

Pfeiffer, S. I. (2021). Optimizing favorable outcomes when counseling the gifted: A best practices approach. Gifted Education International, 37(2), 142–157. https://doi.org/10.1177/0261429420969917

Sanzone, J. L., & Perez, W. Y. (2019). Evaluation of Florida Atlantic University High School. Center for Policy, Research, and Evaluation. New York University Metropolitan Center for Research on Equity and the Transformation of Schools. https://docs.steinhardt.nyu.edu/pdfs/metrocenter/atn293/pre/FAUHS_Report_190418.pdf?_ga=2.204438736.1918068477.1696694501-1311930010.1696694501

Shafran, R., Cooper, Z., & Fairburn, C. G. (2002). Clinical perfectionism: A cognitive–behavioral analysis. Behaviour Research and Therapy, 40(7), 773–791. https://doi.org/10.1016/S0005-7967(01)00059-6

Song, M., Zeiser, K., Atchison, D., & Brodziak de los Reyes, I. (2021). Early college, continued success: Longer-term impact of early college high schools. Journal of Research on Educational Effectiveness, 14(1), 116–142. https://doi.org/10.1080/19345747.2020.1862374

Speirs Neumeister, K. L. (2018). Perfectionism in gifted students. In J. Stoeber (Ed.), The psychology of perfectionism: Theory, research, applications (pp. 134–154). https://doi.org/10.4324/9781315536255

Steele, A. L., Waite, S., Egan, S. J., Finnigan, J., Handley, A., & Wade, T. D. (2013). Psycho-education and group cognitive-behavioral therapy for clinical perfectionism: A case-series evaluation. Behavioral and Cognitive Psychotherapy, 41(2), 129–143. https://doi.org/10.1017/S1352465812000628

Stricker, J., Buecker, S., Schneider, M., & Preckel, F. (2019). Intellectual giftedness and multidimensional perfectionism: A meta-analytic review. Educational Psychology Review, 32(1), 391–414.
https://doi.org/10.1007/s10648-019-09504-1

Stuart, E. A., & Rubin, D. B. (2007). Best practices in quasi-experimental designs: Matching methods for casual inference. In J. W. Osborne (Ed.), Best practices in quantitative methods (p. 155–176). SAGE.

Suldo, S. M., O’Brennan, L. M., Storey, E. D., & Shaunessy-Dedrick, E. (2018). “But I’ve never had to study or get help before!”: Supporting high school students in accelerated courses. National Association of School Psychologists Communiqué, 46(6), 18–21. https://files.eric.ed.gov/fulltext/ED602891.pdf

Suldo, S. M., & Shaffer, E. J. (2008). Looking beyond psychopathology: The dual-factor model of mental health in youth. School Psychology Review, 37(1), 52–68. https://doi.org/10.1080/02796015.2008.12087908

Suldo, S. M., Thalji-Raitano, A., Kiefer, S. M., & Ferron, J. M. (2016). Conceptualizing high school students’ mental health through a dual-factor model. School Psychology Review, 45(4), 434–457.
https://doi.org/10.17105/SPR45-4.434-457

Sweeney, G. M., Donovan, C. L., March, S., & Forbes, Y. (2019). Logging into therapy: Adolescent perceptions of online therapies for mental health problems. Internet Interventions, 15(1), 93–99.
https://doi.org/10.1016/j.invent.2016.12.001

von der Embse, N. (2018). The promise of school-based mental health within multi-tiered systems of support: A response to Humphrey. Psychology of Education Review, 42(1), 39–44. https://eric.ed.gov/?id=EJ1247705

Wood, B. J., & McDaniel, T. (2020). A preliminary investigation of universal mental health screening practices in schools. Children and Youth Services Review, 112, 1–5. https://doi.org/10.1016/j.childyouth.2020.104943

You, S., Furlong, M. J., Dowdy, E., Renshaw, T. L., Smith, D. C., & O’Malley, M. D. (2014). Further validation of the Social and Emotional Health Survey for high school students. Applied Research in Quality of Life, 9(4), 997–1015. https://doi.org/10.1007/z11482-013-9282-2

Zeifman, R. J., Atkey, S. K., Young, R. E., Flett, G. L., Hewitt, P. L., & Goldberg, J. O. (2015). When ideals get in the way of self-care: Perfectionism and self-stigma for seeking psychological help among high school students. Canadian Journal of School Psychology, 30(4), 273–287. https://doi.org/10.1177/0829573515594372

Arielle Bendit, PhD, LMFT, is a postdoctoral fellow at Florida Atlantic University. Melissa Mariani, PhD, is an associate professor at Florida Atlantic University. Paul Peluso, PhD, LMHC, LMFT, is Senior Associate Dean of the College of Education at Florida Atlantic University. Elisa Calabrese, EdD, is Director of Javits Gifted and Talented Education Grant for Florida Atlantic University Lab Schools. Correspondence may be addressed to Arielle Bendit, 777 Glades Road Bldg. SS-8, Rm 229, Boca Raton, Florida 33431, abendit@fau.edu.

Time Period Predicts Severity of Depression and Anxiety Symptoms Among Individuals Exposed to COVID-19: Findings From a Southeastern U.S. University

Wesley B. Webber, W. Leigh Atherton, Kelli S. Russell, Hilary J. Flint, Stephen J. Leierer

The COVID-19 pandemic and efforts to manage it have affected mental health around the world. Although early research on the COVID-19 pandemic showed a general decline in mental health after the pandemic began, mental health in later stages of the pandemic might be improving alongside other changes (e.g., availability of vaccines, return to in-person activities). The present study utilized data from a mental health service intervention for individuals at a southeastern university who were exposed to COVID-19 following the university’s return to in-person operations. This study tested whether time period (August–September 2021 vs. January–February 2022) predicted individuals’ likelihood of being mild or above in depression and anxiety ratings. Results showed that individuals were more likely to be mild or above in both depression and anxiety ratings during August–September of 2021 than January–February of 2022. Suggestions for future research and implications for professional counselors are discussed.

Keywords: COVID-19, mental health, depression, anxiety, university

     The novel coronavirus (COVID-19), first detected in 2019, spread globally at a rapid pace, with the first confirmed case in the United States occurring on January 20, 2020, in the state of Washington  (Centers for Disease Control and Prevention [CDC], 2023). By April 2020, the United States had the most reported deaths in the world due to COVID-19. It was not until December of 2020 that the first round of vaccines, authorized under emergency use authorization, was made available (Food and Drug Administration [FDA], 2021). As of October 2022 in the United States, a total of 97,063,357 cases of COVID-19 had been reported, from which there were 1,065,152 COVID-19–related deaths (CDC, 2023). A reported 111,367,843 individuals aged 5 and above in the United States had received their first booster dose of a COVID-19 vaccine as of October 2022 (CDC, 2023). Previous research has shown that the COVID-19 pandemic and efforts to manage it (e.g., lockdowns, quarantine, isolation) had negative effects on mental health in the United States and internationally (Huckins et al., 2020; Pierce et al., 2020; Son et al., 2020). Based on the extended duration of the pandemic and changes that have occurred during it (e.g., vaccine availability, lessening of initial social restrictions), more recent research has investigated possible changes in mental health in later stages of the COVID-19 pandemic (Fioravanti et al., 2022; McLeish et al., 2022; Tang et al., 2022). The present study adds to this literature by exploring whether psychosocial symptomatology (i.e., depression and anxiety) at a university in the Southeastern United States differed in individuals exposed to COVID-19 during August–September 2021 as compared to individuals exposed to COVID-19 during January–February 2022 (following the university’s return to on-campus operations in August 2021).

Challenges to Mental Health During the COVID-19 Pandemic
     Since the beginning of the COVID-19 pandemic, conceptual and empirical research has focused on ways in which the pandemic and associated stressors might impact mental health (Bzdok & Dunbar, 2020; Marroquín et al., 2020; Şimşir et al., 2022). Implementation of lockdowns to deter spread of the virus led to concerns that social isolation might have severe impacts on mental health (Bzdok & Dunbar, 2020). This hypothesis was empirically supported, as stay-at-home orders and individuals’ reported levels of social distancing were positively associated with depression and anxiety (Marroquín et al., 2020). Individuals’ views on the COVID-19 pandemic evolved quickly at the outset of the pandemic, and perceptions of risk were shown to increase during the pandemic’s first week in the United States (Wise et al., 2020). Growing awareness of the dangers of the virus likely had deleterious effects on mental health; Şimşir et al. (2022) found through a meta-analysis that fear of COVID-19 was associated with a variety of mental health problems. Mental health was also negatively affected by stigmatization associated with the COVID-19 pandemic, as was the case for those exposed to COVID-19 while at their place of work (Schubert et al., 2021). Such stigmatization associated with COVID-19 exposure was found to increase risk for depression and anxiety (Schubert et al., 2021).

The lockdowns and social distancing measures that accompanied early stages of the COVID-19 pandemic also resulted in changes to routines that likely impacted mental health. For some individuals facing lockdowns or other disruptions to typical routines, reductions in physical activity occurred. Individuals who reported greater impact of COVID-19 on their level of physical activity showed greater symptoms of depression and anxiety (Silva et al., 2022). Early in the COVID-19 pandemic, based on people’s increased time spent at home and their concerns about COVID-19 developments, some people increased their media usage (e.g., news outlets, social media). Such increases in media usage were associated with decreases in mental health (Meyer et al., 2020; Riehm et al., 2020). The COVID-19 pandemic had less significant impact on mental health for those with greater tolerance of uncertainty (Rettie & Daniels, 2021) and psychological flexibility (Dawson & Golijani-Moghaddam, 2020). Thus, some individuals were uniquely suited to face the many changes and stressors brought about by the COVID-19 pandemic.

One population that previous research has identified as being especially at risk for negative mental health outcomes during the COVID-19 pandemic is college students (Xiong et al., 2020). For college students, the COVID-19 pandemic occurred alongside other stressors known to be typical for this population such as adjusting to leaving home, navigating new peer groups, and making career decisions (Beiter et al., 2015; Liu et al., 2019). Thus, for many college students, the COVID-19 pandemic disrupted a period of life already filled with many transitions. For example, shortly after the COVID-19 pandemic began, many college students were forced to leave their dormitories and peers as universities transitioned to online delivery of classes (Copeland et al., 2021). Xiong et al. (2020) found through a systematic review that college students were especially vulnerable to negative mental health outcomes at the outset of the COVID-19 pandemic as compared to others in the general population. In the United States, college students’ reported degree of life disruption due to the COVID-19 pandemic was positively associated with depression at the conclusion of the spring 2020 semester (Stamatis et al., 2022). During fall 2020, COVID-19 concerns and previous COVID-19 infection were each found to be associated with higher levels of depression and anxiety among U.S. college students (Oh et al., 2021). Overall, previous research has supported the notion that changes associated with the COVID-19 pandemic had general negative effects on mental health in the general population and in college students specifically.

Changes in Psychosocial Symptomatology Across the COVID-19 Pandemic
     Although research has shown that the COVID-19 pandemic introduced unprecedented challenges and stressors that were associated with mental health problems, another important direction for research has been to characterize overall changes in psychosocial symptomatology as the COVID-19 pandemic progressed. Such research is important given that individuals might psychologically adapt to constant COVID-19 stressors or might benefit from changes that have occurred as the COVID-19 pandemic has progressed (e.g., vaccine availability, lessening of societal restrictions). Initial longitudinal studies comparing individuals’ symptomatology before the COVID-19 pandemic and after its beginning showed that mental health deteriorated after the COVID-19 pandemic began (Elmer et al., 2020; Huckins et al., 2020; Pierce et al., 2020). Prati and Mancini (2021) conducted a meta-analysis of 28 studies that used longitudinal or natural experimental designs and found that depression and anxiety showed small but statistically significant increases after implementation of the initial lockdowns in response to COVID-19. The various changes to ways of life associated with the COVID-19 pandemic appeared to result in a general deterioration in mental health.

Previous research has also explored possible changes in mental health beyond those that were observed in the initial phase of the COVID-19 pandemic. In support of the notion that individuals adapted to changes associated with the COVID-19 pandemic, Fancourt et al. (2021) found that anxiety and depression decreased across the initial lockdown period in the United Kingdom. In contrast, Ozamiz-Etxebarria et al. (2020) found that levels of depression and anxiety were higher 3 weeks into the initial lockdown period in Spain as compared to the beginning of the lockdown. Fioravanti et al. (2022) assessed psychological symptoms longitudinally in an Italian sample at three time points—the beginning of the COVID-19 pandemic and first lockdown (March 2020), the end of the first lockdown phase (May 2020), and during a second wave of COVID-19 with increased societal restrictions (November 2020). Their findings pointed to possible influences of COVID-19 waves and societal restrictions on specific psychosocial symptoms­. Specifically, depression, anxiety, obsessive-compulsive disorder, and post-traumatic stress disorder all decreased at the end of the first lockdown phase (Fioravanti et al., 2022). However, all symptoms besides obsessive-compulsive disorder significantly increased from the end of the first lockdown phase to the second wave of COVID-19 (Fioravanti et al., 2022).

Recent research on mental health among college students in later stages of the COVID-19 pandemic has also focused on possible mental health changes over time (McLeish et al., 2022; Tang et al., 2022). Tang et al. (2022) reported reductions in anxiety and depression in a longitudinal study of university students in the United Kingdom between a first time point (July–September 2020, after the end of lockdown) and a second time point (January–March 2021, when vaccinations were becoming available). In contrast, McLeish et al. (2022) found through a repeated cross-sectional study that depression and anxiety among students at a specific university increased from spring 2020 to fall 2020, with the increases being maintained in spring 2021. The authors noted that vaccines were not widely available at the university until the end of spring 2021 (McLeish et al., 2022). Thus, recent studies have found mixed results as to whether psychosocial symptomatology improved over time during the COVID-19 pandemic. These discrepancies may be due to contextual differences between studies (e.g., differences in data collection time periods, availability of vaccines, or levels of COVID-19 restrictions being implemented during data collection).

The Present Study
     The present study was conducted based on the need for continued research on mental health across the evolving COVID-19 pandemic and based on previous conflicting findings on possible mental health changes in later stages of the COVID-19 pandemic. Given previous research showing detrimental effects of the COVID-19 pandemic on mental health in the general population and in college students, the present study utilized data from a university population. Specifically, an archival dataset was used in the present study to examine data collected during 2021–2022 at a university in the Southeastern United States and to test whether time period would predict severity of depression and anxiety symptoms. Individuals in the study had been exposed to COVID-19 between August–September 2021 or between January–February 2022 and had requested a mental health contact during university-conducted contact tracing. These two time periods corresponded to surges in COVID-19 cases at the university due to the delta and omicron COVID-19 variants, respectively. August–September 2021 also coincided with a return to on-campus operations at the university and therefore captured psychosocial symptomatology at the beginning of a significant transition in the COVID-19 pandemic (i.e., a return to organized in-person activities on a college campus during the evolving pandemic). This study was designed to answer the following research questions:

  1. Among those requesting mental health contact after COVID-19 exposure, was the likelihood of having at least mild depression symptoms different for those whose contact occurred between August–September 2021 as compared to those whose contact occurred between January–February 2022?
  2. Among those requesting mental health contact after COVID-19 exposure, was the likelihood of having at least mild anxiety symptoms different for those whose contact occurred between August–September 2021 as compared to those whose contact occurred between January–February 2022?


Method
 

Design
     A retrospective research design was used to analyze the possible effect of time period on severity of depression and anxiety symptoms among members of a university population who had been exposed to COVID-19 and requested a mental health check-in. The study used a de-identified dataset obtained from the service providers who completed the mental health check-in. We confirmed through consultation with the IRB that the use of archival, de-identified data does not necessitate IRB review.

COVID-19 Mental Health Check-In Dataset
     The archival, de-identified dataset used in the present study was compiled as part of a mental health service occurring between February 2021 and February 2022. Participants in the dataset had tested positive for COVID-19 or been exposed to COVID-19 without a positive test. During university-conducted contact tracing, they were offered and elected to receive a subsequent mental health check-in. Individuals who were contact traced and thereby offered a mental health check-in had become known to contact tracers through one of two routes: (a) they reported their own COVID-19 diagnosis or exposure through a self-reporting mechanism as instructed by the university, or (b) they were reported by another individual as having been diagnosed with or exposed to COVID-19. The dataset used in this study included data collected during the mental health check-ins for those who elected to receive them. This data was collected over the phone and documented in RedCap (a secure web browser–based survey protocol designed for clinical research) at the time of the phone call or within 24 hours. The dataset consisted of data for 211 individuals’ check-ins. For each check-in, the dataset included participants’ demographic information, screening data (for depression, anxiety, and trauma), identified needs of the participant, resources shared with the participant, and the date of data entry.

The present study focused on check-in data for all individuals from the COVID-19 Mental Health Check-in Dataset whose check-in had occurred during one of the two time periods of focus—August–September 2021 or January–February 2022. These two time periods corresponded to surges in COVID-19 cases at the university associated with the delta and omicron COVID-19 variants, respectively. The 149 individuals who checked in during these 4 months represented 70.62% of the total number of check-ins over the 12-month dataset (N = 211), reflecting the surges in COVID-19 cases during these two periods. Of the 149 individuals in the present study, 96 (64.43%) received their check-in during August–September 2021, and 53 (35.57%) received their check-in during January–February 2022. The selection of these two time periods from the larger dataset allowed for comparison of psychosocial symptomatology during comparable levels of COVID-19 infection (i.e., surges associated with two subsequent COVID-19 variants) at comparable points in subsequent academic semesters (i.e., the first 2 months of the fall 2021 and spring 2022 semesters). The present study used only the screening data for depression and anxiety, as the scales for each of these constructs showed good internal consistency (Cronbach’s alpha > .80).

Participants
     The sample in the present study consisted of 149 individuals. The selected individuals’ ages ranged from 17 to 52 (M = 22.21, SD = 7.43). With regard to gender, 67.11% identified as female, 32.21% as male, and 0.67% as non-binary. The reported races of individuals in the study were as follows: 60.4% White, 20.13% African American, 6.71% Hispanic, 3.36% Other, 2.68% Two or more races, 1.34% Middle Eastern, 1.34% Native American, and 0.67% Asian. Some participants preferred not to indicate their race (3.36%). In responding to a question about their ethnicity, 87.25% of individuals identified as not Latinx, 9.40% identified as Latinx, and 3.36% preferred not to answer. With regard to academic level/job title, 32.89% were freshmen, 20.13% were sophomores, 14.09% were juniors, 15.44% were seniors, 7.38% were graduate students, 8.05% were faculty/staff, and 2.01% preferred not to answer. Regarding employment, 53.69% were not employed (including students), 30.20% were employed part-time, 12.75% were employed full-time, and 3.36% preferred not to answer. The relationship statuses of individuals were reported as the following: 87.92% single (never married), 4.7% married, 2.01% single but cohabitating with a significant other, 1.34% in a domestic partnership or civil union, 1.34% separated, 0.67% divorced, and 2.01% preferred not to answer. Table 1 summarizes demographic responses within each of the two time periods and for the full sample.

Measures
Demographic Questionnaire
     Participants responded to seven demographic questions (age, gender, race, ethnicity, academic year/job title, current employment status, and relationship status). They were informed that this information was optional and that they could choose not to answer particular questions.

 

Table 1
Demographic Characteristics of the Sample

 

Demographic

Characteristic

August–September 2021 January–February 2022 Full Sample
n % n % n %
Gender
   Female 69 71.88 31 58.49 100 67.11
   Male 27 28.13 21 39.62 48 32.21
   Non-binary 0 0 1 1.89 1 0.67
Race
   White 56 58.33 34 64.15 90 60.40
   African American 23 23.96 7 13.21 30 20.13
   Hispanic 8 8.33 2 3.77 10 6.71
   Other race 1 1.04 4 7.55 5 3.36
   Two or more races 4 4.17 0 0 4 2.68
   Middle Eastern 2 2.08 0 0 2 1.34
     Native American 1 1.04 1 1.89 2 1.34
     Asian 1 1.04 0 0 1 0.67
     Prefer not to answer 0 0 5 9.43 5 3.36
Ethnicity
     Not Latinx 82 85.42 48 90.57 130 87.25
     Latinx 12 12.50 2 3.77 14 9.40
     Prefer not to answer 2 2.08 3 5.66 5 3.36
Academic Year / Job Title
     Freshman 38 39.58 11 20.75 49 32.89
      Sophomore 18 18.75 12 22.64 30 20.13
      Junior 15 15.63 6 11.32 21 14.09
      Senior 15 15.63 8 15.09 23 15.44
      Graduate Student 6 6.25 5 9.43 11 7.38
      Faculty/Staff 4 4.17 8 15.09 12 8.05
      Prefer not to answer 0 0 3 5.66 3 2.01
Employment
      Not Employed (including student) 62 64.58 18 33.96 80 53.69
      Employed Part-Time 26 27.08 19 35.85 45 30.20
      Employed Full-Time 8 8.33 11 20.75 19 12.75
      Prefer not to answer 0 0 5 9.43 5 3.36
Relationship Status
      Single, never married 87 90.63 44 83.02 131 87.92
      Married 3 3.13 4 7.55 7 4.70
      Single, but cohabitating with a

significant other

2 2.08 1 1.89 3 2.01
      In a domestic partnership or civil union 2 2.08 0 0 2 1.34
      Separated 2 2.08 0 0 2 1.34
      Divorced 0 0 1 1.89 1 0.67
      Prefer not to answer 0 0 3 5.66 3 2.01
Note. Average age was 21.51 (SD = 6.98) in August–September 2021 group, 23.49 (SD = 8.11) in January–February 2022 group, and 22.21 (SD = 7.43) in the full sample.

 

 Patient Health Questionnaire-9 (PHQ-9)
     The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) is a 9-item self-report questionnaire that measures the frequency and severity of depression symptoms over the past 2 weeks. The PHQ-9 has been validated for screening for depression in the general population (Kroenke et al., 2001; Martin et al., 2006). The questionnaire measures frequency of symptoms such as “feeling down, depressed, or hopeless,” and “little interest or pleasure in doing things.” The PHQ-9 uses a 4-point Likert scale to measure frequency of symptoms over the past 2 weeks with the response options of not at all, several days, more than half the days, and nearly every day. Scores of 0, 1, 2, and 3 are assigned to each of the four response categories, and a PHQ-9 total score is derived by adding the scores for each of the nine PHQ-9 items. Minimal depression is indicated by PHQ-9 total scores of 0–4, mild depression by scores of 5–9, moderate depression by scores of 10–14, moderately severe depression by scores of 15–19, and severe depression by scores of 20–27. Question 9 on the PHQ-9 is a single screening question assessing suicide risk. Interviewers were trained in appropriate protocol in the event of a positive screen for this question. Cronbach’s alpha for the PHQ-9 in the present study was .86.

Generalized Anxiety Disorder 7-Item Scale (GAD-7)
     The Generalized Anxiety Disorder 7-Item Scale (GAD-7; Spitzer et al., 2006) is a 7-item self-report anxiety questionnaire that measures the frequency and severity of anxiety symptoms over the past 2 weeks. The GAD-7 has demonstrated reliability and validity as a measure of anxiety in the general population (Löwe et al., 2008). The questionnaire measures symptoms such as “feeling nervous, anxious, or on edge,” and “not being able to stop or control worrying.” The format of the GAD-7 is similar to the PHQ-9, using a 4-point Likert scale to measure frequency of symptoms over the past 2 weeks with response options of not at all, several days, more than half the days, and nearly every day. GAD-7 scores are calculated by assigning scores of 0, 1, 2, and 3 for response categories and then adding the scores from the 7 items to derive a total score ranging from 0 to 21. Minimal anxiety is indicated by total scores of 0–4, mild anxiety by scores of 5–9, moderate anxiety by scores of 10–14, and severe anxiety by scores of 15– 21. Cronbach’s alpha for the GAD-7 in the present study was .86.

Analytic Strategy
     Total scores for the PHQ-9 and GAD-7 were found to be positively skewed for both groups of participants. Binary logistic regression was therefore an appropriate method of analysis for this dataset, as binary logistic regression does not require normality of dependent variables (Tabachnick & Fidell, 2019). For two separate binary logistic regression models, individuals were classified as being either minimal or mild or above in depression (PHQ-9) and anxiety (GAD-7) to create binary outcome variables. This choice of cutoff allowed each model (with time period as predictor) to satisfy the recommendation of Peduzzi et al. (1996) that there be at least 10 cases per outcome per predictor in binary logistic regression.

Prior to performing these intended primary analyses to answer the research questions, preliminary analyses were conducted to determine whether adding control variables to the logistic regression models was warranted. Chi-square tests of independence, Fisher-Freeman-Halton Exact tests, Fisher’s Exact tests, and an independent samples t-test were used to test for possible differences between the two time periods in individuals’ responses to demographic questions. In cases in which responses to demographic questions were shown to be significantly different across the two groups, appropriate tests were used to determine whether the demographic responses in question were associated with either of the two intended dependent variables.

Following the preliminary analyses, the intended two binary logistic regressions were conducted to answer the research questions. In the first binary logistic regression, time period was the predictor
(1 = August–September 2021, 0 = January–February 2022) and PHQ-9 depression category was the outcome (1 = mild or above, 0 = minimal). In the second logistic regression, time period was the predictor (1 = August–September 2021, 0 = January–February 2022) and GAD-7 anxiety category was the outcome (1 = mild or above, 0 = minimal). All analyses were conducted using SPSS Version 28.

Results

Preliminary Demographic Analyses
     Prior to the primary analyses, preliminary analyses were conducted to determine whether the two groups differed in their responses to demographic questions. Fisher-Freeman-Halton Exact tests and an independent samples t-test were used to test for differences between groups in their responses to the seven demographic questions. Two of the seven tests were statistically significant at Bonferroni-corrected alpha level. Specifically, Fisher-Freeman-Halton Exact tests found significant differences between time periods on the race (p = .004) and employment (p < .001) demographic variables.

Based on the above significant results for the race and employment variables across the time periods, 2 x 2 tests were conducted to test for differences between specific race responses and specific employment responses across the two time periods. For these 2 x 2 tests, a chi-square test of independence was used when all expected cell counts were 5 or greater and Fisher’s Exact test was used when any expected cell counts were less than 5. To follow up the significant result for race, 2 x 2 tests were conducted for all pairs of race responses in which 2 x 2 tests were possible (i.e., in which there was at least one observation for each of the two race responses at both time periods). These follow-up 2 x 2 tests of responses to the race question across time periods found no statistically significant differences between pairs of race responses across time periods using Bonferroni-corrected alpha level. Follow-up 2 x 2 tests comparing all pairs of responses to the employment question across time periods found two statistically significant differences using Bonferroni-corrected alpha level. A chi-square test of independence showed that individuals were more likely to be employed full-time during January–February 2022 than August–September 2021 as compared to those not employed (including students), X2 (1, N = 99) = 9.29, p = .002. Fisher’s Exact test showed that individuals were more likely to indicate “prefer not to answer” during January–February 2022 than during August–September 2021 as compared to those indicating “not employed (including students),” p = .001.

The statistically significant tests for race and employment across time periods were followed up with additional tests to determine if depression or anxiety category (minimal vs. mild or above for each) was associated with individuals’ responses to the relevant race and employment questions. A Fisher-Freeman-Halton Exact test showed that depression category was not associated with individuals’ responses to the race question, p = .099. A Fisher-Freeman-Halton Exact test also showed that individuals’ anxiety category was not associated with individuals’ responses to the race question,
p = .386. With regard to employment, tests of association were conducted between the intended dependent variables and the specific employment responses that were found to differ between the two groups. A chi-square test of independence showed that individuals’ status as “not employed” vs. “employed full-time” was not associated with depression category, X2 (1, N = 99) = .63, p = .429. A chi-square test of independence also showed that these employment statuses were not associated with anxiety category, X2 (1, N = 99) = .27, p = .601. Similarly, Fisher’s Exact tests showed that individuals’ employment responses of “prefer not to answer” vs. “not employed (including students)” were not associated with depression category (p = .156) or anxiety category (p = .317). These results were interpreted as indicating that the ways in which individuals in the two time periods differed demographically did not have significant impact on the study’s dependent variables of interest. Therefore, binary logistic regressions were conducted with only time period as a predictor of each dependent variable.

Relationship Between Time Period and Severity of Depression Symptoms
     Most individuals in the study were in the minimal depression range on the PHQ-9 as compared to the other four categories. Figure 1 shows the percentage of individuals falling into each of the five PHQ-9 categories during each of the two time periods.

Figure 1
Percentages of Individuals Falling Into Each of the PHQ-9 Categories for Each of the Two Time Periods

Across both time periods combined (August–September 2021 and January–February 2022), 51 individuals (34.23%) were mild or above in depression while 98 (65.77%) were in the minimal range. Binary logistic regression was used to test whether time period predicted severity of depression symptoms. Time period was entered as a predictor (1 = August–September 2021, 0 = January–February 2022) of depression (1 = mild or above, 0 = minimal depression). The overall binary logistic regression model was found to be statistically significant, χ2(1) = 14.46, p < .001, Cox & Snell R2 = .092, Nagelkerke R2 = .128. In the model, time period was found to be a significant predictor of depression, Wald χ2(1) = 12.17, B = 1.52, SE = .44, p < .001. The model estimated that the odds of being mild or above in depression were 4.56 times higher during August–September 2021 than during January–February 2022 for individuals requesting a mental health check-in following COVID-19 exposure. Specifically, the predicted odds of being mild or above in depression were .81 during August–September 2021 and .18 during January–February 2022.

Relationship Between Time Period and Severity of Anxiety Symptoms
     Most individuals in the study were in the minimal anxiety range on the GAD-7 as compared to the other three categories. Figure 2 shows the percentage of individuals falling into each of the four GAD-7 categories during each of the two time periods.

Figure 2
Percentages of Individuals Falling Into Each of the GAD-7 Categories for Each of the Two Time Periods

Across both time periods combined, 40 individuals (26.85%) reported anxiety at levels of mild or above and 109 individuals (73.15%) reported minimal anxiety. Binary logistic regression was used to test whether time period predicted severity of anxiety symptoms. Time period was entered as a predictor (1 = August–September 2021, 0 = January–February 2022) of anxiety (1 = mild or above, 0 = minimal anxiety). The overall binary logistic regression model was statistically significant, χ2(1) = 6.16, p = .013, Cox & Snell R2 = .041, Nagelkerke R2 = .059. In the model, time period was a significant predictor of anxiety, Wald χ2(1) = 5.51, B = 1.03, SE = .44, p = .019. Odds of being mild or above in anxiety were estimated by the model to be 2.81 times higher during August–September 2021 than during January–February 2022 for individuals requesting a mental health check-in after exposure to COVID-19. Specifically, the predicted odds of being mild or above in anxiety were .50 during August–September 2021 and .18 during January–February 2022.

Discussion

     This study examined whether time period would predict severity of depression and anxiety symptoms in a sample of individuals exposed to COVID-19 at a university in the Southeastern United States. More specifically, the study addressed the possibility that the likelihood of being mild or above in depression and anxiety would differ between two time periods following the university’s return to in-person operations in August 2021. The results of the study showed that the likelihood of being mild or above in depression and the likelihood of being mild or above in anxiety after exposure to COVID-19 were both higher during August–September 2021 than during January–February 2022. This finding is in line with previous research that found improvements in psychosocial symptomatology in later stages of the COVID-19 pandemic (Tang et al., 2022) and in contrast to research that did not find such improvements (McLeish et al., 2022). Based on the results of the present study, it appears likely that factors that differed between the two assessed time periods (first two months of fall 2021 vs. first two months of spring 2022) contributed to the observed difference in likelihood of depression and anxiety symptoms. McLeish et al. (2022) noted that vaccines were not widely available in their study that did not find such differences, while Tang et al. (2022), who did find significant differences, noted that vaccines were available at their second data collection point (January–March 2021). For individuals in the present study, COVID-19 vaccinations were available. Vaccination was strongly encouraged by university administrators following the return to campus, and more individuals on campus were vaccinated in spring 2022 than in fall 2021. Vaccinations might have lessened individuals’ COVID-19 concerns and contributed to more positive psychosocial outcomes during spring 2022 than fall 2021.

Besides vaccinations possibly lessening depression and anxiety symptoms, other environmental circumstances might also have played a role. The two time periods on which this study focused also differed in their proximity to a significant environmental event—a return to in-person operations on the campus where the individuals studied and/or worked. Early research on the mental health impact of COVID-19 highlighted the negative mental health effects of factors such as reduced physical activity (Silva et al., 2022), life disruptions due to the COVID-19 pandemic (Stamatis et al., 2022), and social distancing (Marroquín et al., 2020). Therefore, it is possible that symptoms of depression and anxiety in spring 2022 were affected by changes in specific circumstances known to have negatively impacted mental health earlier in the COVID-19 pandemic. For example, individuals’ physical activity likely increased because of a return to campus, and they might have perceived less disruption to their lives through being able to resume in-person activities. Although individuals in the present study who were exposed to COVID-19 during the first 2 months after the return to campus might have reaped some benefits from the return to more normal environmental circumstances, they might also have faced a period of adjustment. In contrast, individuals exposed to COVID-19 between January and February 2022 might have been more readjusted and reaped greater benefits from the return to campus, thereby reducing depression and anxiety symptoms.

Implications
     This study’s findings on psychosocial symptomatology across time during the COVID-19 pandemic have important implications for the work of counselors. Based on the results of the present study, counselors planning outreach efforts to individuals exposed to COVID-19 should consider that as time passes, these individuals might be more stable with regard to symptoms of depression and anxiety. However, some individuals directly affected by COVID-19 might still be interested in receiving mental health information despite low levels of depression and anxiety. Many individuals in the present study scored as minimal in depression and anxiety but were still interested in receiving a mental health check-in. Thus, counselors should advocate for mental health information and resources to be made available to individuals who are known to be facing stressors related to COVID-19. Counselors should be prepared to have conversations to determine the contextual needs of individuals exposed to COVID-19 rather than relying only on standardized measures of psychosocial symptomatology. For example, counselors working with employees (such as university employees in the present study) should be attentive to the possibility that employees exposed to COVID-19 may be concerned about facing stigma in their workplace due to their exposure (Schubert et al., 2021).

Given that the present study focused on individuals from a university population, the study’s results also have specific implications for college counselors. College counselors should develop approaches to reach students during circumstances that might make traditional outreach challenging. For example, the present study used data from a mental health intervention in which service providers collaborated with university contact tracers to safely provide mental health resources by telephone to individuals exposed to COVID-19. College counselors should be prepared to connect clients with services at a distance. Previous research during the COVID-19 pandemic found that college students were interested in using teletherapy and online self-help resources, particularly if such services were made available for free (Ahuvia et al., 2022).

Besides preparing for flexible modes of service delivery, college counselors should be prepared to deliver interventions most likely to be useful to college students during the COVID-19 pandemic or similar pandemics. Those recently exposed to COVID-19 might benefit from discussing possible fears associated with COVID-19, experiences of stigmatization they might have experienced due to their exposure, and ways to maintain mental health during any period of quarantine or isolation that might be required. Those not recently exposed to COVID-19 might instead benefit from interventions that address other issues that might have resulted from the COVID-19 pandemic or societal responses to it. For example, if circumstances associated with the COVID-19 pandemic led to reductions in a client’s amount of exercise, a counselor can help the client identify ways they might increase their physical activity. Interventions promoting physical activity were found to reduce anxiety and depression in college students during the COVID-19 pandemic (Luo et al., 2022).

Limitations
     This study had limitations that should be considered. First, with the study being retrospective and using secondary data from a clinical intervention, it was not possible to include measures that might have better clarified mechanisms of the changes that were observed in psychosocial symptoms. Thus, the possible explanations above of what might have driven these changes are tentative and future research should test them more directly. Second, individuals in the present study were likely to have been in greater distress than the general university population based on their exposure to COVID-19, which might limit the generalizability of the study’s findings. Third, individuals in the present study were from a single university in the Southeastern United States. Thus, our findings might not generalize to other regions where university-related COVID-19 policies might have differed. Fourth, the decision to create a binary independent variable to reflect time periods (August–September 2021 and January–February 2022) in the present study also entails a limitation. This decision was justifiable on the basis that it allowed for comparisons of individuals at similar points in academic semesters and during comparable periods of COVID-19 infection. However, this analysis decision means that inferences from the study’s results are limited to the two specific time periods that were analyzed. Fifth, individuals in the present study responded to items on the GAD-7 and PHQ-9 through a phone conversation with interviewers. Interviewer-administered surveys have been previously associated with greater tendencies toward socially desirable responses than self-administered surveys (Bowling, 2005). This might limit the present study’s generalizability in contexts where self-administrations of the GAD-7 and PHQ-9 are used.

Future Research
     The results of this study provide important directions for future research. Future researchers who can conduct prospective studies or who have access to larger retrospective datasets should aim to determine specific factors that might lead to improvement in mental health outcomes over time during the COVID-19 pandemic. Knowledge produced by such studies could contribute to clinical applications in the future regarding COVID-19 or other pandemics that might occur. Relatedly, future research with larger samples of demographically diverse participants should explore possible demographic differences in specific mental health trajectories in later stages of the COVID-19 pandemic.

Future research should continue to focus specifically on those who are interested in mental health information and interventions during the COVID-19 pandemic. To follow up this study’s findings, future quantitative and qualitative studies should aim to identify which individuals are interested in receiving mental health services and determine the best ways to deliver services to them. As a globally experienced stressor, the COVID-19 pandemic might have changed some individuals’ views of mental health and/or their receptiveness to mental health outreach. More specifically, some might be more receptive to available mental health information even at lower thresholds of anxiety, depression, or other psychosocial symptoms. Such clients might be interested in preventive services or their interest in mental health information might be driven by other factors. Future studies should address these possibilities more directly than was possible in the present retrospective study.

Conclusion
     Overall, the present study provided a positive picture regarding psychosocial symptomatology in later stages of the COVID-19 pandemic. Results from this study of students and employees at a university in the Southeastern Unites States following their return to campus found that many individuals requesting mental health information after exposure to COVID-19 showed minimal levels of depression and anxiety. Individuals in the study were more likely to be in these minimal ranges during January–February 2022 than August–September 2021. COVID-19 will continue to have effects in individuals’ lives through future infections and potentially through lasting effects of previous stages of the COVID-19 pandemic. As organized in-person activities resume and COVID-19 infections continue, counseling researchers and practitioners should continue efforts to best characterize and address individuals’ mental health needs associated with the COVID-19 pandemic.

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

References

Ahuvia, I. L., Sung, J. Y., Dobias, M. L., Nelson, B. D., Richmond, L. L., London, B., & Schleider, J. L. (2022). College student interest in teletherapy and self-guided mental health supports during the COVID-19 pandemic. Journal of American College Health. Advance online publication.
https://doi.org/10.1080/07448481.2022.2062245

Beiter, R., Nash, R., McCrady, M., Rhoades, D., Linscomb, M., Clarahan, M., & Sammut, S. (2015). The prevalence and correlates of depression, anxiety, and stress in a sample of college students. Journal of Affective Disorders, 173, 90–96. https://doi.org/10.1016/j.jad.2014.10.054

Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27(3), 281–291. https://doi.org/10.1093/pubmed/fdi031

Bzdok, D., & Dunbar, R. I. M. (2020). The neurobiology of social distance. Trends in Cognitive Sciences, 24(9), 717–733. https://doi.org/10.1016/j.tics.2020.05.016

Centers for Disease Control and Prevention. (2023, March 15). COVID-19 timeline. https://www.cdc.gov/museum/timeline/covid19.html

Copeland, W. E., McGinnis, E., Bai, Y., Adams, Z., Nardone, H., Devadanam, V., Rettew, J., & Hudziak, J. J. (2021). Impact of COVID-19 pandemic on college student mental health and wellness. Journal of the American Academy of Child & Adolescent Psychiatry, 60(1), 134–141.e2. https://doi.org/10.1016/j.jaac.2020.08.466

Dawson, D. L., & Golijani-Moghaddam, N. (2020). COVID-19: Psychological flexibility, coping, mental health, and wellbeing in the UK during the pandemic. Journal of Contextual Behavioral Science, 17, 126–134.
https://doi.org/10.1016/j.jcbs.2020.07.010

Elmer, T., Mepham, K., & Stadtfeld, C. (2020). Students under lockdown: Comparisons of students’ social networks and mental health before and during the COVID-19 crisis in Switzerland. PLoS ONE, 15(7), Article e0236337. https://doi.org/10.1371/journal.pone.0236337

Fancourt, D., Steptoe, A., & Bu, F. (2021). Trajectories of anxiety and depressive symptoms during enforced isolation due to COVID-19 in England: A longitudinal observational study. The Lancet Psychiatry, 8(2), 141–149. https://doi.org/10.1016/S2215-0366(20)30482-X

Fioravanti, G., Benucci, S. B., Prostamo, A., Banchi, V., & Casale, S. (2022). Effects of the COVID-19 pandemic on psychological health in a sample of Italian adults: A three-wave longitudinal study. Psychiatry Research, 315, Article 114705. https://doi.org/10.1016/j.psychres.2022.114705

Food and Drug Administration. (2021, August 23). FDA approves first COVID-19 vaccine. https://www.fda.gov/news-events/press-announcements/fda-approves-first-covid-19-vaccine

Huckins, J. F., daSilva, A. W., Wang, W., Hedlund, E., Rogers, C., Nepal, S. K., Wu, J., Obuchi, M., Murphy, E. I., Meyer, M. L., Wagner, D. D., Holtzheimer, P. E., & Campbell, A. T. (2020). Mental health and behavior of college students during the early phases of the COVID-19 pandemic: Longitudinal smartphone and ecological momentary assessment study. Journal of Medical Internet Research, 22(6), Article e20185.
https://doi.org/10.2196/20185

Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x

Liu, C. H., Stevens, C., Wong, S. H. M., Yasui, M., & Chen, J. A. (2019). The prevalence and predictors of mental health diagnoses and suicide among U.S. college students: Implications for addressing disparities in service use. Depression and Anxiety, 36(1), 8–17. https://doi.org/10.1002/da.22830

Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., & Herzberg, P. Y. (2008). Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Medical Care, 46(3), 266–274. https://doi.org/10.1097/MLR.0b013e318160d093

Luo, Q., Zhang, P., Liu, Y., Ma, X., & Jennings, G. (2022). Intervention of physical activity for university students with anxiety and depression during the COVID-19 pandemic prevention and control period: A systematic review and meta-analysis. International Journal of Environmental Research and Public Health, 19(22), 15338. https://doi.org/10.3390/ijerph192215338

Marroquín, B., Vine, V., & Morgan, R. (2020). Mental health during the COVID-19 pandemic: Effects of stay-at-home policies, social distancing behavior, and social resources. Psychiatry Research, 293, Article 113419. https://doi.org/10.1016/j.psychres.2020.113419

Martin, A., Rief, W., Klaiberg, A., & Braehler, E. (2006). Validity of the Brief Patient Health Questionnaire Mood Scale (PHQ-9) in the general population. General Hospital Psychiatry, 28(1), 71–77.
https://doi.org/10.1016/j.genhosppsych.2005.07.003

McLeish, A. C., Walker, K. L., & Hart, J. L. (2022). Changes in internalizing symptoms and anxiety sensitivity among college students during the COVID-19 pandemic. Journal of Psychopathology and Behavioral Assessment, 44, 1021–1028. https://doi.org/10.1007/s10862-022-09990-8

Meyer, J., McDowell, C., Lansing, J., Brower, C., Smith, L., Tully, M., & Herring, M. (2020). Changes in physical activity and sedentary behavior in response to COVID-19 and their associations with mental health in 3052 US adults. International Journal of Environmental Research and Public Health, 17(18), Article 6469.
https://doi.org/10.3390/ijerph17186469

Oh, H., Marinovich, C., Rajkumar, R., Besecker, M., Zhou, S., Jacob, L., Koyanagi, A., & Smith, L. (2021). COVID-19 dimensions are related to depression and anxiety among US college students: Findings from the Healthy Minds Survey 2020. Journal of Affective Disorders, 292, 270–275. https://doi.org/10.1016/j.jad.2021.05.121

Ozamiz-Etxebarria, N., Idoiaga Mondragon, N., Dosil Santamaría, M., & Picaza Gorrotxategi, M. (2020). Psychological symptoms during the two stages of lockdown in response to the COVID-19 outbreak: An investigation in a sample of citizens in Northern Spain. Frontiers in Psychology, 11, Article 1491.
https://doi.org/10.3389/fpsyg.2020.01491

Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49(12), 1373–1379. https://doi.org/10.1016/S0895-4356(96)00236-3

Pierce, M., Hope, H., Ford, T., Hatch, S., Hotopf, M., John, A., Kontopantelis, E., Webb, R., Wessely, S., McManus, S., & Abel, K. M. (2020). Mental health before and during the COVID-19 pandemic: A longitudinal probability sample survey of the UK population. The Lancet Psychiatry, 7(10), 883–892.
https://doi.org/10.1016/S2215-0366(20)30308-4

Prati, G., & Mancini, A. D. (2021). The psychological impact of COVID-19 pandemic lockdowns: A review and meta-analysis of longitudinal studies and natural experiments. Psychological Medicine, 51(2), 201–211. https://doi.org/10.1017/S0033291721000015

Rettie, H., & Daniels, J. (2021). Coping and tolerance of uncertainty: Predictors and mediators of mental health during the COVID-19 pandemic. American Psychologist, 76(3), 427–437. https://doi.org/10.1037/amp0000710

Riehm, K. E., Holingue, C., Kalb, L. G., Bennett, D., Kapteyn, A., Jiang, Q., Veldhuis, C. B., Johnson, R. M., Fallin, M. D., Kreuter, F., Stuart, E. A., & Thrul, J. (2020). Associations between media exposure and mental distress among U.S. adults at the beginning of the COVID-19 pandemic. American Journal of Preventive Medicine, 59(5), 630–638. https://doi.org/10.1016/j.amepre.2020.06.008

Schubert, M., Ludwig, J., Freiberg, A., Hahne, T. M., Romero Starke, K., Girbig, M., Faller, G., Apfelbacher, C., von dem Knesebeck, O., & Seidler, A. (2021). Stigmatization from work-related COVID-19 exposure: A systematic review with meta-analysis. International Journal of Environmental Research and Public Health, 18(12), 6183. https://doi.org/10.3390/ijerph18126183

Silva, D. T. C., Prado, W. L., Cucato, G. G., Correia, M. A., Ritti-Dias, R. M., Lofrano-Prado, M. C., Tebar, W. R., & Christofaro, D. G. D. (2022). Impact of COVID-19 pandemic on physical activity level and screen time is associated with decreased mental health in Brazillian adults: A cross-sectional epidemiological study. Psychiatry Research, 314, Article 114657. https://doi.org/10.1016/j.psychres.2022.114657

Şimşir, Z., Koç, H., Seki, T., & Griffiths, M. D. (2022). The relationship between fear of COVID-19 and mental health problems: A meta-analysis. Death Studies, 46(3), 515–523. https://doi.org/10.1080/07481187.2021.1889097

Son, C., Hegde, S., Smith, A., Wang, X., & Sasangohar, F. (2020). Effects of COVID-19 on college students’ mental health in the United States: Interview survey study. Journal of Medical Internet Research, 22(9), Article e21279. https://doi.org/10.2196/21279

Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097.
https://doi.org/10.1001/archinte.166.10.1092

Stamatis, C. A., Broos, H. C., Hudiburgh, S. E., Dale, S. K., & Timpano, K. R. (2022). A longitudinal investigation of COVID-19 pandemic experiences and mental health among university students. British Journal of Clinical Psychology, 61(2), 385–404. https://doi.org/10.1111/bjc.12351

Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson.

Tang, N. K. Y., McEnery, K. A. M., Chandler, L., Toro, C., Walasek, L., Friend, H., Gu, S., Singh, S. P., & Meyer, C. (2022). Pandemic and student mental health: Mental health symptoms amongst university students and young adults after the first cycle of lockdown in the UK. BJPsych Open, 8(4), Article e138.
https://doi.org/10.1192/bjo.2022.523

Wise, T., Zbozinek, T. D., Michelini, G., Hagan, C. C., & Mobbs, D. (2020). Changes in risk perception and self-reported protective behaviour during the first week of the COVID-19 pandemic in the United States.
Royal Society Open Science, 7(9), Article 200742. https://doi.org/10.1098/rsos.200742

Xiong, J., Lipsitz, O., Nasri, F., Lui, L. M. W., Gill, H., Phan, L., Chen-Li, D., Iacobucci, M., Ho, R., Majeed, A., & McIntyre, R. S. (2020). Impact of COVID-19 pandemic on mental health in the general population: A systematic review. Journal of Affective Disorders, 277, 55–64. https://doi.org/10.1016/j.jad.2020.08.001

 

Wesley B. Webber, PhD, NCC, is a postdoctoral scholar at East Carolina University. W. Leigh Atherton, PhD, LCMHCS, LCAS, CCS, CRC, is an associate professor and program director at East Carolina University. Kelli S. Russell, MPH, RHEd, is a teaching assistant professor at East Carolina University. Hilary J. Flint, NCC, LCMHCA, is a clinical counselor at C&C Betterworks. Stephen J. Leierer, PhD, is a research associate at the Florida State University Career Center. Correspondence may be addressed to Wesley B. Webber, Department of Addictions and Rehabilitation Studies, Mail Stop 677, East Carolina University, 1000 East 5th Street, Greenville, NC 27858-4353, webberw21@ecu.edu.