Multicultural Competence and the Working Alliance as Predictors of Client Outcomes

Nov 13, 2018 | Volume 8 - Issue 4

Jessica Gonzalez, Sejal M. Barden, Julia Sharp

 Exploring client outcomes is a primary goal for counselors; however, gaps in empirical research exist related to the relationship between client outcomes, the working alliance, and counselor characteristics. Thus, the purpose of this investigation was to explore the relationship between the effects of multicultural competence and the working alliance on client outcomes from both client (n = 119) and counselor-in-training (n = 72) perspectives, while controlling for social desirability. Hierarchical regression results indicated counselors-in-training’s perceptions of multicultural competence and client outcome pretest scores were a significant predictor of client outcomes, after controlling for social desirability. Linear mixed effects modeling indicated significant differences in perceptions between both clients and counselors on the working alliance and multicultural competence. Findings highlight the importance of exploring what has already been working for clients before coming to counseling. Additionally, counselors are encouraged to self-reflect and explore how their clients view the relationship between the working alliance and multicultural competence.

Keywords: client outcomes, multicultural competence, working alliance, social desirability, client perspective

The past three decades of research have identified the therapeutic relationship between client and counselor as the most important predictor of change in counseling for clients (Ardito & Rabellino, 2011; Horvath & Bedi, 2002; Norcross, 2002); however, there is limited research on the associations between the working alliance and multicultural competence. Cultivating multicultural competence for counselor trainees has been the focus of considerable empirical research (Horvath & Bedi, 2002), yet the majority of studies have focused on trainees’ self-report of multicultural competence, failing to account for clients’ perceptions of trainees’ competencies (Constantine, 2001; Fuertes et al., 2006). Specifically, more research is needed exploring the influence of multicultural competence as perceived by both clients and counselors-in-training (CITs) on client outcomes (Hays & Erford, 2017; Katz & Hoyt, 2014).

Working Alliance and Client Outcomes

The working alliance is a collaborative approach that refers to the extent of agreement between clients and counselors on the goals, tasks (how to accomplish goals), and bond (development of personal bond between client and counselor) in counseling (Horvath & Greenberg, 1989). The working alliance has been identified as a key factor in positive client outcomes, despite choice of treatment modality or counseling setting (Bachelor, 2013; Baldwin, Wampold, & Imel, 2007). Considerable research has been conducted on the working alliance in relation to clients’ and CITs’ perceptions and client outcomes. Research has shown consistent similarities and differences between clients’ and counselors’ perceptions of the working alliance (Bachelor, 2013; Fitzpatrick, Iwakabe, & Stalikas, 2005; Hatcher & Barends, 1996). For example, Huppert et al. (2014) looked at the effect of counselor characteristics and the therapeutic alliance on client outcomes for clients receiving cognitive behavioral therapy for panic disorder with agoraphobia. The working alliance was measured in Sessions 3 and 9. Multilevel modeling indicated that counselors’ involvement in the alliance predicted attrition. However, client perspective of the working alliance predicted both client outcomes and attrition in counseling.

Studies such as Huppert et al. (2014) highlight the important role that the working alliance has in client outcomes in counseling. However, Drisko (2013) acknowledged that the therapeutic relationship is not the sole predictor of client outcomes and highlighted that additional factors in counseling, combined with a strong therapeutic relationship, can influence outcomes. Other common factors can include client motivation and counselor characteristics such as multicultural competence. Collins and Arthur (2010) described the working alliance as the cornerstone in the counseling process that facilitates a transformative collaborative approach in helping clients explore and understand their cultural self-awareness.

Multicultural Competence and Client Outcomes

In 1992, Sue, Arredondo, and McDavis developed the Multicultural Counseling Competencies, and in 1996 Arredondo and colleagues presented a paper outlining the Tripartite Model of Multicultural Counseling that categorized multicultural competence into three factors: awareness, knowledge, and skills. More recently, the Association for Multicultural Counseling and Development and the American Counseling Association (ACA) have endorsed a set of updated competencies, including a social justice framework entitled the Multicultural and Social Justice Counseling Competencies (MSJCC; Ratts, Singh, Nassar-McMillan, Butler, & McCullough, 2015). Research supports positive associations between clients’ perceptions of their counselors’ multicultural competence and (a) client outcomes (Owen, Leach, Wampold, & Rodolfa, 2011); (b) the counseling relationship (Fuertes & Brobst, 2002; Fuertes et al., 2006; Li & Kim, 2004; Pope-Davis et al., 2002); and (c) satisfaction with counseling (Constantine, 2002; Fuertes & Brobst, 2002). These associations show how influential clients’ perceptions of their counselors’ multicultural competence are based on a variety of aspects of the counseling process. However, the majority of studies have focused on exploring counselors’ multicultural competence from only the counselor’s perspective (Worthington, Soth-McNett, & Moreno, 2007).

Self-report multicultural measures have been criticized for being prone to participants responding in a socially desirable manner and having a tendency to measure anticipated behaviors of multicultural competence rather than actual behaviors and attitudes of multicultural competence (Constantine & Ladany, 2000; Worthington, Mobley, Franks, & Tan, 2000). In addition, counselors’ ratings of their multicultural competence can differ from ratings from an observer (e.g., supervisor; Worthington et al., 2000) or their client (Smith & Trimble, 2016). Social desirability is a response bias in which research participants attempt to make a good impression when completing research studies by answering in an overly positive manner (Crowne & Marlowe, 1960). One way researchers can minimize the potential threat of social desirability is to input a social desirability scale (Drisko, 2013) and to control for social desirability, which can improve the accuracy of the research design (McKibben & Silvia, 2016).

In addition to the majority of studies only looking at counselors’ perspectives, there is a need for further research on how CITs’ multicultural competence associates with client outcomes (D’Andrea & Heckman, 2008). For example, Soto, Smith, Griner, Rodríguez, and Bernal (2018) conducted a meta-analysis looking at how many studies have explored how client outcomes are related to their counselors’ level of multicultural competence. Only 15 studies were found that explored client outcomes and counselors’ multicultural competence. From the 15 studies, 73% appeared since 2010, including several unpublished dissertations (40%). The fact that only 15 studies were identified that met inclusion criteria for this study and were found several decades after the multicultural competencies have emerged suggests the need for further investigation on this topic (Soto et al., 2018). Two specific studies, Owen et al. (2011) and Tao, Owen, Pace, and Imel (2015), explored the relationships between multicultural competence and the counseling process. Owen and colleagues’ findings indicated a   positive association between clients’ ratings of their counselors’ multicultural competence and client outcomes. Tao and colleagues’ meta-analysis comparing the correlations and effect sizes between quantitative studies (between the years of 2002–2014) of multicultural competence and other measures of the clinical process indicated that clients ratings of their counselors’ multicultural competence accounted for 37% of the variance in the working alliance. Owen et al.’s and Tao et al.’s findings highlight the need to further explore the dynamics between clients’ and counselors’ perceptions of multicultural competence and the working alliance.

Overall, the lack of multicultural competence outcome research may be a hindrance to counselors being able to fulfill the ACA Code of Ethics because of a lack of empirical justification (D’Andrea & Heckman, 2008). In order for multicultural competence scholarship to further advance, professional counseling organizations and scholars (ACA, 2014; Bachelor, 2013; Council for Accreditation of Counseling and Related Educational Programs, 2016; Owen et al., 2011) recommend exploring how multicultural competence may influence client outcomes. Additionally, research is needed exploring the similarities and differences between clients’ and counselors’ views on the working alliance and multicultural competence. Further, in self-report counseling investigations, researchers can minimize potential threat to the study by using a social desirability scale as a control variable (Drisko, 2013; McKibben & Silvia, 2016). Thus, the purpose of this investigation was to explore the relationship between the effects of multicultural competence and the working alliance on client outcomes from both client and CIT perspectives, while controlling for social desirability.

As such, we aimed to answer three research questions: (a) Do CITs’ multicultural competence and the working alliance (as perceived by clients) predict client outcomes, while controlling for social desirability from the client’s perspective? (b) Do CITs’ multicultural competence and the working alliance (as perceived by counselors) predict client outcomes, while controlling for social desirability from the CIT’s perspective? and (c) What differences exist between clients’ and CITs’ perceptions of CITs’ multicultural competence and the working alliance, while controlling for social desirability?

Method

Participants

This investigation was conducted at a university-based community counseling research center located in the southeastern region of the United States. The primary investigator worked in the clinic in which the research study was conducted; thus, convenience sampling was used. CITs’ criteria to participate in this study was that the student had to be enrolled in their first or second semester of practicum in a master’s-level counselor education program. In addition, client criteria to participate was that they had to be an adult (over the age of 18) receiving counseling services from the CITs at the counseling research center. A total of 146 adult clients and 85 CITs participated in this study. Missing values and clients who completed the assessments more than twice were removed, yielding a response rate of 82% for clients and 84% for CITs.

Client participants self-identified as female (n = 71, 59.7%) and male (n = 48, 40.3%). The number of clients by age range was: 18–30 (n = 56, 47.1%), 31–40 (n = 27, 47.1%), 41–50 (n = 22, 18.5%), 51–60 (n = 12, 10.1%), and 61–65 (n = 2, 1.7%). Lastly, clients identified as White (n = 64, 53.8%), African American/Black (non-Hispanic, n = 21, 17.6%), Hispanic/Latino (n = 20, 16.8%), Biracial/Multiracial (n = 7, 5.9%), American Indian (n = 2, 1.7%), Asian (n = 1, 8%), and Other (n = 4, 3.4%). CIT participants self-identified as female (n = 61, 84.7%) and as male (n = 11, 15.3%). A majority of counselors were between the ages of 21–26 (n = 54, 75%), followed by 27–37 (n = 18, 25%). CITs identified as White (n = 48, 66.7%), African American/Black (non-Hispanic, n = 7, 9.7%), Hispanic/Latino (n = 7, 9.7%), Biracial/Multiracial (n = 8, 11.1%), Asian (n = 1, 1.4%), and Other (n = 1, 1.4%).

Procedure

Approval to conduct the study was obtained from the university’s institutional review board and the clinical director of the counseling research center. First, the researcher administered the consent for research during CITs’ practicum orientation and explained the purpose and voluntary nature of the study. CITs received instructions on how to administer consent for research to clients. Counselors received small tokens (a mechanical pencil and a small piece of candy) from the researcher during the practicum orientation as an incentive to complete the surveys and provide them to clients. Clinic services where the research was conducted include free counseling. Clients were already receiving free counseling services, and if they chose not to participate in this study, they would still continue to receive free counseling.

The researcher instructed CITs to provide clients with the explanation of research at the start of their first counseling session. If clients chose to participate, the CIT administered the Outcome Questionnaire 45.2 (OQ45.2; Lambert et al., 1996) assessment at the end of their first and third sessions in the counseling room. In addition, clients and CITs were instructed to complete the demographic questionnaire, the Cross-Cultural Counseling Inventory-Revised (CCCI-R; LaFromboise, Coleman, & Hernandez, 1991), the Working Alliance Inventory-Short Form (WAI-S; Horvath & Greenberg, 1989; Tracey & Kokotovic, 1989), and the Reynolds Marlowe-Crown Social Desirability Scale-Short Form A (SDS; Reynolds, 1982) after their third session was completed. Data were collected after completion of the third counseling session based on preliminary analysis on adult client retention rates at the counseling research center indicating that after the fourth counseling session, client retention rate drops by 60%. In addition, the working alliance is generally measured between the first and fifth sessions (Horvath & Bedi, 2002; Norcross, 2002).

Data were entered and then analyzed by SPSS. Prior to beginning analysis, several preliminary analyses were conducted to explore relationships among variables. Assumptions for normality, homogeneity of variance, linearity, and multicollinearity were met. To reduce the likelihood of violating the assumption of independence, clients were used as a static variable, or a variable that only has one independent observation. Utilizing static variables was important due to the possibility for the same client to have received counseling services during the two semesters in which the researcher collected the data, increasing the potential violation for the assumption of independence. Thus, if the same client had multiple ratings on assessments, they were removed from the data set, resulting in the removal of three clients. Researchers used correlation analysis, hierarchical regression, and linear mixed-effects modeling to explore their research questions.

Measures

The CCCI-R (LaFromboise et al., 1991) was used to measure client and counselor perceptions of CIT multicultural counseling competence in this investigation. The CCCI-R was developed based on the multicultural competencies defined by the Education and Training Committee of Division 17 of the American Psychological Association (Sue et al., 1982). The CCCI-R is a 20-item assessment, rated on a 6-point Likert scale intended for observer report of a counselor’s level of cultural awareness, knowledge, and skill. LaFromboise and colleagues (1991) reported an overall internal consistency coefficient alpha of .95, with an inter-item correlation between .18 and .73. Although the CCCI-R was developed to be completed by supervisors, it has been adapted for use with counselors and clients (e.g., Client: My counselor is aware of his or her own cultural heritage; Counselor: I am aware of my own cultural heritage; Fuertes et al., 2006; Owen et al., 2011). The CCCI-R is scored utilizing total scores, with higher scores indicating more perceived multicultural competence. Cronbach’s alpha results for this study were .92 for clients and .85 for CITs (Lafromboise et al., 1991).

The WAI-S (Horvath & Greenberg, 1989; Tracey & Kokotovic, 1989) was used to measure client and CIT perceptions about the strength of the working alliance relationship in counseling. The WAI-S is a 12-item assessment rated on a 7-point Likert scale ranging from 1 to 7 (1 = never to 7 = always), intended to measure the strength of the therapeutic relationship as perceived by client and counselor (e.g., Client: I am confident in my counselor’s ability to help me; Counselor: I am confident in my ability to help my client; Bachelor, 2013; Fitzpatrick et al., 2005; Hatcher & Barends, 1996). Tracey and Kokotovic (1989) indicated strong internal consistency for both the client version (α = .98) and the counselor version (α = .95) of the WAI-S. The WAI-S total score is the summation of three subscales (task, bond, and goal), with higher scores indicating a stronger therapeutic relationship. Cronbach’s alpha results for this study were .82 for clients and .81 for CITs.

The SDS (Reynolds,1982) was used to measure social desirability in this study. The SDS is a shortened version of the original Marlow Crowne Social Desirability Scale (MCSDS; Crowne & Marlow, 1960). The SDS is an 11-item dichotomous (i.e., 0 = True, 1 = False) scale designed to assess whether participants are responding truthfully in response to assessments or answering in a biased way to put forward a more socially desirable self-image (e.g., I’m always willing to admit when I make a mistake). Scoring ranges from 0–11, with a higher score indicating participant likelihood of answering in a socially desirable manner to avoid disapproval from others. Reliability for the shortened social desirability scales has been adequate (Reynolds, 1982). Cronbach’s alpha results for this study were .68 for clients and .73 for CITs. Clients’ SDS Cronbach’s alpha levels were slightly lower than the CITs’ levels; however, some authors, such as Aiken (2000), have indicated that a Cronbach’s alpha between .60 and .70 is adequate, and Streiner (2003) has indicated that the reliability on a scale of clinical samples such as the clients in this study can be different than those measured on the general population.

The OQ 45.2 (Lambert et al., 1996) contains 45 items rated on a 5-point Likert scale ranging from 0–4 (0 = almost always to 4 = never) and intended to measure clients’ distress status (e.g., I feel blue; I feel lonely). The OQ 45.2 has been used in various settings, including community clinics in a university setting similar to the one in this investigation (e.g., Wolgast, Lambert, & Puschner, 2004). The OQ 45.2 total score consists of the sum of scores of three subscales (i.e., symptomatic distress, interpersonal relationships, and social roles) and the reverse scores of nine items, with higher scores indicating more distress among clients. The total score cut off is set at 63, indicating that scores above 63 are of clinical significance (Lambert et al., 1996). Reported overall internal consistency for OQ total score (α = 93) and three subscales (α = .70) is strong (Lambert et al., 1996). Cronbach’s alpha results for this study were .82 for the OQ 45.2 pretest and .83 for the OQ 45.2 posttest.

Results

Average total scores for clients on the OQ 45.2 pretest, completed on the first session, were M = 69.37 and SD = 25.009. Average OQ 45.2 posttest scores, completed on the third session, were M = 63.73 and SD = 27.56. Average total SDS scores for clients were M = 5.74 and SD = 2.27, and average scores for CITs were M = 5.71 and SD = 2.66. Average total score of clients’ CCCI-R ratings of their CITs’ multicultural competence after completion of the third counseling session were M = 102.81 and SD = 10.42. CITs’ ratings of their own multicultural competence were M = 96.98 and SD = 7.66. Lastly, average total WAI-S scores for clients were M = 64.63 and SD = 8.0, and CITs’ scores were M = 59.40 and SD = 7.61.

A Pearson product two-tailed correlation identified four significant relationships between the variables with effect sizes ranging from small to large (Cohen, 1992). Positive relationships were indicated between clients’ perceptions of CITs’ multicultural competence and the working alliance (r =.571, p <.05), as well as CITs’ perceptions of their multicultural competence and the working alliance (r = .623, p < .05), and between the OQ 45.2 pre- and posttest scores (r = .884, p < .05). Further, a positive relationship was found between clients’ and counselors’ perceptions of the working alliance (r = .199, p < .05) and between social desirability scores on CITs’ CCCI-R responses (r = .233, p < .05); however, the effect sizes were small. The positive relationships indicate that the direction of one construct is associated with the direction of the other. For example, how a client rates their CIT’s multicultural competence is associated with the strength (high or low) of the working alliance. Lastly, a negative relationship was found between clients’ social desirability scores with both client outcome OQ 45.2 pretest scores (r = -.233, p < .05) and OQ 45.2 posttest scores (r = -.277, p < .05). This negative relationship means that higher scores on one instrument are associated with lower scores on another.

Predictors of Client Outcomes

In order to assess whether multicultural competence or the working alliance predicted client outcomes, the third-session OQ 45.2 posttest score was the dependent variable and the pretest score of the OQ 45.2 was the control variable. A hierarchical regression is used when the researcher has a theoretical basis to specify the order in which the independent variables are entered into the model (Tabachnick & Fidell, 2013). In the following analyses, social desirability and OQ 45.2 first-session scores were used as control variables. It is common practice within social sciences to use pretest scores as a control variable and posttest scores as a dependent measure in order to reduce error variance and create more powerful tests for data analysis (Tabachnick & Fidell, 2013). Also, social desirability was used as a control variable because of the relationships indicated in the correlation analysis with SDS, OQ 45.2, and CITs’ CCCI-R responses. Further, SDS scores were used as a control variable to minimize potential threat to the study (Drisko, 2013), which can improve the accuracy of the research design (McKibben & Silvia, 2016), because self-report measures have been shown to have a strong likelihood of participants responding in a socially desirable manner (DeVellis, 2003; Gall, Gall, & Borg, 2007).

Hierarchical multiple regression analysis was used to explore whether CITs’ multicultural competence (CCCI-R) and working alliance (WAI-S; as perceived by clients) predicted client outcome (OQ 45.2 pretest), while controlling for social desirability (SDS) from clients’ perspective and clients’ outcome pretest scores (OQ 45.2 posttest). Client outcome OQ 45.2 pretest scores and SDS scores were entered in the first block, explaining 78.6% [F (2, 116) = 213.3, p < .05] of the variance in client outcome OQ 45.2 posttest scores. After entry of clients’ CCCI-R and WAI-S total scores in the second block, the total variance explained by the model as a whole was 78.9%, [F (4, 114) = 106.80 p < .05]. The introduction of clients’ CCCI-R and WAI-S scores only explained an additional variance of 0.3%, after controlling for client pretest scores and social desirability [R2 change = .003, F (2, 114) = .851, p > .05]. In the final model, only one of the four predictor variables was statistically significant, client outcome pretest score (b = .859, p < .05; see Table 1). The final model indicated a large effect size (R² = .789; Cohen, 1992). Close to 79% of the variance in posttest scores was accounted for by OQ 45.2 first-session scores on client outcomes, after controlling for social desirability response.

Table 1

Hierarchical Regression Client Perspective

 Note. N = 119 clients; CCCI-R Counselor Multicultural Competence; WAI-S Working Alliance. *p < .05.
Dependent Variable: Client Outcome Posttest.

Another hierarchical multiple regression analysis was used to explore whether CITs’ multicultural competence (CCCI-R) and working alliance (WAI-S; as perceived by counselors) predicted client outcomes (OQ 45.2 pretest), while controlling for social desirability (SDS) from the CITs’ perspective (OQ 45.2 posttest). Client outcome pretest score and CITs’ SDS total scores were entered in the first block, explaining 78.1% of the variance [F (2,116) = 206.60, p < .05] in client outcome OQ 45.2 posttest scores. After entry of counselors’ CCCI-R and WAI-S total scores in the second block, the total variance explained by the model as a whole was 79.6% [F (4,114) = 111.38, p < .05]. The introduction of counselors’ CCCI-R and WAI-S scores explained an additional variance of 1.5%, after controlling for client pretest score and social desirability [R2 change = .015, F (2, 114) = 4.32, p < .05]. In the final model, two of the four predictor variables were statistically significant: client outcome pretest score (b = .894, p < .05) and counselors’ CCCI-R (b = -.157, p < .05; see Table 2). The final model indicated a large effect size (R² =.796; Cohen, 1992). In this model, 80% of the variance in posttest scores was accounted for by OQ 45.2 first session scores on client outcomes and CITs’ multicultural competence, after controlling for social desirability response.

The final research question explored the differences that exist between clients’ and counselors’ perceptions of CITs’ multicultural competence and the working alliance, while controlling for social desirability. In order to resolve the possibility of non-independence in this data set (West, Welch, & Galecki, 2007), a linear mixed-effects model was used to compare clients and counselors (fixed effect) for the dependent variables of multicultural competence and the working alliance. Thus, accounting for client observations nested within counselors (i.e., some CITs had several clients). There was a significant difference between counselor and client perceptions of CITs’ multicultural competence while controlling for social desirability: [F (1,174.38) = 30.43, p < 0.05]. The average CCCI-R score for clients was 5.91 more than the average for CITs, after controlling for social desirability. Similarly, there was a significant difference between counselor and client perceptions of the working alliance (WAI-S): [F (1, 176.20) = 79.98, p < 0.05]. The average WAI-S score for clients was 9.85 more than the average for CITs, controlling for social desirability. Thus, clients rated CITs’ multicultural competence and the working alliance higher than CITs rated themselves.

Table 2

Hierarchical Regression Counselor Perspective

Note. N = 72 clients; CCCI-R Counselor Multicultural Competence; WAI-S Working Alliance. *p <.05.
Dependent Variable: Client Outcome Posttest.

Discussion

The aim of this investigation was to explore the relationship between client outcomes, counselors’ multicultural competence, the working alliance, and social desirability from both clients’ and CITs’ perspectives. Hierarchical regression results indicated that clients’ perspectives of their CITs’ multicultural competence and the working alliance did not predict client outcomes, although CITs’ perceptions of their multicultural competence did, modestly, after controlling for counselors’ social desirability scores. In a related investigation, Owen et al. (2011) compared differences in perceptions of counselors’ multicultural competence between clients and CITs. Results from their intra-class correlation (ICC) analysis indicated that CITs’ perceptions accounted for 8.5% (ICC = .085) of the variance in client outcomes, although clients’ perceptions of CITs’ multicultural competence were not related to clients’ counseling outcomes, which is consistent with the findings from this investigation. In contrast, results from this investigation on the working alliance and lack of predictive ability on client outcomes are incongruent with previous research that indicates a strong association between the working alliance and client outcomes (Horvath, Del Re, Flückiger, & Symonds, 2011; Norcross, 2011). Although results from one hierarchical regression did not indicate significant predictability of the working alliance on client outcomes, a Pearson product correlation conducted before regression analysis supported the positive associations between clients’ perceptions of CITs’ multicultural competence and the working alliance, as well as CITs’ perceptions of their multicultural competence and the working alliance. Further, correlational results indicated a small association between clients’ and CITs’ perceptions of the working alliance, and between CITs’ social desirability scores and CCCI-R responses.

Potential explanations for some of the insignificant findings in this investigation include the cross-sectional research design on the constructs of multicultural competence and the working alliance. In a cross-sectional research design, the researcher looks at a snapshot of constructs at one point in time (Gall et al., 2007). In this investigation, multicultural competence and the working alliance were assessed after the third session for both clients and counselors. Thus, assessing multicultural competence and the working alliance after the third session may not have been enough time for clients to evaluate their counseling relationship or their CITs’ multicultural competence. For example, Fitzpatrick et al. (2005) explored clients’ perceptions of the working alliance utilizing the WAI-S over three phases of counseling (e.g., early: 2–4 sessions; middle: midpoint; late: fourth, third, or second to last). Fitzpatrick and colleagues (2005) conducted a MANOVA with two within-subject design factors. The two factors were phases of counseling (i.e., early, middle, late) and WAI subscales (i.e., task, bond, goal). Results indicated as a whole, client-rated alliance increased over time. Therefore, results of this analysis may have been different if multicultural competence and the working alliance were measured over time.

Linear modeling results indicated significant differences between client and CIT perceptions of the working alliance and counselors’ multicultural competence after controlling for social desirability. In addition, upon inspection of the mean scores between clients and CITs, clients rated their CITs’ multicultural competence and the working alliance higher than CITs rated their multicultural competence and the working alliance. Similar to this investigation, Depue, Lambie, Liu, and Gonzalez (2016) found significant differences on client and CIT ratings of the working alliance, with clients rating the working alliance higher than counselors. Contrastingly, Fuertes and colleagues (2006) found no significant differences between the working alliance for clients or CITs and significant differences between perceptions of counselors’ multicultural competence, with CITs’ ratings being higher than clients, highlighting mixed research findings.

A factor that may influence the perceptions of clients and CITs is the way clients and counselors would define counseling terms. First, clients and CITs may differ in their definition of what a quality therapeutic relationship or what a culturally responsive CIT looks like. For example, counselors may view the strength of the therapeutic relationship based on client progress (Bachelor & Horvath, 1999), while clients may view the quality of the relationship based on how much unconditional positive regard they sense from their counselors (Norcross, 2011). Similarly, with multicultural competence, Pope-Davis et al. (2002) suggested that clients may not perceive multicultural competence in the same way as counselors. A common theme found in Pope-Davis et al.’s (2002) qualitative investigation on client perceptions of culturally relevant components in counseling indicated that the need for integration of culture in counseling was only relevant if the client self-identified their culture as a core value in their life. On the other hand, counselors may view their level of multicultural competence based on how much knowledge they have about their clients’ cultures.

Second, counselors’ level of experience might influence the way they rate themselves. For example, novice counselors, such as the participants in this investigation, often have anxiety that can negatively influence their beliefs about their counseling performance (Rønnestad & Skovholt, 2003; Stoltenberg & McNeill, 2010). Barden and Greene (2015) explored the relationship between counselor education students’ levels of self-reported multicultural counseling competence and multicultural counseling self-efficacy, with results indicating that students who had been in graduate education longer had higher self-reported multicultural counseling competence and higher levels of multicultural knowledge, highlighting a potential explanation for lower multicultural competence ratings in the current investigation.

Implications for Counselors

In this investigation, results highlighted that clients and CITs perceive the working alliance and counselors’ multicultural competence differently. Counselors might want to give assessments such as the CCCI-R (LaFromboise et al., 1991) or the WAI-S (Tracey & Kokotovic, 1989) in session to facilitate discussions with clients. For example, if counselors see that their client strongly disagrees with the CCCI-R assessment question 20, “My counselor acknowledges and is comfortable with cultural differences,” counselors can utilize this as a discussion point to address any cultural differences that may be interfering with the counseling process. Furthermore, in this study, positive relationships were shown between clients’ and counselors’ perceptions of counselors’ multicultural competence and the working alliance. Given these associations, counselors are encouraged to self-reflect and explore how their clients view the relationship between the working alliance and multicultural competence. Slone and Owen (2015) explored the relationship between the effects of the therapeutic relationship, counselors’ level of comfort in session, and the systematic alliance on client outcomes between counselors and clients. Multilevel model analysis revealed that client outcome improved when counselors checked in with clients about how the therapeutic relationship was going, when counselors had a high comfort level in session, and when clients had perceived interpersonal networks that aligned with the goals and tasks in counseling. Thus, counselors are encouraged to check in with clients about their views at multiple times throughout the counseling process. For example, CITs can ask clients probing points early on to promote discussion on the working alliance and multicultural competence, such as, “What are you looking for in a counseling relationship?” or “Please tell me a little bit about your culture.” Moreover, counselors can check in with a client mid-session and ask, “How has our counseling relationship been going?” or “What would improve our counseling relationship?”

This study also highlighted the importance of exploring what has already been working for clients before coming to counseling. The therapeutic relationship has been shown to have the most explained variance in client outcomes (Norcross, 2011; Wampold & Imel, 2015); however, in this investigation, it was found that 80% of the variance in client outcomes after the third session was predetermined. Given that close to 80% of the variance in posttest scores were accounted for by OQ 45.2 first-session scores on client outcomes after controlling for social desirability responses, counselors are encouraged to explore what coping strategies clients are already using that have been helpful with their clients’ presenting issues during the first session. In addition, counselor educators can consider that three weeks of counseling may not be enough time to show clinically significant change in client outcomes. Furthermore, three weeks in counseling may not be enough time to show how the working alliance and CITs’ multicultural competence may influence client outcomes. Lastly, given that there was a positive relationship between CITs’ social desirability scores and their ratings of their multicultural competence, counselor educators who supervise CITs are encouraged to explore their supervisees’ expectations and comfort in discussing developing multicultural competence.

Limitations and Suggestions for Future Research

The first limitation is that the multicultural competence and working alliance assessments were collected in a cross-sectional manner, limiting the results to a singular time point. Second, the generalizability to populations other than novice counselors or clients within a university setting is low. Third, at the time data collection for this investigation was completed, there was not a validated formative assessment developed to explore the updated social justice framework based on the new MSJCC competencies, so the instrument used was based on the Multicultural Competence Tripartite Model. Despite the limitations from this investigation, the use of a social desirability scale, an emphasis on both clients’ and CITs’ perceptions, and the study’s implications contribute to the empirical research on multicultural competence and the working alliance.

There are several implications for future research that are suggested from this study. First, researchers can conduct a longitudinal design and increase data collection points for assessing client outcome (e.g., first, fifth, tenth, and fifteenth sessions) to determine if and when clinically significant change in client outcomes occurs. Second, further exploration is needed of the perceptions of counselors who have completed their training programs to see how results may differ. Third, researchers are encouraged to develop a formative assessment tool to explore the new MSJCCs (Ratts et al., 2015) and replicate a similar study. Researchers are encouraged to explore, from the clients’ perspectives, how their counselors are implementing multicultural and social justice competencies. Fourth, investigators can implement a mixed method design (e.g., qualitative and quantitative) to explore factors that influence client outcomes for brief therapy. Utilizing a qualitative component may help counselors and counselor educators gain insight into what clients perceive a culturally sensitive counselor to be or what a strong working alliance looks like. Lastly, counselor educators can continue to investigate how social desirability, if at all, influences participants’ responses on counseling assessments.

Conflict of Interest and Funding Disclosure

The authors reported no conflict of interest

or funding contributions for the development

of this manuscript.

References

Aiken, L. R. (2000). Psychological testing and assessment (10th ed.). Boston, MA: Allyn & Bacon.

American Counseling Association. (2014). ACA code of ethics. Alexandria, VA: Author.

Ardito, R. B., & Rabellino, D. (2011). Therapeutic alliance and outcome of psychotherapy: Historical excursus, measurements, and prospects for research. Frontiers in Psychology, 2, 270. doi:10.3389/fpsyg.2011.00270

Arredondo, P., Toporek, R., Brown, S. P., Jones, J., Locke, D. C., Sanchez, J., & Stadler, H. (1996). Operationalization of the multicultural counseling competencies. Journal of Multicultural Counseling and Development, 24, 42–78. doi:10.1002/j.2161-1912.1996.tb00288.x

Bachelor, A. (2013). Clients’ and therapists’ views of the therapeutic alliance: Similarities, differences and relationship to therapy outcome. Clinical Psychology & Psychotherapy, 20(2), 118–135. doi:10.1002/cpp.792

Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M. A. Hubble, B. L. Duncan, & S. D. Miller (Eds.), The heart and soul of change: What works in therapy (pp. 133–178). Washington, DC: American Psychological Association.

Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75, 842–852. doi:10.1037/0022-006X.75.6.842

Barden, S. M., & Greene, J. H. (2015). An investigation of multicultural counseling competence and multicultural counseling self-efficacy for counselors-in-training. International Journal for the Advancement of Counselling, 37, 41–53. doi:10.1007/s10447-014-9224-1

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.

Collins, S., & Arthur, N. (2010). Culture-infused counselling: A model for developing multicultural competence. Counselling Psychology Quarterly, 23, 217–233. doi:10.1080/09515071003798212

Constantine, M. G. (2001). Predictors of observer ratings of multicultural counseling competence in Black, Latino, and White American trainees. Journal of Counseling Psychology, 48, 456–462.
doi:10.1037/0022-0167.48.4.456

Constantine, M. G. (2002). Predictors of satisfaction with counseling: Racial and ethnic minority clients’ attitudes toward counseling and ratings of their counselors’ general and multicultural counseling competence. Journal of Counseling Psychology, 49, 255–263. doi:10.1037/0022-0167.49.2.255

Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology, 47, 155–164. doi:10.1037/0022-0167.47.2.155

Council for Accreditation of Counseling and Related Educational Programs. (2016). CACREP Standards. Retrieved from http://www.cacrep.org/for-programs/2016-cacrep-standards/

Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology.
Journal of Consulting Psychology, 24, 349–354. doi:10.1037/h0047358

D’Andrea, M., & Heckman, E. F. (2008). A 40-year review of multicultural counseling outcome research: Outlining a future research agenda for the multicultural counseling movement. Journal of Counseling & Development, 86, 356–363. doi:10.1002/j.1556-6678.2008.tb00520.x

DePue, M. K., Lambie, G. W., Liu, R., & Gonzalez, J. (2016). Investigating supervisory relationships and therapeutic alliances using structural equation modeling. Counselor Education and Supervision, 55, 263–277. doi:10.1002/ceas.12053

DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage.

Drisko, J. (2013). The common factors model: Its place in clinical practice and research. Smith College Studies in Social Work, 83, 398–413. doi:10.1080/00377317.2013.833435

Fitzpatrick, M. R., Iwakabe, S., & Stalikas, A. (2005). Perspective divergence in the working alliance. Psychotherapy Research, 15(1–2), 69–80. doi:10.1080/10503300512331327056

Fuertes, J. N., & Brobst, K. (2002). Clients’ ratings of counselor multicultural competency. Cultural Diversity and Ethnic Minority Psychology, 8, 214–223. doi:10.1037/1099-9809.8.3.214

Fuertes, J. N., Stracuzzi, T. I., Bennett, J., Scheinholz, J., Mislowack, A., Mindy, H., & Cheng, D. (2006). Therapist multicultural competency: A study of therapy dyads. Psychotherapy: Theory, Research, Practice, Training, 43, 480–490. doi:10.1037/0033-3204.43.4.480

Gall, M. D., Gall J. P., & Borg, W. R. (2007). Educational research: An introduction (8th ed.). Boston, MA: Allyn & Bacon.

Hatcher, R. L., & Barends, A. W. (1996). Patients’ view of the alliance in psychotherapy: Exploratory factor analysis of three alliance measures. Journal of Consulting and Clinical Psychology, 64, 1326–1336. doi:10.1037/0022-006X.64.6.1326

Hays, D. G., & Erford, B. T. (2017). Developing multicultural counseling competence: A systems approach (3rd ed.). New York, NY: Pearson.

Horvath, A. O., & Bedi, R. P. (2002). The alliance. In J. C. Norcross (Ed.), Psychotherapy relationships that work: Therapist contributions and responsiveness to patients (pp. 37–70). New York, NY: Oxford University Press.

Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy: Theory, Research, Practice, Training, 48, 9–16. doi:10.1037/a0022186

Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223–233. doi:10.1037/0022-0167.36.2.223

Huppert, J. D., Kivity, Y., Barlow, D. H., Gorman, J. M., Shear, M. K., & Woods, S. W. (2014). Therapist effects
and the outcome–alliance correlation in cognitive behavioral therapy for panic disorder with
agoraphobia. Behaviour Research and Therapy, 52, 26–34. doi:10.1016/j.brat.2013.11.001

Katz, A. D., & Hoyt, W. T. (2014). The influence of multicultural counseling competence and anti-Black prejudice on therapists’ outcome expectancies. Journal of Counseling Psychology, 61, 299–305.
doi:10.1037/a0036134

LaFromboise, T. D., Coleman, H. L. K., & Hernandez, A. (1991). Development and factor structure of the Cross-Cultural Counseling Inventory—Revised. Professional Psychology: Research and Practice, 22, 380–388. doi:10.1037/0735-7028.22.5.380

Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Vermeersch, D. A., Clouse, G. C., & Yanchar, S. C. (1996). The reliability and validity of the Outcome Questionnaire. Clinical Psychology & Psychotherapy, 3, 249–258. doi:10.1002/(SICI)1099-0879(199612)3:4<249::AID-CPP106>3.0.CO;2-S

Li, L. C., & Kim, B. S. (2004). Effects of counseling style and client adherence to Asian cultural values on counseling process with Asian American college students. Journal of Counseling Psychology, 51(2), 158-167. doi:10.1037/0022-0167.51.2.158}

McKibben, W. B., & Silvia, P. J. (2016). Inattentive and socially desirable responding: Addressing subtle threats to validity in quantitative counseling research. Counseling Outcome Research and Evaluation, 7, 53–64. doi:10.1177/2150137815613135

Norcross, J. C. (Ed.). (2002). Psychotherapy relationships that work: Therapist contributions and responsiveness to patients. New York, NY: Oxford University Press.

Norcross, J. C. (Ed.). (2011). Psychotherapy relationships that work: Evidence-based responsiveness (2nd ed.). New York, NY: Oxford University Press.

Owen, J., Leach, M. M., Wampold, B., & Rodolfa, E. (2011). Client and therapist variability in clients’ perceptions of their therapists’ multicultural competencies. Journal of Counseling Psychology, 58, 1–9. doi:10.1037/a0021496

Pope-Davis, D. B., Toporek, R. L., Ortega-Villalobos, L., Ligiéro, D. P., Brittan-Powell, C. S., Liu, W. M., . . . & Liang, C. T. H. (2002). Client perspectives of multicultural counseling competence: A qualitative examination. The Counseling Psychologist, 30, 355–393. doi:10.1177/0011000002303001

Ratts, M. J., Singh, A. A., Nassar-McMillan, S., Butler, S. K., & McCullough, J. R. (2015). Multicultural and social justice counseling competencies. Retrieved from www.counseling.org/knowledge-center/competencies

Reynolds, W. M. (1982). Development of reliable and valid short forms of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology, 38, 119–25.
doi:10.1002/1097-4679(198201)38:1<119::AID-JCLP2270380118>3.0.CO;2-I

Rønnestad, M. H., & Skovholt, T. M. (2003). The journey of the counselor and therapist: Research findings and perspectives on professional development. Journal of Career Development, 30, 5–44. doi:10.1023/A:1025173508081

Slone, N. C., & Owen, J. (2015). Therapist alliance activity, therapist comfort, and systematic alliance on individual psychotherapy outcome. Journal of Psychotherapy Integration, 25, 275–288.
doi:10.1037/a0039562

Smith, T. B., & Trimble, J. E. (2016). Foundations of multicultural psychology: Research to inform effective practice. Washington, DC: American Psychological Association.

Soto, A., Smith, T. B., Griner, D., Rodríguez, M. D., & Bernal, G. (2018). Cultural adaptations and therapist multicultural competence: Two meta-analytic reviews. Journal of Clinical Psychology, 1–17.
doi:10.1002/jclp.22679

Stoltenberg, C. D., & McNeill, B. W. (2010). IDM supervision: An integrative developmental model for supervising counselors and therapists (3rd ed.). New York, NY: Routledge.

Streiner, D. L., (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99–103. doi:10.1207/S15327752JPA8001_18

Sue, D. W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Counseling & Development, 70, 477–486.
doi:10.1002/j.1556-6676.1992.tb01642.x

Sue, D. W., Bernier, J. E., Durran, A., Feinberg, L., Pedersen, P., Smith, E. J., & Vasquez-Nuttall, E. (1982). Position paper: Cross-cultural counseling competencies. The Counseling Psychologist, 10(2), 45–52. doi:10.1177/0011000082102008

Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston, MA: Allyn & Bacon.

Tao, K. W., Owen, J., Pace, B. T., & Imel, Z. E. (2015). A meta-analysis of multicultural competencies and psychotherapy process and outcome. Journal of Counseling Psychology, 62, 337–350.
doi:10.1037/cou0000086

Tracey, J. T., & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 1, 207–210. doi:10.1037/1040-3590.1.3.207

Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York, NY: Routledge.

West, B. T., Welch, K. E., & Galecki, A. T. (2007). Linear mixed models: A practical guide using statistical software. Boca Raton, FL: Chapman-Hall/CRC.

Wolgast, B. M., Lambert, M. J., & Puschner, B. (2004). The dose-response relationship at a college counseling center. Journal of College Student Psychotherapy, 18(2), 15–29. doi:10.1300/J035v18n02_03

Worthington, R. L., Mobley, M., Franks, R. P., & Tan, J. A. (2000). Multicultural counseling competencies: Verbal content, counselor attributions, and social desirability. Journal of Counseling Psychology, 47, 460–468. doi:10.1037/0022-0167.47.4.460

Worthington, R. L., Soth-McNett, A. M., & Moreno, M. V. (2007). Multicultural counseling competencies research: A 20-year content analysis. Journal of Counseling Psychology, 54, 351–361.
doi:10.1037/0022-0167.54.4.351