A Qualitative Analysis of Ableist Microaggressions

Jennifer M. Cook, Melissa D. Deroche, Lee Za Ong

The phenomenon of microaggressions is well established within the counseling literature, particularly as it relates to race, ethnicity, gender, and affectual orientation. However, research related to disability or ableist microaggressions is still in its infancy, so counseling professionals have limited information about experiences of disability and ableist microaggressions. The purpose of this qualitative content analysis was to describe participants’ self-reported experiences with ableist microaggressions. Participants (N = 90) had a diagnosed disability and the majority (91.11%) identified as having two or more nondominant identities beyond their disability. We report two categories and 10 themes. While participants were part of the general population, we position our discussion and implications within the context of professional counseling to increase counseling professionals’ awareness and knowledge so counselors can avoid ableist microaggressions and provide affirmative counseling services to persons with disabilities.

Keywords: disability, ableist microaggressions, professional counseling, nondominant identities, affirmative counseling

Day by day, what you choose, what you think, and what you do is who you become.
—Heraclitus, pre-Socratic philosopher

     Each person is a complex makeup of dominant and nondominant sociocultural identities. Individuals with dominant cultural identities (e.g., able-bodied, White, middle social class) experience societal privilege, have more sociocultural influence, and have unencumbered access to resources. People with nondominant identities, including people with disabilities (PWD), people of color, and people in lower social class, frequently have less influence and experience structural and interpersonal inequities, limitations, and discrimination (Sue & Spanierman, 2020). As such, people with nondominant identities often experience microaggressions. Microaggressions are unintentional or deliberate verbal, nonverbal, and/or environmental messages that convey disapproval, distaste, and condemnation of an individual based on their nondominant identity (Sue et al., 2007).

Professional counselors are aware and knowledgeable that their identity constellation and their
experiences with microaggressions, as well as those of their clients, impact their worldviews, experiences, and—importantly—the counseling relationship (Ratts et al., 2016). While microaggressions associated with several cultural identities have been well-researched within counseling (e.g., race, ethnicity, gender, affectual orientation), others, like ableist microaggressions, have been examined far less frequently (Deroche et al., 2024). The purpose of this article is to describe the microaggression experiences that PWD (N = 90) encounter. Our intention is to increase counseling professionals’ awareness and knowledge about ableist microaggressions so they can examine their beliefs about disability, identify how they may have participated in ableist microaggressions and, ultimately, provide affirmative counseling services to PWD.

Literature Review

Although the term microaggressions was coined by Pierce in the 1970s, it was not until 2007 that it took hold within the allied helping professions (Sue et al., 2007). Initially, the term was used to describe experiences based on race, yet the term has been applied more broadly to the dismissive experiences people with other nondominant identities (e.g., gender, affectual/sexual orientation) encounter (Sue & Spanierman, 2020). In 2010, Keller and Galgay initiated foundational research about the microaggressions that PWD experience. Through their qualitative study, they identified eight microaggression domains experienced by PWD and described their harmful effects on the psychological and emotional well-being of PWD. Those eight domains are: (a) denial of identity, (b) denial of privacy, (c) helplessness, (d) secondary gain, (e) spread effect, (f) patronization, (g) second-class citizenship, and (h) desexualization (i.e., ignoring or avoiding the sexual needs, wants, or desires of PWD). This study marked the beginning of ableist microaggressions research that led scholars not only to naming (e.g., Dávila, 2015) and measuring (e.g., Conover et al., 2017a) specific microaggressions toward PWD, but also describing experiences with ableist microaggressions within specific disability groups (e.g., Coalson et al., 2022; Eisenman et al., 2020) and exploring the impact for specific cultural groups of PWD (e.g., Miller & Smith, 2021).

Before continuing further, it is important for us to explain our use of the term ableist microaggressions, rather than the term disability microaggressions, because it deviates from the typical convention used to name microaggressions (e.g., racial microaggressions, gender microaggressions). While some authors have used the term disability microaggressions (e.g., Dávila, 2015), we believe that this term undercuts and minimizes PWD’s microaggression experiences, as it fails to explicitly communicate that these microaggressions are forms of ableism. Therefore, to validate PWD’s experiences and to align with the disability movement’s philosophy of diversity and social justice, we use the term ableist microaggressions (Perrin, 2019).

The qualitative ableist microaggression studies we reviewed all utilized and endorsed the themes Keller and Galgay (2010) found in their qualitative study, while adding nuance and new information about ableist microaggressions. For instance, Olkin et al.’s (2019) focus group research with women who had both hidden and apparent disabilities (N = 30) supported Keller and Galgay’s eight themes while identifying two others: medical professionals not believing PWD’s symptoms and experiences of having their disability discounted based on appearing young and/or healthy. Similarly, Coalson et al. (2022), who utilized focus groups with adults who stutter (N = 7), endorsed six of Keller and Galgay’s themes and identified participants’ perceptions of microaggressive behaviors (i.e., Exonerated the Listener, Benefit of the Doubt, Focusing on Benefits, and Aggression Viewed as Microaggression) while noting that some participants had minimal or no microaggression experiences.

Although Eisenman et al. (2020) endorsed five of Keller and Galgay’s (2010) themes, they took a different approach to how they analyzed and organized their findings by using Sue et al.’s (2007) microaggression taxonomy. Of note, these researchers were the first to identify and establish microaffirmations within disability microaggressions research. According to Rolón-Dow and Davison (2018) microaffirmations are:

behaviors, verbal remarks or environmental cues experienced by individuals from minoritized racial groups in the course of their everyday lives that affirm their racial identities, acknowledge their racialized realities, resist racism or advance cultural and ideological norms of racial justice. (p. 1)

Like microaggressions, microaffirmations may be intentional or unintentional, but they have a positive rather than a negative impact on people with nondominant racial identities. Eisenman et al. (2020) found all four race-related microaffirmation types identified by Rolón-Dow and Davison (2021)—Microrecognitions, Microvalidations, Microtransformations, and Microprotections—with their sample of people with intellectual disabilities.

Finally, Miller and Smith (2021) conducted individual interviews (N = 25) with undergraduate and graduate students who identified as members of the LGBTQ community with a disability. They, too, found Keller and Galgay’s (2010) domains present in their study and identified eight additional categories. Five categories captured cultural components in addition to disability (i.e., Biphobia, Intersectionality Microaggression, Queer Passing/Disclosure, Racism, and Sexism), while the remaining three were specific to ableist microaggression–focused data: Ableism Avoidance, Faculty Accommodations, and Structural Ableism/Inaccessibility.

The purpose of our study is to add to the burgeoning disability and microaggressions discourse by analyzing participants’ responses to a qualitative prompt offered to them after they completed the Ableist Microaggression Scale (AMS; Conover et al., 2017b). We corroborate prior research findings while adding novel findings that increase professional knowledge about ableist microaggressions and their impact.

Methodology

To ensure compliance with Section 508 of the Rehabilitation Act, the federal law that requires PWD to have access to electronic information equivalent to that available to nondisabled individuals, we utilized digital accessibility tools on the internet platform used for this study (Qualtrics) and recruited PWD to test the accessibility of the study survey and questions. The data analyzed and reported in this article were part of a larger, IRB-approved study (N = 201) in which we investigated participants’ ableist microaggression experiences quantitatively using the AMS (Conover et al., 2017b) to uncover whether participants’ AMS scores were impacted by visibility of disability, type of disability, and their other nondominant identities (Deroche et al., 2024). After participants completed the survey, they were invited to provide a written response to the open-ended question: “What, if any, information do you think would be helpful for us to know about your personal experiences regarding ableist microaggressions?” Ninety participants (44.77% of the overall sample) responded with rich data that warranted analysis and reporting in an independent article. Because the open-ended question occurred after participants completed the AMS, we agreed that the survey likely influenced their responses, so we chose to conduct a content analysis using an a priori codebook grounded in the AMS subscales (Minimization, Denial of Personhood, Otherization, and Helplessness; Conover et al., 2017b), with additional coding categories for data that did not fit the a priori codes (i.e., Fortitude/Resilience/Coping, Contextual Factors, Impact of Microaggressions/Ableism on Mental Health/Wellness, Microaggression Experiences Are Different Depending on Visibility of Disability, Internalized Ableism, and Microaggressions Include Identities Other Than Disability).

Procedure
     Using online data collection via Qualtrics survey, we recruited participants nationally by contacting disability organizations, listservs, social media, and professional contacts who work with organizations that serve PWD. The recruitment included a description of the research; inclusion criteria; and a confidential, anonymized survey link. The survey was Section 508–compliant and optimized to be taken on a computer or mobile device. Data were collected over a 3-month period.

Inclusion Criteria and Participants
     To participate in the study, individuals (a) were at least 18 years of age, (b) had earned a high school diploma or GED, and (c) had a diagnosed disability. Under the Americans with Disabilities Act (ADA), the term disability is defined within the context of a person’s significant limitations to engage in major life activities. Different agencies and organizations such as the World Health Organization and the U.S. Social Security Administration define disability differently (Patel & Brown, 2017). For this study, we categorized disability as (a) physical disability (i.e., mobility-related disability), (b) sensory disability (i.e., seeing- or hearing-related disability), (c) psychiatric/mental disability (e.g., bipolar disorder, depression, post-traumatic stress disorder), or (d) neurodevelopmental disability (e.g., autism spectrum disorder, learning disability, or ADHD). Participants’ disabilities were apparent/visible (i.e., recognizable by others without the person disclosing they have a disability) or hidden (i.e., others are unlikely to know the individual has a disability, so the person must disclose they have a disability for it to be known), and they could identify with one or more disability categories listed above. Ninety individuals provided usable responses. Table 1 details participant demographics. The bulk of the sample, 84.43%, identified as having two (36.66%), three (26.66%), or four (21.11%) nondominant cultural identities out of the six identities the study targeted, while the rest of the sample comprised individuals who noted six (n = 2; 2.22%), five (n = 4; 4.4%), one (n = 7; 7.77%), or no (n = 1; 1.11%) nondominant identities.

Of note, a higher percentage of participants with hidden or both apparent and hidden disabilities participated in the qualitative portion of the study compared to those who completed only the quantitative portion (45.5% compared to 41.8% and 33.3% compared to 27.4%, respectively). Similarly, there was a lower response rate from participants who earned a high school diploma or GED (5.6%), completed an associate degree or trade school (7.8%), completed some college (7.8%), or earned a doctoral degree (10%). There was an increase in responses from participants who earned a bachelor’s degree (26.7% compared to 21.9% in the quantitative portion) or a master’s degree (42.2% compared to 35.8%, respectively).

Data Analysis
     We analyzed data for this study using MacQueen et al.’s (1998) framework to create a codebook to promote coder consistency. We established six codes, four of which were definitionally congruent with the AMS subscales (i.e., Helplessness, Minimization, Denial of Personhood, and Otherization; Conover et al., 2017a). While we used Conover et al.’s definitions as the foundation, we utilized Keller and Galgay’s (2010) definitions to add additional nuance. The next code, Other Data, was an a priori code reserved for data that did not fit the AMS subscale codes. After completing the pilot, we added a sixth code, Fortitude/Resilience/Coping, to capture data that demonstrated ways in which participants developed strengths, dealt with adversity and microaggressions, and persevered despite their microaggressive experiences. Identifying PWD’s fortitude/resilience/coping abilities is indicative of a strengths-based framework that promotes inclusion, equity, and higher quality of life. Research has shown that resilience in PWD such as improved well-being, higher social role satisfaction, and lower mental health symptoms are correlated with positive psychological and employment outcomes (Ordway et al., 2020; Norwood et al., 2022).  Once this code was established, the Other Data code was used for any data that did not fit the five a priori codes. After the pilot, we added to the codebook definitions for clarity—though no codes were changed. All codes we established had substantial representation in the data and are reported as themes in the results section. The auditor (second author Melissa D. Deroche) gave feedback on the codebook and confirmed the codebook was sound prior to analysis.

Table 1
Demographic Characteristics of Participants (N = 90)

Variable        n          %
Disability Type
Single Type: Physical 21 23.33
Single Type: Sensory 17 18.88
Single Type: Neurodevelopmental 6 6.66
Single Type: Psychiatric/Mental Health 6 6.66
Combination (2): Physical and Psychiatric/Mental Health 8 8.88
Combination (2): Neurodevelopmental and Psychiatric/Mental Health 6 6.66
Combination (2): Sensory and Psychiatric/Mental Health 5 5.55
Combination (2): Sensory and Physical 4 4.44
Combination (2): Neurodevelopmental and Physical 2 2.22
Combination (2): Sensory and Neurodevelopmental 2 2.22
Combination (3): Physical, Psychiatric/Mental Health, Neurodevelopmental 4 4.44
Combination (3): Physical, Sensory, Neurodevelopmental 4 4.44
Combination (3): Sensory, Psychiatric/Mental Health, Neurodevelopmental 2 2.22
Combination (3): Physical, Sensory, Psychiatric/Mental Health 1 1.11
Combination (4): Physical, Sensory, Psychiatric/Mental Health,

Neurodevelopmental

2 2.22
Visibility of Disability
Visible/Apparent 19 21.11
Hidden/Concealed 41 45.55
Both 30 33.33
Biological Sex/Sex Assigned at Birth
Female 74 82.22
Male 16 17.77
Gender Identity
Gender Fluid/Gender Queer 6 6.66
Man 16 17.77
Woman 68 75.55
Affectual/Sexual Orientation
Asexual 2 2.22
Bisexual 9 10.00
Gay 2 2.22
Heterosexual 68 75.55
Lesbian 3 3.33
Pansexual 4 4.44
Queer 1 1.11
Questioning 1 1.11
Racial/Ethnic Identity
African American/Black 4 4.44
Asian or Pacific Islander 3 3.33
Biracial 2 2.22
Euro-American/White 69 76.66
Indigenous 1 1.11
Jewish 5 5.55
Latino/a or Hispanic 3 3.33
Middle Eastern 1 1.11
Multiracial 2 2.22
Religious/Spiritual Identity
Atheist 8 8.88
Catholic 12 13.3
Jewish 4 4.44
Not Religious 1 1.11
Pagan 1 1.11
Protestant 36 40.00
Questioning 2 2.22
Spiritual Not Religious 5 5.55
Unitarian Universalist 2 2.22
Self-Identify in Another Way 19 21.11
Highest Level of Education
High School Diploma or GED 5 5.55
Associate or Trade School Degree 7 7.77
Some College, No Degree 7 7.77
Bachelor’s Degree 24 26.66
Master’s Degree 38 42.22
PhD, EdD, JD, MD, etc. 9 10.00
No Response 1 1.11
Employment Status
Full-Time 40 44.44
Part-Time 16 17.77
Retired 9 10.00
Student 11 12.22
Unemployed 14 15.55
Employment Compared to Training and Skills
Training/Education/Skills are lower than job responsibilities/position 2 2.22
Training/Education/Skills are on par with job responsibilities/position 42 46.66
Training/Education/Skills exceed job responsibilities/position 24 26.66
Not applicable 22 24.44

We began analysis by piloting 10% of the data (n = 9) using the initial codebook (Boyatzis, 1998). Two researchers (first and third authors Jennifer M. Cook and Lee Za Ong) coded data independently and then worked together to reach consensus. Once the pilot analysis was complete, we coded the remaining data and recoded pilot data to ensure they fit the revised coding frame. After all data were coded, we further coded the data that were assigned to Other Data using in vivo codes to establish codes that best captured the data. We identified five codes within Other Data: Contextual Factors, Impact of Microaggressions/Ableism on Mental Health/Wellness, Microaggression Experiences Are Different Depending on Visibility of Disability, Internalized Ableism, and Microaggressions Include Identities Other Than Disability. 

Trustworthiness
     Cook and Ong coded all data independently and then met to reach consensus. Prior to coding commencement, we identified our beliefs and potential biases about the data and discussed how they might impact coding; we continued these conversations throughout analysis. For the pilot coding phase, independent coder agreement prior to consensus was 40%. Independent coder agreement prior to consensus during regular coding was 56%, and 69% for Other Data independent coding. We reached consensus for all coded data through a team meetings consensus process (Boyatzis, 1998). Finally, we utilized an auditor (Deroche). Deroche reviewed all consensus findings during all analysis stages. The coding team met with the auditor to resolve questions and discrepancies, such as a few instances in which data were misassigned to a code.

Research Team
     The research team comprised three cisgender women between the ages of 45 and 55 who are all licensed professional counselors and work as counselor educators. Cook and Deroche identify as White and hold PhDs in counselor education, while Ong holds a PhD in rehabilitation psychology and is Asian American of Chinese descent and an immigrant from Malaysia. Deroche identifies as a person with a disability, Deroche and Ong have worked extensively with PWD, and all three authors have conducted research about PWD. Cook has abundant publications in qualitative research designs related to multicultural counseling. Finally, all three authors have extensive research training and experience in qualitative and quantitative research designs.

Findings

The findings described below are organized into two categories: findings that align with the AMS subscales and unique findings that are independent of the AMS subscales. Themes are listed in their appropriate category with participants’ quotes to illustrate and substantiate each theme (see Table 2). When we provide participant quotes, we refer to them by their randomly assigned participant numbers (e.g., P105, P109).

Table 2
Categories and Themes

Category/Theme      n % of Sample
Category 1: Findings That Align With the AMS Subscales
Minimization 35           38.88
Denial of Personhood 26           28.88
Otherization 17           18.88
Helplessness 16           17.77
Category 2: Unique Findings Independent of the AMS Subscales
Fortitude/Resilience/Coping 27           30.00
Contextual Factors 17           18.88
Impact of Microaggressions/Ableism on Mental Health/Wellness 10             9.00
Microaggression Experiences Are Different Depending on Visibility of Disability 6             6.66
Internalized Ableism 4             4.44
Microaggressions Include Identities Other Than Disability 4             4.44

Note. N = 90.

Category 1: Findings That Align With the AMS Subscales
     Our analysis revealed that the AMS a priori codes fit the study data. As such, the codes were transitioned to themes: Minimization (n = 35), Denial of Personhood (n = 26), Otherization (n = 17), and Helplessness (n = 16). The quotes selected for each theme illustrate the lived experiences of the theme definitions and add context and nuance about the impact of ableist microaggressions.

Minimization
     Conover et al. (2017a) defined Minimization as microaggression experiences demonstrating the belief that PWD are “overstating their impairment or needs” and that “individuals with a disability could be able-bodied if they wanted to be or that they are actually able-bodied” (p. 581). Thirty-five of the 90 participants’ responses (33.33%) indicated instances of Minimization.

For example, P105 described incidents from their formative years that highlight the belief that PWD are, in fact, able-bodied and overstating their impairment:

As a child, children and adults alike would test the limits of my blindness. My piers [sic] would ask me how many fingers they were holding up. And in one instance, teachers lined a hallway with chairs to see if I’d run into them. Spoiler alert, I did.

P109 spoke to their interactions with family that highlight how disbelief about a person’s disability can result in Minimization:

Family is really bad. They still don’t believe me. I was asked (when I couldn’t climb stairs into a restaurant) are you trying to make a point? My visible disability has gotten worse over 40 years. I think because they saw me before I started using a cane, they just won’t believe me.

P158 illuminated a fallacy that can result in Minimization: “Because my disability is invisible people assume I need no help, [and] when I do, they discount my disability. I hear, ‘you don’t look like you have a disability‚’ ‘don’t sell yourself short.’”

Finally, P137 spoke to the blame that underlies Minimization:

On[e] of the most frequent microaggressions encountered living with my particular invisible disability (type 1 diabetes) is the ableist idea that health is entirely a personal responsibility. There is this assumption that whatever problems we face with our health are a direct result of poor choices (dietary, financial) completely ignoring the systematic problems with for-profit health care in this country.

Denial of Personhood
     Denial of Personhood is characterized by PWD being “treated with the assumption that a physical disability indicates decreased mental capacity and therefore, being reduced to one’s physicality” (Conover et al., 2017a, p. 581); such microaggressions can occur “when any aspect of a person’s identity other than disability is ignored or denied” (Keller & Galgay, 2010, p. 249). Twenty-six participants (28.88%) endorsed this theme. For example, P142 described their experiences in the workplace that illustrate the erroneous belief that PWD have diminished mental capacity: “All my life I was pushed out of jobs for not hearing. People would actually tell me, ‘if you can’t hear—how can you do anything’ even though all my performance reviews exceeded expectations.” P123 spoke to a similar sentiment: “[I] am often asked ‘what’s wrong with you?’ ‘how did you get through college?’” Finally, P173 summarized the belief that seemingly underlies Denial of Personhood microaggressions and issued a corrective action:

Disabled doesn’t mean stupid. We can figure out most things for ourselves and if we can’t we know to ask for help. Don’t tell us how to live our lives or say we don’t deserve love, happiness and children. If you don’t know the level of someone’s disability you shouldn’t have the right to judge them about such things.

Otherization
     Seventeen participants (18.88%) described Otherization as part of their narrative responses. Otherization microaggressions are those in which PWD are “treated as abnormal, an oddity, or nonhuman, and imply people with disabilities are or should be outside the natural order” (Conover et al., 2017a, p. 581) and that their “rights to equality are denied” (Keller & Galgay, 2010, p. 249). Participants shared several examples of these types of microaggressions. For instance, P140 shared:

When we (PWD) ask for simple things (e.g., can you turn on the captioning) and people grumble, say they can’t, etc. it just reinforces that we’re not on equal footing and at least for me it eats away a little bit every time.

P185 indicated another manifestation of Otherization: “As a deaf person, I get frustrated when whoever I’m talking to stops listening when someone else (non-deaf person) speaks verbally, leaving me mid-sentence.” P108 shared that they have been “prayed for in public without asking,” while P106 expressed, “I hate when people compliment me on how well I push my chair or say I must have super strong arms. I just have normal arms not athletic looking or anything.” 

Helplessness
     Helplessness microaggressions are those in which PWD are “treated as if they are incapable, useless, dependent, or broken, and imply they are unable to perform any activity without assistance” (Conover et al., 2017a, p. 581). Sixteen participants (17.77%) described Helplessness microaggressions. For P174, the most common Helplessness microaggression they experience is when “people speak to the person I am with instead of to me. Drives me crazy! Worse is when the person I’m with answers for me.” P126 corroborated the “devastating” nature of when “people make decisions for you.” P129 shared that, “As a person with an invisible disability, I most often encounter microaggressions in the form of unsolicited advice when I disclose my disability.” Similarly, P134 noted:

Although my disability is not apparent, if people know about it, they often just act on my behalf without asking me for input or feedback. That is very frustrating and often does not change even if I bring it up to the individual who does it.

This final quote from P134 is powerful because it, like P174’s experience, demonstrates how people without disabilities participate in perpetuating ableism even when they were not the ones who initiated it.

Category 2: Unique Findings Independent of the AMS Subscales
     As we indicated earlier, we separated data that did not fit into AMS codes and coded them using in vivo codes. This analysis resulted in six novel themes (i.e., Fortitude/Resilience/Coping, Contextual Factors, Impact of Microaggressions/Ableism on Mental Health/Wellness, Microaggression Experiences Are Different Depending on Visibility of Disability, Internalized Ableism, and Microaggressions Include Identities Other Than Disability) that are independent from the AMS-driven themes discussed in the prior section, yet are interrelated because they add unique insights and helpful context for understanding ableist microaggressions within the lived experiences of PWD.

Fortitude/Resilience/Coping
     We defined Fortitude/Resilience/Coping as ways in which participants have developed strength, dealt with adversity/microaggressions, and persevered despite their microaggressive experiences. Thirty percent (n = 27) of participants disclosed a wide range of attitudinal and experiential tactics related to this theme. P103 shared, “I maintain what I call a healthy sense of humor about my own body and being disabled,” while P145 demonstrated a sense of humor as they shared how they cope:

I just have to remind them and myself that my brain works differently and that I am just as competent as anyone else. I have learned not to beat myself up when I forget something or can’t get my paperwork done correctly for the tenth time. (I really hate paperwork.)

Participants 138 and 127 both spoke directly to the role knowledge plays. P138 shared:

I want to put out there that knowledge & understanding are power. Knowing & understanding your rights as a person with a disability as well as knowing & understanding your unique experience with your own disability (to the best of your ability) is key to making forward strides in environments that can often times feel ableist.

P127 spoke to knowledge, too, with their belief that “most microaggressions stem from a lack of education. I am often the first person they have met with a disability and the experience makes them uncomfortable.”

Finally, P187 spoke to the power of their resilience and its impact on their life, experiences which they draw from to help others:

I’ve been physically and emotionally abused my entire life, until I took control and stopped it. I’m middle aged and it took me 40 years to forgive everything that I’ve . . . had to endure. Never from my family, or close friends, but it’s been a difficult life, and now I’m all ok with it and try to help others with disabilities that are having a hard time.

Contextual Factors
     Seventeen participants (18.88%) described Contextual Factors, which are data that depict relational, situational, or environmental elements that impact participants’ experiences of ableist microaggressions.

P110 shared thatmicroaggressions can be hard to label because they can vary based on the relationship you have with the person.” P175 added: “Most times the microaggression I receive are by people when they don’t know me, or first meet me, as opposed to get to know me better.” P162 spoke to additional situational/relational nuances: “I have very different experiences depending on what assistive technology I’m using in a given space (basically to what degree I pass as able-bodied) and how people know me.”

P163 spoke to relational roles as well as environmental context: “The attitudes about me are distinctly divided between the power structures. A case manager, medical doctor, neighbor or family member will certainly show their attitude differently. The same goes for academic settings [versus] job placement.” For P152, “The worst comments have come from mental health therapists [who] are medical professionals who should be the most compassionate towards their patients.”

P117 and P131 both identified situational differences they have noticed. P117 shared, “I find that people have treated me differently at different ages and stages in my life, particularly when I was raising three children as a divorced mom.” P131 identified their work environment as positive: “I work in the field of vocational rehabilitation so [I] interact with more people who have a more nuanced understanding of disability than the general population.” However, P165 offered an alternate view, noting that “many microaggressions are more insidious or come from within the disabled community.”

Impact of Microaggressions/Ableism on Mental Health/Wellness
     Ten participants (9%) expressed how microaggressions and ableism experiences have impacted their mental health and wellness. P172 stated, “I struggle with my mental wellness and I have been hospitalized for severe depression that manifests from a combination of my disability and situations that are overwhelming.” P157 expressed a similar combination effect of having a disability and being “ostracized” by others: “The combination is very heavy on my heart and leaves me feeling incredibly alone.”

P159 expressed feeling “pathetic and weak. Sometimes I feel useless and disgraced. Most of the time I feel dumb and stupid.” P103 added additional impacts while acknowledging the differences between their experiences and those of their colleagues of color: “None of these [microaggressions] were overt, but all contributed to stress and frustration and generalized anxiety. I have seen much worse with coworkers of color and disabled Black and Brown folks in my community.”

P126 admitted that completing the study survey “evoked difficult memories.” Additionally, this participant described the turmoil and cognitive dissonance they experience:

I’m reminded taking this survey of the inner conflict with identifying as disabled. Is my disability qualifying enough, will I be rejected? I felt hints of defensiveness emerge, like imposter syndrome. I also recognize that I desire to be abled and that keeps the conflict churning.

Microaggression Experiences Are Different Depending on Visibility of Disability
     Six participants (6.6%) spoke to how individuals with hidden disabilities experience microaggressions differently than individuals with visible/apparent disabilities. P141 asserted that “because my disabilities are hidden, I don’t hear many microaggressions regarding me,” and P183 corroborated that microaggressions are “different the more severe and obvious the disabilities are.”

P146 suggested that “invisible disabilities offer up a whole different category of microaggressions than those with visible disabilities,” and P151 added that “hidden disabilities is [sic] a double edged sword,” highlighting both the privilege and the dismissiveness hidden disabilities can bring. P150 emphasized the privilege of others not knowing about their disability: “In some ways, this benefits me because I’m not associated with the stigma of a disability.”

Internalized Ableism
     A small number of participants (n = 4) expressed comments that were consistent with Internalized Ableism. Internalized Ableism includes believing the stereotypes, myths, and misconceptions about PWD, such as the notion that all disabilities are visible and that PWD cannot live independently, and it can manifest as beliefs about their own disability or others’ disabilities. One manifestation of Internalized Ableism is when a PWD expresses that another’s disability is not real or true compared to their own disability. For example, P112 stated:Every time I go out I have great difficulty finding available accessible parking. I watch & people using the spots are walking/functioning just fine. Sick of hearing about ‘hidden disability.’ I think the majority are inconsiderate lazy people.”

Another manifestation of Internalized Ableism can be when PWD deny the existence of ableist microaggressions. P183 shared:

I don’t think that most people have microaggressions toward PWD. Maybe that’s different the more severe and obvious the disabilities are. It tends to be older people like 60s or 70s that treat me differently period it seems like the younger generation just sees most of us as people not disabled people. And I also think the term ableist separates PWD and people without. If we don’t want to be labeled, we shouldn’t label them.

Microaggressions Include Identities Other Than Disability
     For this final theme, four participants (4.44%) spoke to the complexity related to microaggressions when a PWD has additional nondominant cultural identities. P167 expressed the compounding effect: “I have multiple minoritized identities—the intersection leads to more biases.” P161 articulated the inherent confusion when one has multiple nondominant identities: “I do not know whether I am treated in the ways I indicated because of my disabilities or because I am a person of color.” These quotes highlight the inherent increase and subsequent impact on PWD who have more than one nondominant cultural identity.

Discussion

The purpose of our analysis was to illuminate participants’ lived experiences with ableist microaggressions that were important to them. We revealed contextual information about participants’ experiences that aligned with the AMS subscales (i.e., Minimization, Denial of Personhood, Otherization, and Helplessness). Although prior qualitative ableist microaggression studies (e.g., Coalson et al., 2022; Eisenman et al., 2020; Olkin et al., 2019) grounded their research in Keller and Galgay’s (2010) eight categories rather than in Conover et al.’s (2017a) four subscales, it is fair to say that our findings substantiate other researchers’ findings because Conover et al.’s four subscales were devised based on Keller and Galgay’s findings.

While the corroboration of prior research findings based on the AMS subscales is illustrative and essential, the crucial findings from this study lie in the unique themes that arose from the in vivo coding process (i.e., Fortitude/Resilience/Coping, Contextual Factors, Impact of Microaggressions/Ableism on Mental Health/Wellness, Microaggression Experiences Are Different Depending on Visibility of Disability, Internalized Ableism, and Microaggressions Include Identities Other Than Disability). These themes introduce both novel and less-explored aspects of disability and of ableist microaggressions.

Fortitude/Resilience/Coping is a unique theme. Participants described how they became stronger and persevered despite microaggressive experiences. Eisenman et al. (2020) were the first to identify microaffirmations within ableist microaggressions research and Coalson et al. (2022) found that their participants perceived benefits that came from microaggressive experiences; both are important contributions. However, both instances of seeming positives related to ableist microaggressions in these studies are framed within the context of how others acted toward PWD rather than the autonomous choices and personal development of the person with the disability in the face of adversity. Our findings demonstrate PWD’s abilities—both innate qualities and learned skills—that rendered life-giving fortitude, resilience, and coping in which they are personally empowered and persevere despite external stimuli; they are not dependent upon whether others act appropriately. This is a key finding for counselors because they have the ability to create a therapeutic environment in which PWD can process, develop, and refine their fortitude, resilience, and coping further, acknowledging that PWD have these skills already.

Unsurprisingly, some participants spoke to the impact of ableist microaggressions and ableism on their mental health and wellness; these impacts included depression, loneliness, stress, frustration, and feeling “pathetic and weak.” What was surprising is that only 9% of the sample spoke to this impact directly, given how well-documented the harmful mental health effects of microaggressions are (Sue & Spanierman, 2020). This seeming underrepresentation of mental health ramifications amongst participants led us to wonder, based on the high percentage of participants (30%) who endorsed Fortitude/Resilience/Coping, whether this specific sample had a uniquely high ability to cope with adversity as compared to the overall disability population or if it is possible that ableist microaggression experiences have begun to decrease. While we are unable to answer these questions directly as part of this study, we posit three considerations: (a) microaggressions continue to have a negative effect on some PWD and need to be screened for and attended to within the counseling process; (b) screening for and helping clients with disabilities name, develop, or refine coping, fortitude, and resilience can prove beneficial; and (c) it is worthwhile to continue to work to reduce microaggressive behaviors in every way possible.

Although we had an independent theme in which participants indicated the differences between apparent and hidden disabilities, the participant quotes within every theme illustrate these differences as well. For instance, within the Minimization theme, P137 highlighted that those with hidden disabilities may be told that “personal responsibility” is the cause of their disability, while P105 and P109 spoke to having to “prove” their apparent disability to others, including family. Having to prove one’s disability or not being believed tracks with several other researchers’ findings including Olkin et al. (2019), who found that medical professionals did not believe PWD’s symptoms and experiences. The Helplessness theme revealed differences such as P129 receiving unsolicited advice once people learn of their hidden disability; however, this theme revealed similarities, too. Participants with both apparent and hidden disabilities experienced others acting on their behalf without their consent.

The Microaggression Experiences Are Different Depending on Visibility of Disability theme may explain why a higher percentage of participants with hidden disabilities or those who have both hidden and apparent disabilities participated in the qualitative portion of the study than those with apparent disabilities, which was the higher percentage in the quantitative part of the study. By definition, microaggressions can leave those who experience them questioning whether what they experienced was real, and this could be compounded when PWD have hidden disabilities; these participants may have needed to express their experiences more than those with apparent disabilities. While our data demonstrate that having a hidden disability may be a protective factor from experiencing ableist microaggressions, their disability experience often can be overlooked or ignored, resulting in a form of minimization that is both congruent with and distinct from the Minimization subscale definition.

Participants made a case for how Contextual Factors, defined as relational, situational, and/or environmental components, impact microaggression experiences. Implicitly, several authors spoke to what we have named as Contextual Factors (e.g., Coalson et al., 2022; Eisenman et al., 2020; Miller & Smith, 2021), yet the specificity and nuance participants provided in this study warranted a distinct theme. Relationally, participants noted that whether the perpetrator knew them and if there was a relational power differential between them and the perpetrator (e.g., doctors or counselors vs. family member or neighbor) makes a difference. Damningly, P152 stated that “the worst comments” they have received “have come from mental health therapists.” Participants noted, too, that work environments, life stage, the type of assistive technology they are using at the time, and being part of the disability community can all be impactful in both affirming and deleterious ways. It is imperative that counselors assess and understand thoroughly each client’s specific contextual factors so they can identify ways in which clients have internal and external resources and support, as well as areas in which they may want strategies, support, resources, and, potentially, advocacy intervention.

A small number of participants (n = 4) spoke to Internalized Ableism. Although this was a less robust theme, it was important to report because it adds to professional knowledge about what some clients with disabilities might believe and express during counseling sessions. We defined Internalized Ableism as participants expressing stereotypes, myths, and misconceptions about PWD that can manifest as beliefs about their own disability or the disabilities of others. One participant expressed disdain for hidden disabilities and expressed disbelief about others’ needs to use parking for disabled persons, while another participant questioned whether most PWD experience ableist microaggressions. While our study findings are not congruent with these statements, counselors must take clients’ expressions seriously, work to understand how clients have developed these beliefs, and seek to understand their impact on the client who is stating them.

Finally, four participants indicated that Microaggressions Include Identities Other Than Disability. Given the high percentage of the sample that had multiple nondominant identities, it is curious that so few participants spoke to this phenomenon. However, we theorize that this may have to do with identity salience (Hunt et al., 2006) and the fact that this was a study about ableist microaggressions. For the participants who spoke to this theme, the important features they reported were the compounding effect of microaggressions when one has multiple nondominant identities and the inherent confusion that results from microaggressive experiences, particularly when one has multiple nondominant identities. Again, counselors must screen for and be prepared to address the complexity and the impact of ableist microaggressions based on each client’s unique identities and experiences.

Implications for Practice
     The study findings illustrate the ubiquitous, troubling, and impactful nature of ableist microaggressions. These findings expose many counselors, supervisors, and educators to a world they may not know well or at all, while for others, these findings validate experiences they know all too well personally and professionally. We began this article with a quote from the pre-Socratic philosopher Heraclitus: “Day by day, what you choose, what you think, and what you do is who you become.” This quote captures the charge we are issuing to counseling professionals: It is time to take action to become counseling professionals who think as, act as, and are disability-affirming professionals. The task at hand is for each counseling professional to decide what steps to take next to strengthen their disability-affirming identity based on their current awareness, knowledge, and skill level, as well as how they can enact their disability-affirming identity based on their professional roles.

Fundamentally, disability-affirming professionals validate, support, encourage, and advocate for and with PWD consistently throughout their professional activities. For many, this begins with developing their awareness and knowledge, followed later by their skills. Based on the findings presented in this article, we suggest counseling professionals engage in self-reflexivity by examining the ways in which they have unwittingly adopted the dominant discourses about disability, what they believe about the abilities and lives of PWD, how they understand disability within the context of other nondominant identities, and the ways in which they have participated in perpetuating ableist microaggressions. Without engaging in disability self-awareness development, professionals risk conveying ableist microaggressions to clients that can result in early termination, impede the therapeutic relationship, and/or inflict additional psychological harm (Sue & Spanierman, 2020). For example, counselors may assume that clients with disabilities have diminished social–emotional learning skills compared to clients without disabilities and initiate formalized assessment based on this assumption. While counselors should be attuned to all clients’ social–emotional skills, it can be damaging to PWD’s sense of self and the counseling relationship to assume their social–emotional learning skills are deficient rather than assessing how environments are not conducive to PWD’s social–emotional needs (Lindsay et al., 2023).

Counselors’ self-reflexive process is meant to foster self-awareness; to better equip counselors to recognize ableist microaggressions in clients’ stories when they occur in personal, training, and professional environments; and for them to avoid unintentionally communicating ableist microaggressions in their practice. To start this process, we encourage counselors to question whether any of the study findings rang true, whether as someone who has experienced ableist microaggressions or as one who has perpetrated them, and to ascertain whether their attitudes and beliefs about PWD differ based on the visibility of disability. Additionally, we proffer that counselors who engage in self-reflective activities, such as the ones mentioned above, and those who learn more about PWD’s lives and experiences are more apt to create a plan to work through any negative attitudes or biases they have and, in turn, refine their skills so they are more disability-affirming in their practice. Counselors who engage in these processes will benefit those they serve, whether clients, students, or supervisees.

This study represents only a slice of the microaggression experiences of PWD. We concur with Rivas and Hill (2023) that counselors must adopt an evolving commitment to develop disability counseling effectiveness. Ways that counselors can take steps toward developing their disability-affirmative counselor identity and effectiveness include familiarizing themselves with and applying the American Rehabilitation Counseling Association (ARCA) disability competencies (Chapin et al., 2018); reading additional studies (e.g., Olkin et al., 2019; Peters et al., 2017); listening to podcasts (e.g., Swenor & Reed, n.d.); reading blogs and books (e.g., Heumann & Joiner, 2020); and watching shows and movies that highlight PWD’s experiences, microaggressive and otherwise—PWD are telling their stories and want others to learn from them.

Within the relational context, no matter one’s professional roles, it is important to be prepared to attend to the interaction of identity constellations within professional relationships and the power dynamics that are present (Ratts et al., 2016). Broaching these topics initially, including ability status and similarities and differences with our experiences, is a helpful start; however, this is the beginning of the process, not the entire process. Accordingly, clinicians must continually assess PWD’s contextual factors and their impact, lived experiences of their multiple identities, resilience, fortitude, and coping skills. To do so, clinicians must first create space for clients to process their microaggression experiences through actively listening to their stories; allowing PWD to openly express their frustrations, anger, or other emotions; and validating their experiences using advanced empathy. In other words, it is critical not to dismiss such topics nor unilaterally make them the presenting problem—balance is needed to attend to microaggression experiences appropriately. Essentially, counselors need to guide clients to discern the impact and to identify what they need rather than doing it for them, and to be ready, willing, and able to advocate with and on behalf of clients. All advocacy actions must be discussed with clients so as to center their autonomy.

Clients’ resiliencies and strengths must be fostered unceasingly. It is not uncommon for clients who have experienced ableist microaggressions to feel diminished and worthless and to question their purpose. Counselors must prioritize assisting clients in naming their strengths and telling stories about how they have developed resiliencies, and they must encourage clients to draw on both when facing adversity—particularly ableist microaggressions. While the goal is to eradicate ableist microaggressions, we must reinforce with clients that they are armed with tools to safeguard against ableist microaggressions’ impact and that they can seek trusted support when they need it.

As we move forward into the future as disability-affirming counseling professionals, counselor educators and supervisors have a specific charge to include disability status and disability/ableist microaggressions as part of their professional endeavors when working with students and supervisees. For many, the aforementioned recommendations likely apply because they, too, did not receive education about disability and disability microaggressions (Deroche et al., 2020). This is a setback, but not a limitation. Counselor educators and supervisors are continual learners who seek additional awareness, knowledge, skills, and advocacy actions to positively impact their work with counselors-in-training. Webinars, disability-specific conference sessions, and engaging with community disability organizations are helpful ways to start, and we recommend counselor educators and supervisors engage in the same self-examination strategies mentioned above to begin combating any biases they may hold about PWD. More specifically, counselor educators and supervisors can introduce and teach the ARCA disability competencies to trainees and supervisees, deliberately integrate self-exploration activities regarding disability into coursework, direct trainees and supervisees to inquire about ability status in intake and assessment procedures, and use cultural broaching behaviors to model appropriate use with clients (Deroche et al., 2020).

Limitations and Future Research

There are important limitations to consider to contextualize the study findings. The data used in this analysis were the result of one open-ended prompt as part of a larger quantitative study. Although participants offered robust and illustrative responses, it is a significant limitation that no follow-up questions were asked. Additionally, because the study utilized the AMS (Conover et al., 2017b), we analyzed data using the AMS subscales. While this was an appropriate choice given the context, it limited our ability to compare our findings with other qualitative studies that used Keller and Galgay (2010) to explain their findings.

We recommend that future research investigates the unique themes from this study in more detail to ascertain whether they are applicable to the larger PWD population. We suggest that focus groups combined with individual interviews may help to tease out nuances and could potentially lead to developing theory related to ableist microaggressions and best practices that will support PWD. Finally, we propose that more in-depth intersectionality research would benefit PWD and the professionals who serve them. The confounding nature of microaggressions combined with individuals’ unique identity compositions that often include both nondominant and dominant identities can make this type of research challenging, yet both are the reality for many PWD and this research is therefore needed.

Conclusion

Ableist microaggressions are ubiquitous and damaging to PWD. Through our analysis, we found that participants’ experiences corroborated prior researchers’ findings related to established ableist microaggression categories and added new knowledge by introducing six novel themes. We envision a disability-affirmative counseling profession and offered concrete recommendations for clinicians, supervisors, and counselor educators. Together, we can create a reality in which all PWD who seek counseling services will experience relief, validation, and empowerment as we work to create a society that provides access to all.

 

Conflict of Interest and Funding Disclosure
The authors reported no conflict of interest
or funding contributions for the development
of this manuscript.

 

References

Boyatzis, R. E. (1998). Transforming qualitative information. SAGE.

Chapin, M., McCarthy, H., Shaw, L., Bradham-Cousar, M., Chapman, R., Nosek, M., Peterson, S., Yilmaz, Z., & Ysasi, N. (2018). Disability-related counseling competencies. American Rehabilitation Counseling Association (ARCA) Task Force on Competencies for Counseling Persons with Disabilities. https://arcaweb.org/wp-content/uploads/2023/12/ARCA-Disability-Related-Counseling-Competencies.pdf

Coalson, G. A., Crawford, A., Treleaven, S. B., Byrd, C. T., Davis, L., Dang, L., Edgerly, J., & Turk, A. (2022). Microaggression and the adult stuttering experience. Journal of Communication Disorders, 95, 106180. https://doi.org/10.1016/j.jcomdis.2021.106180

Conover, K. J., Israel, T., & Nylund-Gibson, K. (2017a). Development and validation of the Ableist Microaggressions Scale. The Counseling Psychologist, 45(4), 570–599. https://doi.org/10.1177/0011000017715317

Conover, K. J., Israel, T., & Nylund-Gibson, K. (2017b). Ableist Microaggressions Scale (AMS) [Database record]. APA PsycTESTS. https://doi.org/10.1037/t64519-000

Dávila, B. (2015). Critical race theory, disability microaggressions, and Latina/o student experiences in special education. Race, Ethnicity, and Education, 18(4), 443–468. https://doi.org/10.1080/13613324.2014.885422

Deroche, M. D., Herlihy, B., & Lyons, M. L. (2020). Counselor trainee self-perceived disability competence: Implications for training. Counselor Education and Supervision, 59(3), 187–199. https://doi.org/10.1002/ceas.12183

Deroche, M. D., Ong, L. Z., & Cook, J. M. (2024). Ableist microaggressions, disability characteristics, and nondominant identities. The Professional Counselor, 13(4), 404–417. https://doi.org/10.15241/mdd.13.4.404

Eisenman, L. T., Rolón-Dow, R., Freedman, B., Davison, A., & Yates, N. (2020). “Disabled or not, people just want to feel welcome”: Stories of microaggressions and microaffirmations from college students with intellectual disability. Critical Education, 11(17), 1–21.

Heumann, J., & Joiner, K. (2020). Being Heumann: An unrepentant memoir of a disability rights activist. Beacon Press.

Hunt, B., Matthews, C., Milsom, A., & Lammel, J. A. (2006). Lesbians with physical disabilities: A qualitative study of their experiences with counseling. Journal of Counseling & Development, 84(2), 163–173. https://doi.org/10.1002/j.1556-6678.2006.tb00392.x

Keller, R. M., & Galgay, C. E. (2010). Microaggressive experiences of people with disabilities. In D. W. Sue (Ed.), Microaggressions and marginality: Manifestation, dynamics, and impact (pp. 241–268). Wiley.

Lindsay, S., Fuentes, K., Ragunathan, S., Lamaj, L., & Dyson, J. (2023). Ableism within health care professions: A systematic review of the experiences and impact of discrimination against health care providers with disabilities. Disability and Rehabilitation, 45(17), 2715–2731. https://doi.org/10.1080/09638288.2022.2107086

MacQueen, K. M., McLellan, E., Kay, K., & Milstein, B. (1998). Codebook development for team-based qualitative analysis. Cultural Anthropology Methods, 10(2), 31–36. https://doi.org/10.1177/1525822X980100020301

Miller, R. A., & Smith, A. C. (2021). Microaggressions experienced by LGBTQ students with disabilities. Journal of Student Affairs Research and Practice, 58(5), 491–506. https://doi.org/10.1080/19496591.2020.1835669

Norwood, M. F., Lakhani, A., Hedderman, B., & Kendall, E. (2022). Does being psychologically resilient assist in optimising physical outcomes from a spinal cord injury? Findings from a systematic scoping review. Disability and Rehabilitation, 44(20), 6082–6093. https://doi.org/10.1080/09638288.2021.1952320

Olkin, R., Hayward, H., Abbene, M. S., & VanHeel, G. (2019). The experiences of microaggressions against women with visible and invisible disabilities. Journal of Social Issues, 75(3), 757–785. https://doi.org/10.1111/josi.12342

Ordway, A. R., Johnson, K. L., Amtmann, D., Bocell, F. D., Jensen, M. P., & Molton, I. R. (2020). The relationship between resilience, self-efficacy, and employment in people with physical disabilities. Rehabilitation Counseling Bulletin, 63(4), 195–205. https://doi.org/10.1177/0034355219886660

Patel, D. R., & Brown, K. A. (2017). An overview of the conceptual framework and definitions of disability. International Journal of Child Health and Human Development, 10(3), 247–252.

Perrin, P. B. (2019). Diversity and social justice in disability: The heart and soul of rehabilitation psychology. Rehabilitation Psychology, 64(2), 105–110. https://doi.org/10.1037/rep0000278

Peters, H. J., Schwenk, H. N., Ahlstrom, Z. R., & Mclalwain, L. N. (2017). Microaggressions: The experience of individuals with mental illness. Counselling Psychology Quarterly, 30(1), 86–112. https://doi.org/10.1080/09515070.2016.1164666

Ratts, M. J., Singh, A. A., Nassar-McMillan, S., Butler, S. K., & McCullough, J. R. (2016). Multicultural and social justice counseling competencies: Guidelines for the counseling profession. Journal of Multicultural Counseling and Development, 44(1), 28–48. https://doi.org/10.1002/jmcd.12035

Rivas, M., & Hill, N. R. (2023). A grounded theory of counselors’ post-graduation development of disability counseling effectiveness. Journal of Counselor Preparation and Supervision, 17(1). https://digitalcommons.sacredheart.edu/jcps/vol17/iss1/7

Rolón-Dow, R., & Davison, A. (2018). Racial microaffirmations: Learning from student stories of moments that matter. Diversity Discourse, 1(4), 1–9. https://udspace.udel.edu/server/api/core/bitstreams/4230ddcd-b196-4ee2-a5bf-18120b2b1f24/content

Rolón-Dow, R., & Davison, A. (2021). Theorizing racial microaffirmations: A critical race/LatCrit approach. Race Ethnicity and Education, 24(2), 245–261. https://doi.org/10.1080/13613324.2020.1798381

Sue, D. W., Capodilupo, C. M., Torino, G. C., Bucceri, J. M., Holder, A. M. B., Nadal, K. L., & Esquilin, M. (2007). Racial microaggressions in everyday life: Implications for clinical practice. American Psychologist, 62(4), 271–286. https://doi.org/10.1037/0003-066X.62.4.271

Sue, D. W., & Spanierman, L. (2020). Microaggressions in everyday life (2nd ed.). Wiley.

Swenor, B., & Reed, N. (Hosts). (n.d.). Included: The disability equity podcast [Audio podcast]. Johns Hopkins University Disability Health Research Center. https://disabilityhealth.jhu.edu/included

Jennifer M. Cook, PhD, NCC, ACS, LPC, is an associate professor at the University of Texas at San Antonio. Melissa D. Deroche, PhD, NCC, ACS, LPC-S, is an assistant professor at Tarleton State University. Lee Za Ong, PhD, LPC, CRC, is an assistant professor at Marquette University. Correspondence may be addressed to Jennifer M. Cook, University of Texas at San Antonio, Department of Counseling, 501 W. Cesar E. Chavez Blvd, San Antonio, TX 78207, jennifer.cook@utsa.edu.

Enhancing Assessment Literacy in Professional Counseling: A Practical Overview of Factor Analysis

Michael T. Kalkbrenner

Assessment literacy is an essential competency area for professional counselors who administer tests and interpret the results of participants’ scores. Using factor analysis to demonstrate internal structure validity of test scores is a key element of assessment literacy. The underuse of psychometrically sound instrumentation in professional counseling is alarming, as a careful review and critique of the internal structure of test scores is vital for ensuring the integrity of clients’ results. A professional counselor’s utilization of instrumentation without evidence of the internal structure validity of scores can have a number of negative consequences for their clients, including misdiagnoses and inappropriate treatment planning. The extant literature includes a series of articles on the major types and extensions of factor analysis, including exploratory factor analysis, confirmatory factor analysis (CFA), higher-order CFA, and multiple-group CFA. However, reading multiple psychometric articles can be overwhelming for professional counselors who are looking for comparative guidelines to evaluate the validity evidence of scores on instruments before administering them to clients. This article provides an overview for the layperson of the major types and extensions of factor analysis and can serve as reference for professional counselors who work in clinical, research, and educational settings.

Keywords: Factor analysis, overview, professional counseling, internal structure, validity

Professional counselors have a duty to ensure the veracity of tests before interpreting the results of clients’ scores because clients rely on their counselors to administer and interpret the results of tests that accurately represent their lived experience (American Educational Research Association [AERA] et al., 2014; National Board for Certified Counselors [NBCC], 2016). Internal structure validity of test scores is a key assessment literacy area and involves the extent to which the test items cluster together and represent the intended construct of measurement.

Factor analysis is a method for testing the internal structure of scores on instruments in professional counseling (Kalkbrenner, 2021b; Mvududu & Sink, 2013). The rigor of quantitative research, including psychometrics, has been identified as a weakness of the discipline, and instrumentation with sound psychometric evidence is underutilized by professional counselors (Castillo, 2020; C.-C. Chen et al., 2020; Mvududu & Sink, 2013; Tate et al., 2014). As a result, there is an imperative need for assessment literacy resources in the professional counseling literature, as assessment literacy is a critical competency for professional counselors who work in clinical, research, and educational settings alike.

Assessment Literacy in Professional Counseling
Assessment literacy is a crucial proficiency area for professional counselors, as counselors in a variety of the specialty areas of the Council for Accreditation of Counseling and Related Educational Programs (2015), such as clinical rehabilitation (5.D.1.g. & 5.D.3.a.), clinical mental health (5.C.1.e. & 5.C.3.a.), and addiction (5.A.1.f. & 5.A.3.a.), select and administer tests to clients and use the results to inform diagnosis and treatment planning, and to evaluate the utility of clinical interventions (Mvududu & Sink, 2013; NBCC, 2016; Neukrug & Fawcett, 2015). The extant literature includes a series of articles on factor analysis, including exploratory factor analysis (EFA; Watson, 2017), confirmatory factor analysis (CFA; Lewis, 2017), higher-order CFA (Credé & Harms, 2015), and multiple-group CFA (Dimitrov, 2010). However, reading several articles on factor analysis is likely to overwhelm professional counselors who are looking for a desk reference and/or comparative guidelines to evaluate the validity evidence of scores on instruments before administering them to clients. To these ends, professional counselors need a single resource (“one-stop shop”) that provides a brief and practical overview of factor analysis. The primary purpose of this manuscript is to provide an overview for the layperson of the major types and extensions of factor analysis that counselors can use as a desk reference.

Construct Validity and Internal Structure

     Construct validity, the degree to which a test measures its intended theoretical trait, is a foundation of assessment literacy for demonstrating validity evidence of test scores (Bandalos & Finney, 2019). Internal structure validity, more specifically, is an essential aspect of construct validity and assessment literacy. Internal structure validity is vital for determining the extent to which items on a test combine to represent the construct of measurement (Bandalos & Finney, 2019). Factor analysis is a key method for testing the internal structure of scores on instruments in professional counseling as well as in social sciences research in general (Bandalos & Finney, 2019; Kalkbrenner, 2021b; Mvududu & Sink, 2013). In the following sections, I will provide a practical overview of the two primary methodologies of factor analysis (EFA and CFA) as well as the two main extensions of CFA (higher-order CFA and multiple-group CFA). These factor analytic techniques are particularly important elements of assessment literacy for professional counselors, as they are among the most common psychometric analyses used to validate scores on psychological screening tools (Kalkbrenner, 2021b). Readers might find it helpful to refer to Figure 1 before reading further to become familiar with some common psychometric terms that are discussed in this article and terms that also tend to appear in the measurement literature.

Figure 1

Technical and Layperson’s Definitions of Common Psychometric Terms
Note. Italicized terms are defined in this figure.

Exploratory Factor Analysis
EFA is “exploratory” in that the analysis reveals how, if at all, test items band together to form factors or subscales (Mvududu & Sink, 2013; Watson, 2017). EFA has utility for testing the factor structure (i.e., how the test items group together to form one or more scales) for newly developed or untested instruments. When evaluating the rigor of EFA in an existing psychometric study or conducting an EFA firsthand, counselors should consider sample size, assumption checking, preliminary testing, factor extraction, factor retention, factor rotation, and naming rotated factors (see Figure 2).

EFA: Sample Size, Assumption Checking, and Preliminary Testing
     Researchers should carefully select the minimum sample size for EFA before initiating data collection (Mvududu & Sink, 2013). My 2021 study (Kalkbrenner, 2021b) recommended that the minimal a priori sample size for EFA include either a subjects-to-variables ratio (STV) of 10:1 (at least 10 participants for each test item) or 200 participants, whichever produces a larger sample. EFA tends to be robust to moderate violations of normality; however, results are enriched if data are normally distributed (Mvududu & Sink, 2013). A review of skewness and kurtosis values is one way to test for univariate normality; according to Dimitrov (2012), extreme deviations from normality include skewness values > ±2 and kurtosis > ±7; however, ideally these values are ≤ ±1 (Mvududu & Sink, 2013). The Shapiro-Wilk and Kolmogorov-Smirnov tests can also be computed to test for normality, with non-significant p-values indicating that the parametric properties of the data are not statistically different from a normal distribution (Field, 2018); however, the Shapiro-Wilk and Kolmogorov-Smirnov tests are sensitive to large sample sizes and should be interpreted cautiously. In addition, the data should be tested for linearity (Mvududu & Sink, 2013). Furthermore, extreme univariate and multivariate outliers must be identified and dealt with (i.e., removed, transformed, or winsorized; see Field, 2018) before a researcher can proceed with factor analysis. Univariate outliers can be identified via z-scores (> 3.29), box plots, or scatter plots, and multivariate outliers can be discovered by computing Mahalanobis distance (see Field, 2018).

Figure 2

Flow Chart for Reviewing Exploratory Factor Analysis

 

Three preliminary tests are necessary to determine if data are factorable, including (a) an inter-item correlation matrix, (b) the Kaiser–Meyer–Olkin (KMO) test for sampling adequacy, and (c) Bartlett’s test of sphericity (Beavers et al., 2013; Mvududu & Sink, 2013; Watson, 2017). The purpose of computing an inter-item correlation matrix is to identify redundant items (highly correlated) and individual items that do not fit with any of the other items (weakly correlated). An inter-item correlation matrix is factorable if a number of correlation coefficients for each item are between approximately r = .20 and r = .80 or .85 (Mvududu & Sink, 2013; Watson, 2017). Generally, a factor or subscale should be composed of at least three items (Mvududu & Sink, 2013); thus, an item should display intercorrelations between r = .20 and r = .80/.85 with at least three other items. However, inter-item correlations in this range with five to 10+ items are desirable (depending on the total number of items in the inter-item correlation matrix).

Bartlett’s test of sphericity is computed to test if the inter-item correlation matrix is an identity matrix, in which the correlations between the items is zero (Mvududu & Sink, 2013). An identity matrix is completely unfactorable (Mvududu & Sink, 2013); thus, desirable findings are a significant p-value, indicating that the correlation matrix is significantly different from an identity matrix. Finally, before proceeding with EFA, researchers should compute the KMO test for sampling adequacy, which is a measure of the shared variance among the items in the correlation matrix (Watson, 2017). Kaiser (1974) suggested the following guidelines for interpreting KMO values: “in the .90s – marvelous, in the .80s – meritorious, in the .70s – middling, in the .60s – mediocre, in the .50s – miserable, below .50 – unacceptable” (p. 35).

Factor Extraction Methods
     Factor extraction produces a factor solution by dividing up shared variance (also known as common variance) between each test item from its unique variance, or variance that is not shared with any other variables, and error variance, or variation in an item that cannot be accounted for by the factor solution (Mvududu & Sink, 2013). Historically, principal component analysis (PCA) was the dominant factor extraction method used in social sciences research. PCA, however, is now considered a method of data reduction rather than an approach to factor analysis because PCA extracts all of the variance (shared, unique, and error) in the model. Thus, although PCA can reduce the number of items in an inter-item correlation matrix, one cannot be sure if the factor solution is held together by shared variance (a potential theoretical model) or just by random error variance.

More contemporary factor extraction methods that only extract shared variance—for example, principal axis factoring (PAF) and maximum likelihood (ML) estimation methods—are generally recommended for EFA (Mvududu & Sink, 2013). PAF has utility if the data violate the assumption of normality, as PAF is robust to modest violations of normality (Mvududu & Sink, 2013). If, however, data are largely consistent with a normal distribution (skewness and kurtosis values ≤ ±1), researchers should consider using the ML extraction method. ML is advantageous, as it computes the likelihood that the inter-item correlation matrix was acquired from a population in which the extracted factor solution is a derivative of the scores on the items (Watson, 2017).

     Factor Retention. Once a factor extraction method is deployed, psychometric researchers are tasked with retaining the most parsimonious (simple) factor solution (Watson, 2017), as the purpose of factor analysis is to account for the maximum proportion of variance (ideally, 50%–75%+) in an inter-item correlation matrix while retaining the fewest possible number of items and factors (Mvududu & Sink, 2013). Four of the most commonly used criteria for determining the appropriate number of factors to retain in social sciences research include the (a) Kaiser criterion, (b) percentage of variance among items explained by each factor, (c) scree plot, and (d) parallel analysis (Mvududu & Sink, 2013; Watson, 2017). Kaiser’s criterion is a standard for retaining factors with Eigenvalues (EV) ≥ 1. An EV represents the proportion of variance that is explained by each factor in relation to the total amount of variance in the factor matrix.

The Kaiser criterion tends to overestimate the number of retainable factors; however, this criterion can be used to extract an initial factor solution (i.e., when computing the EFA for the first time). Interpreting the percentage of variance among items explained by each factor is another factor retention criterion based on the notion that a factor must account for a large enough percentage of variance to be considered meaningful (Mvududu & Sink, 2013). Typically, a factor should account for at least 5% of the variance in the total model. A scree plot is a graphical representation or a line graph that depicts the number of factors on the X-axis and the corresponding EVs on the Y-axis (see Figure 6 in Mvududu & Sink, 2013, p. 87, for a sample scree plot). The cutoff for the number of factors to retain is portrayed by a clear bend in the line graph, indicating the point at which additional factors fail to contribute a substantive amount of variance to the total model. Finally, in a parallel analysis, EVs are generated from a random data set based on the number of items and the sample size of the real (sample) data. The factors from the sample data with EVs larger than the EVs from the randomly generated data are retained based on the notion that these factors explain more variance than would be expected by random chance. In some instances, these four criteria will reveal different factor solutions. In such cases, researchers should retain the simplest factor solution that makes both statistical and substantive sense.

     Factor Rotation. After determining the number of factors to retain, researchers seek to uncover the association between the items and the factors or subscales (i.e., determining which items load on which factors) and strive to find simple structure or items with high factor loadings (close to ±1) on one factor and low factor loadings (near zero) on the other factors (Watson, 2017). The factors are rotated on vectors to enhance the readability or detection of simple structure (Mvududu & Sink, 2013). Orthogonal rotation methods (e.g., varimax, equamax, and quartimax) are appropriate when a researcher is measuring distinct or uncorrelated constructs of measurement. However, orthogonal rotation methods are rarely appropriate for use in counseling research, as counselors almost exclusively appraise variables that display some degree of inter-correlation (Mvududu & Sink, 2013). Oblique rotation methods (e.g., direct oblimin and promax) are generally more appropriate in counseling research, as they allow factors to inter-correlate by rotating the data on vectors at angles less than 90. The nature of oblique rotations allows the total variance accounted for by each factor to overlap; thus, the total variance explained in a post–oblique rotated factor solution can be misleading (Bandalos & Finney, 2019). For example, the total variance accounted for in a post–oblique rotated factor solution might add up to more than 100%. To this end, counselors should report the total variance explained by the factor solution before rotation as well as the sum of each factor’s squared structure coefficient following an oblique factor rotation.

Following factor rotation, researchers examine a number of factor retention criteria to determine the items that load on each factor (Watson, 2017). Commonality values (h2) represent the proportion of variance that the extracted factor solution explains for each item. Items with h2 values that range between .30 and .99 should be retained, as they share an adequate amount of shared variance with the other items and factors (Watson, 2017). Items with small h2 values (< .30) should be considered for removal. However, commonality values should not be too high (≥ 1), as this suggests one’s sample size was insufficient or too many factors were extracted (Watson, 2017). Items with problematic h2 values should be removed one at a time, and the EFA should be re-computed after each removal because these values will fluctuate following each deletion. Oblique factor rotation methods produce two matrices, including the pattern matrix, which displays the relationship between the items and a factor while controlling for the items’ association with the other factors, and the structure matrix, which depicts the correlation between the items and all of the factors (Mvududu & Sink, 2013). Researchers should examine both the pattern and the structure matrices and interpret the one that displays the clearest evidence of simple structure with the least evidence of cross-loadings.

Items should display a factor loading of at least ≥ .40 (≥ .50 is desirable) to mark a factor. Items that fail to meet a minimum factor loading of ≥ .40 should be deleted. Cross-loading is evident when an item displays factor loadings ≥ .30 to .35 on two or more factors (Beavers et al., 2013; Mvududu & Sink, 2013; Watson, 2017). Researchers may elect to assign a variable to one factor if that item’s loading is .10 higher than the next highest loading. Items that cross-load might also be deleted. Once again, items should be deleted one at a time and the EFA should be re-computed after each removal.

Naming the Rotated Factors
     The final step in EFA is naming the rotated factors; factor names should be brief (approximately one to four words) and capture the theoretical meaning of the group of items that comprise the factor (Mvududu & Sink, 2013). This is a subjective process, and the literature is lacking consistent guidelines for the process of naming factors. A research team can be incorporated into the process of naming their factors. Test developers can separately name each factor and then meet with their research team to discuss and eventually come to an agreement about the most appropriate name for each factor.

Confirmatory Factor Analysis
     CFA is an application of structural equation modeling for testing the extent to which a hypothesized factor solution (e.g., the factor solution that emerged in the EFA or another existing factor solution) demonstrates an adequate fit with a different sample (Kahn, 2006; Lewis, 2017). When validating scores on a new test, investigators should compute both EFA and CFA with two different samples from the same population, as the emergent internal structure in EFA can vary substantially. Researchers can collect two sequential samples or they may elect to collect one large sample and divide it into two smaller samples, one for EFA and the second for CFA.

Evaluating model fit in CFA is a complex task that is typically determined by examining the collective implications of multiple goodness-of-fit (GOF) indices, which include absolute, incremental, and parsimonious (Lewis, 2017). Absolute fit indices evaluate the extent to which the hypothesized model or the dimensionality of the existing measure fits with the data collected from a new sample. Incremental fit indices compare the improvement in fit between the hypothesized model and a null model (also referred to as an independence model) in which there is no correlation between observed variables. Parsimonious fit indices take the model’s complexity into account by testing the extent to which model fit is improved by estimating fewer pathways (i.e., creating a more parsimonious or simple model). Psychometric researchers generally report a combination of absolute, incremental, and parsimonious fit indices to demonstrate acceptable model fit (Mvududu & Sink, 2013). Table 1 includes tentative guidelines for interpreting model fit based on the synthesized recommendations of leading psychometric researchers from a comprehensive search of the measurement literature (Byrne, 2016; Dimitrov, 2012; Fabrigar et al., 1999; Hooper et al., 2008; Hu & Bentler, 1999; Kahn, 2006; Lewis, 2017; Mvududu & Sink, 2013; Schreiber et al., 2006; Worthington & Whittaker, 2006).

Table 1

Fit Indices and Tentative Thresholds for Evaluating Model Fit

Note. The fit indices and benchmarks to estimate the degree of model fit in this table are offered as tentative guidelines for scores on attitudinal measures based on the synthesized recommendations of numerous psychometric researchers (see citations in the “Confirmatory Factor Analysis” section of this article). The list of fit indices in this table are not all-inclusive (i.e., not all of them are typically reported). There is no universal approach for determining which fit indices to investigate nor are there any absolute thresholds for determining the degree of model fit. No single fix index is sufficient for determining model fit. Researchers are tasked with selecting and interpreting fit indices holistically (i.e., collectively), in ways that make both statistical and substantive sense based on their construct of measurement and goals of the study.
*.90 to .94 can denote an acceptable model fit for incremental fix indices; however, the majority of values should be ≥ .95.

 

Model Respecification
     The results of a CFA might reveal a poor or unacceptable model fit (see Table 1), indicating that the dimensionality of the hypothesized model that emerged from the EFA was not replicated or confirmed with a second sample (Mvududu & Sink, 2013). CFA is a rigorous model-fitting procedure and poor model fit in a CFA might indicate that the EFA-derived factor solution is insufficient for appraising the construct of measurement. CFA, however, is a more stringent test of structural validity than EFA, and psychometric researchers sometimes refer to the modification indices (also referred to as Lagrange multiplier statistics), which denote the expected decrease in the X2 value (i.e., degree of improvement in model fit) if the parameter is freely estimated (Dimitrov, 2012). In these instances, correlating the error terms between items or removing problematic items will improve model fit; however, when considering model respecification, psychometric researchers should proceed cautiously, if at all, as a strong theoretical justification is necessary to defend model respecification (Byrne, 2016; Lewis, 2017; Schreiber et al., 2006). Researchers should also be clear that model respecification causes the CFA to become an EFA because they are investigating the dimensionality of a different or modified model rather than confirming the structure of an existing, hypothesized model.

Higher-Order CFA
     Higher-order CFA is an extension of CFA that allows researchers to test nested models and determine if a second-order latent variable (factor) explains the associations between the factors in a single-order CFA (Credé & Harms, 2015). Similar to single-order CFA (see Figure 3, Model 1) in which the test items cluster together to form the factors or subscales, higher-order CFA reveals if the factors are related to one another strongly enough to suggest the presence of a global factor (see Figure 3, Model 3). Suppose, for example, the test developer of a scale for measuring dimensions of the therapeutic alliance confirmed the three following subscales via single-order CFA (see Figure 3, Model 1): Empathy, Unconditional Positive Regard, and Congruence. Computing a higher-order CFA would reveal if a higher-order construct, which the research team might name Therapeutic Climate, is present in the data. In other words, higher-order CFA reveals if Empathy, Unconditional Positive Regard, and Congruence, collectively, comprise the second-order factor of Therapeutic Climate.

Determining if a higher-order factor explains the co-variation (association) between single-order factors is a complex undertaking. Thus, researchers should consider a number of criteria when deciding if their data are appropriate for higher-order CFA (Credé & Harms, 2015). First, moderate-to-strong associations (co-variance) should exist between first-order factors. Second, the unidimensional factor solution (see Figure 3, Model 2) should display a poor model fit (see Table 1) with the data. Third, theoretical support should exist for the presence of a higher-order factor. Referring to the example in the previous paragraph, person-centered therapy provides a theory-based explanation for the presence of a second-order or global factor (Therapeutic Climate) based on the integration of the single-order factors (Empathy, Unconditional Positive Regard, and Congruence). In other words, the presence of a second-order factor suggests that Therapeutic Climate explains the strong association between Empathy, Unconditional Positive Regard, and Congruence.

Finally, the single-order factors should display strong factor loadings (approximately ≥ .70) on the higher-order factor. However, there is not an absolute consensus among psychometric researchers regarding the criteria for higher-order CFA and the criteria summarized in this section are not a dualistic decision rule for retaining or rejecting a higher-order model. Thus, researchers are tasked with presenting that their data meet a number of criteria to justify the presence of a higher-order factor. If the results of a higher-order CFA reveal an acceptable model fit (see Table 1), researchers should directly compare (e.g., chi-squared test of difference) the single-order and higher-order models to determine if one model demonstrates a superior fit with the data at a statistically significant level.

Figure 3

Single-Order, Unidimensional, and Higher-Order Factor Solutions

 

Multiple-Group Confirmatory Factor Analysis
     Multiple-group confirmatory factor analysis (MCFA) is an extension of CFA for testing the factorial invariance (psychometric equivalence) of a scale across subgroups of a sample or population (C.-C. Chen et al., 2020; Dimitrov, 2010). In other words, MCFA has utility for testing the extent to which a particular construct has the same meaning across different groups of a larger sample or population. Suppose, for example, the developer of the Therapeutic Climate scale (see example in the previous section) validated scores on their scale with undergraduate college students. Invariance testing has potential to provide further support for the internal structure validity of the scale by testing whether Empathy, Unconditional Positive Regard, and Congruence have the same meaning across different subgroups of undergraduate college students (e.g., between different gender identities, ethnic identities, age groups, and other subgroups of the larger sample).

     Levels of Invariance. Factorial invariance can be tested in a number of different ways and includes the following primary levels or aspects: (a) configural invariance, (b) measurement (metric, scalar, and strict) invariance, and (c) structural invariance (Dimitrov, 2010, 2012). Configural invariance (also referred to as pattern invariance) serves as the baseline mode (typically the best fitting model with the data), which is used as the point of comparison when testing for metric, scalar, and structural invariance. In layperson’s terms, configural invariance is a test of whether the scales are approximately similar across groups.

Measurement invariance includes testing for metric and scalar invariance. Metric invariance is a test of whether each test item makes an approximately equal contribution (i.e., approximately equal factor loadings) to the latent variable (composite scale score). In layperson’s terms, metric invariance evaluates if the scale reasonably captures the same construct. Scalar invariance adds a layer of rigor to metric invariance by testing if the differences between the average scores on the items are attributed to differences in the latent variable means. In layperson’s terms, scalar invariance indicates that if the scores change over time, they change in the same way.

Strict invariance is the most stringent level of measurement invariance testing and tests if the sum total of the items’ unique variance (item variation that is not in common with the factor) is comparable to the error variance across groups. In layperson’s terms, the presence of strict invariance demonstrates that score differences between groups are exclusively due to differences in the common latent variables. Strict invariance, however, is typically not examined in social sciences research because the latent factors are not composed of residuals. Thus, residuals are negligible when evaluating mean differences in latent scores (Putnick & Bornstein, 2016).

Finally, structural invariance is a test of whether the latent factor variances are equivalent to the factor covariances (Dimitrov, 2010, 2012). Structural invariance tests the null hypothesis that there are no statistically significant differences between the unconstrained and constrained models (i.e., determines if the unconstrained model is equivalent to the constrained model). Establishing structural invariance indicates that when the structural pathways are allowed to vary across the two groups, they naturally produce equal results, which supports the notion that the structure of the model is invariant across both groups. In layperson’s terms, the presence of structural invariance indicates that the pathways (directionality) between variables behave in the same way across both groups. It is necessary to establish configural and metric invariance prior to testing for structural invariance.

     Sample Size and Criteria for Evaluating Invariance. Researchers should check their sample size before computing invariance testing, as small samples (approximately < 200) can overestimate model fit (Dimitrov, 2010). Similar to single-order CFA, no absolute sample size guidelines exist in the literature for invariance testing. Generally, a minimum sample of at least 200 participants per group is recommended for invariance testing (although < 200 to 300+ is advantageous). Referring back to the Therapeutic Climate scale example (see the previous section), investigators would need a minimum sample of 400 if they were seeking to test the invariance of the scale by generational status (200 first generation + 200 non-first generation = 400). The minimum sample size would increase as more levels are added. For example, a minimum sample of 600 would be recommended if investigators quantified generational status on three levels (200 first generation + 200 second generation + 200 third generation and beyond = 600).

Factorial invariance is investigated through a computation of the change in model fit at each level of invariance testing (F. F. Chen, 2007). Historically, the Satorra and Bentler chi-square difference test was the sole criteria for testing factorial invariance, with a non-significant p-value indicating factorial invariance (Putnick & Bornstein, 2016). The chi-square difference test is still commonly reported by contemporary psychometric researchers; however, it is rarely used as the sole criteria for determining invariance, as the test is sensitive to large samples. The combined recommendations of F. F. Chen (2007) and Putnick and Bornstein (2016) include the following thresholds for investigating invariance: ≤ ∆ 0.010 in CFI, ≤ ∆ 0.015 in RMSEA, and ≤ ∆ 0.030 in SRMR for metric invariance or ≤ ∆ 0.015 in SRMR for scalar invariance. In a simulation study, Kang et al. (2016) found that McDonald’s NCI (MNCI) outperformed the CFI in terms of stability. Kang et al. (2016) recommend < ∆ 0.007 in MNCI for the 5th percentile and ≤ ∆ 0.007 in MNCI for the 1st percentile as cutoff values for measurement quality. Strong measurement invariance is achieved when both metric and scalar invariance are met, and weak invariance is accomplished when only metric invariance is present (Dimitrov, 2010).

Exemplar Review of a Psychometric Study

     The following section will include a review of an exemplar psychometric study based on the recommendations for EFA (see Figure 2) and CFA (see Table 1) that are provided in this manuscript. In 2020, I collaborated with Ryan Flinn on the development and validation of scores on the Mental Distress Response Scale (MDRS) for appraising how college students are likely to respond when encountering a peer in mental distress (Kalkbrenner & Flinn, 2020). A total of 13 items were entered into an EFA. Following the steps for EFA (see Figure 1), the sample size (N = 569) exceeded the guidelines for sample size that I published in my 2021 article (Kalkbrenner, 2021b), including an STV of 10:1 or 200 participants, whichever produces a larger sample. Flinn and I (2020) ensured that our 2020 study’s data were consistent with a normal distribution (skewness & kurtosis values ≤ ±1) and computed preliminary assumption checking, including inter-item correlation matrix, KMO (.73), and Bartlett’s test of sphericity (p < .001).

An ML factor extraction method was employed, as the data were largely consistent (skewness & kurtosis values ≤ ±1) with a normal distribution. We used the three most rigorous factor retention criteria—percentage of variance accounted for, scree test, and parallel analysis—to extract a two-factor solution. An oblique factor rotation method (direct oblimin) was employed, as the two factors were correlated. We referred to the recommended factor retention criteria, including h2 values .30 to .99, factor loadings ≥ .40, and cross-loading ≥ .30, to eliminate one item with low commonalities and two cross-loading items. Using a research team, we named the first factor Diminish/Avoid, as each item that marked this factor reflected a dismissive or evasive response to encountering a peer in mental distress. The second factor was named Approach/Encourage because each item that marked this factor included a response to a peer in mental distress that was active and likely to help connect their peer to mental health support services.

Our next step was to compute a CFA by administering the MDRS to a second sample of undergraduate college students to confirm the two-dimensional factor solution that emerged in the EFA. The sample size (N = 247) was sufficient for CFA (STV > 10:1 and > 200 participants). The MDRS items were entered into a CFA and the following GOF indices emerged: CMIN = χ2 (34) = 61.34, p = .003, CMIN/DF = 1.80, CFI = .96, IFI = .96, RMSEA = .06, 90% CI [0.03, 0.08], and SRMR = .04. A comparison between our GOF indices from the 2020 study with the thresholds for evaluating model fit in Table 1 reveal an acceptable-to-strong fit between the MDRS model and the data. Collectively, our 2020 procedures for EFA and CFA were consistent with the recommendations in this manuscript.

Implications for the Profession

Implications for Counseling Practitioners
     Assessment literacy is a vital component of professional counseling practice, as counselors who practice in a variety of specialty areas select and administer tests to clients and use the results to inform diagnosis and treatment planning (C.-C. Chen et al., 2020; Mvududu & Sink, 2013; NBCC, 2016; Neukrug & Fawcett, 2015). It is important to note that test results alone should not be used to make diagnoses, as tests are not inherently valid (Kalkbrenner, 2021b). In fact, the authors of the Diagnostic and Statistical Manual of Mental Disorders stated that “scores from standardized measures and interview sources must be interpreted using clinical judgment” (American Psychiatric Association, 2013, p. 37). Professional counselors can use test results to inform their diagnoses; however, diagnostic decision making should ultimately come down to a counselor’s clinical judgment.

Counseling practitioners can refer to this manuscript as a reference for evaluating the internal structure validity of scores on a test to help determine the extent to which, if any at all, the test in question is appropriate for use with clients. When evaluating the rigor of an EFA for example, professional counselors can refer to this manuscript to evaluate the extent to which test developers followed the appropriate procedures (e.g., preliminary assumption checking, factor extraction, retention, and rotation [see Figure 2]). Professional counselors are encouraged to pay particular attention to the factor extraction method that the test developers employed, as PCA is sometimes used in lieu of more appropriate methods (e.g., PAF/ML). Relatedly, professional counselors should be vigilant when evaluating the factor rotation method employed by test developers because oblique rotation methods are typically more appropriate than orthogonal (e.g., varimax) for counseling tests.

CFA is one of the most commonly used tests of the internal structure validity of scores on psychological assessments (Kalkbrenner, 2021b). Professional counselors can compare the CFA fit indices in a test manual or journal article to the benchmarks in Table 1 and come to their own conclusion about the internal structure validity of scores on a test before using it with clients. Relatedly, the layperson’s definitions of common psychometric terms in Figure 1 might have utility for increasing professional counselors’ assessment literacy by helping them decipher some of the psychometric jargon that commonly appears in psychometric studies and test manuals.

Implications for Counselor Education
     Assessment literacy begins in one’s counselor education program and it is imperative that counselor educators teach their students to be proficient in recognizing and evaluating internal structure validity evidence of test scores. Teaching internal structure validity evidence can be an especially challenging pursuit because counseling students tend to fear learning about psychometrics and statistics (Castillo, 2020; Steele & Rawls, 2015), which can contribute to their reticence and uncertainty when encountering psychometric research. This reticence can lead one to read the methodology section of a psychometric study briefly, if at all. Counselor educators might suggest the present article as a resource for students taking classes in research methods and assessment as well as for students who are completing their practicum, internship, or dissertation who are evaluating the rigor of existing measures for use with clients or research participants.

Counselor educators should urge their students not to skip over the methodology section of a psychometric study. When selecting instrumentation for use with clients or research participants, counseling students and professionals should begin by reviewing the methodology sections of journal articles and test manuals to ensure that test developers employed rigorous and empirically supported procedures for test development and score validation. Professional counselors and their students can compare the empirical steps and guidelines for structural validation of scores that are presented in this manuscript with the information in test manuals and journal articles of existing instrumentation to evaluate its internal structure. Counselor educators who teach classes in assessment or psychometrics might integrate an instrument evaluation assignment into the course in which students select a psychological instrument and critique its psychometric properties. Another way that counselor educators who teach classes in current issues, research methods, assessment, or ethics can facilitate their students’ assessment literacy development is by creating an assignment that requires students to interview a psychometric researcher. Students can find psychometric researchers by reviewing the editorial board members and authors of articles published in the two peer-reviewed journals of the Association for Assessment and Research in Counseling, Measurement and Evaluation in Counseling and Development and Counseling Outcome Research and Evaluation. Students might increase their interest and understanding about the necessity of assessment literacy by talking to researchers who are passionate about psychometrics.

Assessment Literacy: Additional Considerations

Internal structure validity of scores is a crucial component of assessment literacy for evaluating the construct validity of test scores (Bandalos & Finney, 2019). Assessment literacy, however, is a vast construct and professional counselors should consider a number of additional aspects of test worthiness when evaluating the potential utility of instrumentation for use with clients. Reviewing these additional considerations is beyond the scope of this manuscript; however, readers can refer to the following features of assessment literacy and corresponding resources: reliability (Kalkbrenner, 2021a), practicality (Neukrug & Fawcett, 2015), steps in the instrument development process (Kalkbrenner, 2021b), and convergent and divergent validity evidence of scores (Swank & Mullen, 2017). Moreover, the discussion of internal structure validity evidence of scores in this manuscript is based on Classical Test Theory (CTT), which tends to be an appropriate platform for attitudinal measures. However, Item Response Theory (see Amarnani, 2009) is an alternative to CTT with particular utility for achievement and aptitude testing.

Cross-Cultural Considerations in Assessment Literacy
     Professional counselors have an ethical obligation to consider the cross-cultural fairness of a test before use with clients, as the validity of test scores are culturally dependent (American Counseling Association [ACA], 2014; Kane, 2010; Neukrug & Fawcett, 2015; Swanepoel & Kruger, 2011). Cross-cultural fairness (also known as test fairness) in testing and assessment “refers to the comparability of score meanings across individuals, groups or settings” (Swanepoel & Kruger, 2011, p. 10). There exists some overlap between internal structure validity and cross-cultural fairness; however, some distinct differences exist as well.

Using CFA to confirm the factor structure of an established test with participants from a different culture is one way to investigate the cross-cultural fairness of scores. Suppose, for example, an investigator found acceptable internal structure validity evidence (see Table 1) for scores on an anxiety inventory that was normed in America with participants in Eastern Europe who identify with a collectivist cultural background. Such findings would suggest that the dimensionality of the anxiety inventory extends to the sample of Eastern European participants. However, internal structure validity testing alone might not be sufficient for testing the cross-cultural fairness of scores, as factor analysis does not test for content validity. In other words, although the CFA confirmed the dimensionality of an American model with a sample of Eastern European participants, the analysis did not take potential qualitative differences about the construct of measurement (anxiety severity) into account. It is possible (and perhaps likely) that the lived experience of anxiety differs between those living in two different cultures. Accordingly, a systems-level approach to test development and score validation can have utility for enhancing the cross-cultural fairness of scores (Swanepoel & Kruger, 2011).

A Systems-Level Approach to Test Development and Score Validation
     Swanepoel and Kruger (2011) outlined a systemic approach to test development that involves circularity, which includes incorporating qualitative inquiry into the test development process, as qualitative inquiry has utility for uncovering the nuances of participants’ lived experiences that quantitative data fail to capture. For example, an exploratory-sequential mixed-methods design in which qualitative findings are used to guide the quantitative analyses is a particularly good fit with systemic approaches to test development and score validation. Referring to the example in the previous section, test developers might conduct qualitative interviews to develop a grounded theory of anxiety severity in the context of the collectivist culture. The grounded theory findings could then be used as the theoretical framework (see Kalkbrenner, 2021b) for a psychometric study aimed at testing the generalizability of the qualitative findings. Thus, in addition to evaluating the rigor of factor analytic results, professional counselors should also review the cultural context in which test items were developed before administering a test to clients.

Language adaptions of instrumentation are another relevant cross-cultural fairness consideration in counseling research and practice. Word-for-word translations alone are insufficient for capturing cross-cultural fairness of instrumentation, as culture extends beyond just language (Lenz et al., 2017; Swanepoel & Kruger, 2011). Pure word-for-word translations can also cause semantic errors. For example, feeling “fed up” might translate to feeling angry in one language and to feeling full after a meal in another language. Accordingly, professional counselors should ensure that a translated instrument was subjected to rigorous procedures for maintaining cross-cultural fairness. Reviewing such procedures is beyond the scope of this manuscript; however, Lenz et al. (2017) outlined a 6-step process for language translation and cross-cultural adaptation of instruments.

Conclusion

Gaining a deeper understanding of the major approaches to factor analysis for demonstrating internal structure validity in counseling research has potential to increase assessment literacy among professional counselors who work in a variety of specialty areas. It should be noted that the thresholds for interpreting the strength of internal structure validity coefficients that are provided throughout this manuscript should be used as tentative guidelines, not unconditional standards. Ultimately, internal structure validity is a function of test scores and the construct of measurement. The stakes or consequences of test results should be considered when making final decisions about the strength of validity coefficients. As professional counselors increase their familiarity with factor analysis, they will most likely become more cognizant of the strengths and limitations of counseling-related tests to determine their utility for use with clients. The practical overview of factor analysis presented in this manuscript can serve as a one-stop shop or resource that professional counselors can refer to as a reference for selecting tests with validated scores for use with clients, a primer for teaching courses, and a resource for conducting their own research.

 

Conflict of Interest and Funding Disclosure
The author reported no conflict of interest
or funding contributions for the development
of this manuscript.


References

Amarnani, R. (2009). Two theories, one theta: A gentle introduction to item response theory as an alternative to classical test theory. The International Journal of Educational and Psychological Assessment, 3, 104–109.

American Counseling Association. (2014). ACA code of ethics. https://www.counseling.org/resources/aca-code-of-ethics.pdf

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (2014). Standards for educational and psychological testing. https://www.apa.org/science/programs/testing/standards

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).
https://doi.org/10.1176/appi.books.9780890425596

Bandalos, D. L., & Finney, S. J. (2019). Factor analysis: Exploratory and confirmatory. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (2nd ed., pp. 98–122). Routledge.

Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research and Evaluation, 18(5/6), 1–13. https://doi.org/10.7275/qv2q-rk76

Byrne, B. M. (2016). Structural equation modeling with AMOS: Basic concepts, applications, and programming (3rd ed.). Routledge.

Castillo, J. H. (2020). Teaching counseling students the science of research. In M. O. Adekson (Ed.), Beginning your counseling career: Graduate preparation and beyond (pp. 122–130). Routledge.

Chen, C.-C., Lau, J. M., Richardson, G. B., & Dai, C.-L. (2020). Measurement invariance testing in counseling. Journal of Professional Counseling: Practice, Theory & Research, 47(2), 89–104.
https://doi.org/10.1080/15566382.2020.1795806

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

Council for Accreditation of Counseling and Related Educational Programs. (2015). 2016 CACREP standards. http://www.cacrep.org/wp-content/uploads/2017/08/2016-Standards-with-citations.pdf

Credé, M., & Harms, P. D. (2015). 25 years of higher-order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations. Journal of Organizational
Behavior
, 36(6), 845–872. https://doi.org/10.1002/job.2008

Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121–149. https://doi.org/10.1177/0748175610373459

Dimitrov, D. M. (2012). Statistical methods for validation of assessment scale data in counseling and related fields. American Counseling Association.

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299.
https://doi.org/10.1037/1082-989X.4.3.272

Field, A. (2018). Discovering statistics using IBM SPSS statistics (5th ed.). SAGE.

Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53–60.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Kahn, J. H. (2006). Factor analysis in counseling psychology research, training, and practice: Principles, advances, and applications. The Counseling Psychologist, 34(5), 684–718. https://doi.org/10.1177/0011000006286347

Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36. https://doi.org/10.1007/BF02291575

Kalkbrenner, M. T. (2021a). Alpha, omega, and H internal consistency reliability estimates: Reviewing these options and when to use them. Counseling Outcome Research and Evaluation. Advance online publication. https://doi.org/10.1080/21501378.2021.1940118

Kalkbrenner, M. T. (2021b). A practical guide to instrument development and score validation in the social sciences: The MEASURE Approach. Practical Assessment, Research, and Evaluation, 26, Article 1. https://scholarworks.umass.edu/pare/vol26/iss1/1

Kalkbrenner, M. T., & Flinn, R. E. (2020). The Mental Distress Response Scale and promoting peer-to-peer mental health support: Implications for college counselors and student affairs officials. Journal of College Student Development, 61(2), 246–251. https://doi.org/10.1353/csd.2020.0021

Kane, M. (2010). Validity and fairness. Language Testing, 27(2), 177–182. https://doi.org/10.1177/0265532209349467

Kang, Y., McNeish, D. M., & Hancock, G. R. (2016). The role of measurement quality on practical guidelines for assessing measurement and structural invariance. Educational and Psychological Measurement, 76(4), 533–561. https://doi.org/10.1177/0013164415603764

Lenz, A. S., Gómez Soler, I., Dell’Aquilla, J., & Uribe, P. M. (2017). Translation and cross-cultural adaptation of assessments for use in counseling research. Measurement and Evaluation in Counseling and Development, 50(4), 224–231. https://doi.org/10.1080/07481756.2017.1320947

Lewis, T. F. (2017). Evidence regarding the internal structure: Confirmatory factor analysis. Measurement and Evaluation in Counseling and Development, 50(4), 239–247. https://doi.org/10.1080/07481756.2017.1336929

Mvududu, N. H., & Sink, C. A. (2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4(2), 75–98. https://doi.org/10.1177/2150137813494766

National Board for Certified Counselors. (2016). NBCC code of ethics. https://www.nbcc.org/Assets/Ethics/NBCCCodeofEthics.pdf

Neukrug, E. S., & Fawcett, R. C. (2015). Essentials of testing and assessment: A practical guide for counselors, social workers, and psychologists (3rd ed.). Cengage.

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90.  https://doi.org/10.1016/j.dr.2016.06.004

Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Educational Research, 99(6), 323–338.
https://doi:10.3200/JOER.99.6.323-338

Steele, J. M., & Rawls, G. J. (2015). Quantitative research attitudes and research training perceptions among master’s-level students. Counselor Education and Supervision, 54(2), 134–146. https://doi.org/10.1002/ceas.12010

Swanepoel, I., & Kruger, C. (2011). Revisiting validity in cross-cultural psychometric-test development: A systems-informed shift towards qualitative research designs. South African Journal of Psychiatry, 17(1), 10–15. https://doi.org/10.4102/sajpsychiatry.v17i1.250

Swank, J. M., & Mullen, P. R. (2017). Evaluating evidence for conceptually related constructs using bivariate correlations. Measurement and Evaluation in Counseling and Development, 50(4), 270–274.
https://doi.org/10.1080/07481756.2017.1339562

Tate, K. A., Bloom, M. L., Tassara, M. H., & Caperton, W. (2014). Counselor competence, performance assessment, and program evaluation: Using psychometric instruments. Measurement and Evaluation in Counseling and Development, 47(4), 291–306. https://doi.org/10.1177/0748175614538063

Watson, J. C. (2017). Establishing evidence for internal structure using exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 50(4), 232–238. https://doi.org/10.1080/07481756.2017.1336931

Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127

Michael T. Kalkbrenner, PhD, NCC, is an associate professor at New Mexico State University. Correspondence may be addressed to Michael T. Kalkbrenner, Department of Counseling and Educational Psychology, New Mexico State University, Las Cruces, NM 88003, mkalk001@nmsu.edu.