V. Casey Dozier, James P. Sampson, Robert C. Reardon
John Holland’s Self-Directed Search (SDS) is a career assessment that consists of several booklets designed to be self-scored and self-administered. It simulates what a practitioner and an individual might do together in a career counseling session (e.g., review preferred activities and occupations; review competencies, abilities and possible career course; and consider RIASEC theory). This study examined how individuals used two different interpretive materials with the SDS assessment: (1) two paper booklets and (2) the computer-generated SDS Interpretive Report (SDS:IR). Participants receiving the SDS:IR were more likely to recall their SDS summary codes and expand their options than those receiving the two paper booklets.
Keywords: Self-Directed Search, SDS Interpretive Report, RIASEC theory, career course, John Holland, career assessment
The Self-Directed Search (SDS; Holland, 1994a) is a career intervention based on John Holland’s (1997) RIASEC theory (Realistic, Investigative, Artistic, Social, Enterprising, Conventional). The basic form of the SDS consists of several booklets designed to be self-scored and self-administered. It simulates what a practitioner and an individual might do together in a counseling session (e.g., review preferred activities and occupations, and review competencies and abilities). In this study, all students in two sections of a college career course completed the paper version of the SDS Form R Assessment booklet (Holland, 1994a) using the paper Occupations Finder (OF; Holland, 2000). Students in one section of the course received the SDS:IR generated by the SDS software system (IR; Reardon & PAR, 2001). Those in the other section received two paper booklets, You and Your Career (YYC; Holland, 1994b) and the Educational Opportunities Finder (EOF; Rosen, Holmberg, & Holland, 1997). These interpretive materials were both designed to answer common questions about the SDS assessment results and about how the findings might be used to improve college students’ career decision-making.
The YYC reusable booklet gives the practitioner a tool for interpreting the results of the SDS assessment activity, and for providing a more complete career intervention (Reardon & Lenz, 1998). Although the YYC booklet was designed to make the SDS a more self-directed activity and support the practitioner’s work, little research has investigated the usefulness of this booklet (Holland, Fritzsche, & Powell, 1994). The EOF is a listing of more than 700 educational programs at three degree levels according to three-letter Holland summary codes. The computer-generated interpretive report uses the SDS assessment results and includes content materials from the OF, YYC, and EOF booklets to produce a customized report based on an individual’s scores from the assessment booklet.
Holland’s work has been referenced in more than 1,600 citations, and the SDS is reported in hundreds of studies (Ruff, Reardon, & Bertoch, 2008). For example, Holland (1997) reported more than two dozen SDS studies with high school students, college students, and adults that demonstrated the psychometric properties of the inventory, examined the efficacy of the RIASEC typology, and investigated the relationship between the Big Five personality factors and RIASEC theory. Despite the popularity of the SDS, we were unable to locate any studies investigating the efficacy of the interpretive materials that support the SDS or the varied ways it might be used (e.g., in-person, booklet alone, personalized computer-generated report).
Whiston and James (2013) concluded that although Brown and Ryan Krane (2000) found that individualized interpretations and feedback were one of five key ingredients in successful career interventions, “there is little outcome research related to precisely how practitioners should interpret the results of career assessments” (p. 571). Moreover, they noted that in analyzing studies published between 1983 and 1995, no treatment/control comparison studies addressed individual test interpretation (Whiston, Sexton, & Lasoff, 1998). We sought to address this gap in the literature with this study.
The authors wanted to learn more about the impact of these two different interpretive materials on SDS users. Did users remember their SDS results or codes? How much time did they spend with each of the materials? Did the materials help users expand their career options? Did users prefer one method over the other?
Method
Participants
The sample consisted of 51 undergraduate students enrolled in two sections of a college career course. Common reasons for enrolling in the course were to explore career options and learn more about career decision-making. Participant ages ranged from 18 to 26 years (M = 21.14, SD = 1.16). The demographic breakdown of the sample was 52.9% Caucasian, 31.4% African-American, 9.8% Hispanic/Latino, 3.9% other, and 2% Asian. Academic class was 62.7% seniors, 19.6% juniors, 15.7% sophomores, and 2% freshmen.
Procedures
After the university institutional review board (IRB) approved this study, each student in the course was given a folder during the first week of class containing an informed consent document, a demographic form, the paper-and-pencil version of the SDS Form R Assessment booklet, and the OF. Completion of the SDS was a basic course assignment, whereas participation in the study was optional. Those completing the forms were given five points of extra credit toward their grade for participating, which was 1% of the total points available in the course.
After the first week of classes, students in each section of the course were randomly assigned to receive either the client interpretive report (Group 1) generated by the SDS software portfolio computer system or the YYC and EOF booklets (Group 2). The Interpretive Reports were given after the SDS summary scores from each completed paper SDS assessment booklet were manually entered into the computer. Five days after students received the SDS results and interpretive materials, the SDS Feedback Form was administered to gather information about students’ satisfaction with the materials and their effectiveness.
Instruments
The Self-Directed Search Form R. The original paper form of the SDS was first published in 1970 and revised in 1977, 1985, and 1994 (Reardon & Lenz, 1998). The assessment is based on Holland’s RIASEC theory, is self-administered, and takes 35–45 minutes to complete. The cost per administration (Assessment booklet, OF, YYC booklet) is about $4.04 (PAR, 2009). The SDS Assessment booklet includes a measure of expressed interests or vocational aspirations (the Daydreams Section) and a measure of assessed interests. The latter is obtained when users respond to SDS items in four sections: Activities (66 items endorsed like or dislike); Competencies (66 items endorsed yes or no to assess skills assess); Occupations (84 occupations that are endorsed yes or no to assess those of interest); and Self-Estimates (12 scale ratings [1 is low and 7 is high] to indicate self-estimates of skills and abilities as compared to those of a similar age across each RIASEC type). A three-letter summary code is derived by totaling the number of “yes” or “like” responses from the three sections (Activities, Competencies, Occupations) and two Self-Estimates ratings in the Assessment booklet.
Intercorrelations among the SDS: Form R and measures of vocational aspiration and college major indicate concurrent validity for male and female college students ranging from .32 to .39 (Holland et al., 1994). Substantial reliability for the summary scales on the SDS are indicated by the internal consistency coefficients (KR-20) ranging from .90 to .94, and test-retest reliability coefficients ranged from .76 to .89 (Holland et al., 1994). Overall, support exists for both the reliability and validity of the SDS.
You and Your Career. YYC is a seven-page booklet developed for use in conjunction with the SDS Form R Assessment booklet. YYC includes descriptions of the Holland codes and personality types, suggestions for engaging in career planning, and information on how to use Holland codes. This reusable booklet gives the practitioner a tool for interpreting the results of the SDS assessment activity, and for providing a more complete career intervention (Reardon & Lenz, 1998). Although the YYC booklet was designed to make the SDS a more self-directed activity and support the practitioner’s work, little research has investigated the usefulness of this booklet (Holland et al., 1994). The YYC costs about $1.28 per copy (PAR, 2009).
Educational Opportunities Finder. The EOF was first published in 1987 as the College Majors Finder and lists more than 750 technical and college-level fields of study, alphabetically and by three-letter Holland code and degree level (2 year, 4 year, and postgraduate). It is designed to help individuals connect Holland codes to varied educational and training options. It costs about $2.16 and may be reused (PAR, 2009).
SDS Interpretive Report. The SDS: IR produces a 10- to 12-page single-spaced Interpretive Report based on an individual’s SDS summary scores and provides a personalized list of occupations using five SDS Summary codes. Each occupation listed includes the DOT number, estimated education needed, and on-the-job training required. The IR also includes major fields of study and leisure options. The Interpretive Report is produced by the software portfolio system that costs $525 for unlimited use (PAR, 2009).
SDS Feedback Form. The items on the SDS Feedback Form were adapted from the Computer-Assisted Career Guidance (CACG) Evaluation Form (Peterson, Ryan-Jones, Sampson, & Reardon, 1988) used in studies of computer-based guidance systems (e.g., Discover, SIGI). The original CACG Evaluation Form had alpha reliabilities ranging from .77 to .87. The SDS Feedback Form included open-ended questions assessing how much time participants spent reading their materials, whether they accurately recalled their three-letter summary code, and the information learned by reading the materials. Four specific items from the Feedback Form were used to assess participants’ ability to expand or narrow their options.
Occupational Alternatives Question. The Occupational Alternatives Question (OAQ; Zener & Schnuelle, 1972, 1976; modified by Slaney, 1980) is a measure of occupational decidedness that asks respondents to list the number of occupations they are considering and the level of decidedness pertaining to these occupations. The OAQ includes two parts: (a) “List all of the occupations you are considering right now” and (b) “Which occupation is your first choice? If undecided, write undecided.” The OAQ is scored on a scale from one to four and is rated as follows: 1 = a first choice is given with no alternatives; 2 = a first choice is given with alternatives listed as well; 3 = no first choice is given, only alternatives; and, 4 = no choices or alternatives are given. The higher the OAQ score, the less decided the individual. The OAQ has been found to have convergent validity with other measures of career indecision, including the Satisfaction with Career Scale, the Vocational Decision Making Difficulties Scale, and the Career Decision Scale (Slaney, Stafford, & Russell, 1981; Walker & Peterson, 2012).
Satisfaction with Choice Question. The Satisfaction with Choice Question (SCQ; Zener & Schnuelle, 1972, 1976; modified by Holland, Gottfredson, & Nafziger, 1975) asks a single question, “How well satisfied are you with your first choice?” and is used to assess one’s level of satisfaction with career choice. This item is rated on a scale from one to six, and is scored as follows: 1 = well satisfied with choice; 2 = satisfied, but have a few doubts; 3 = not sure; 4 = dissatisfied and intend to remain; 5 = very dissatisfied and intend to change; and, 6 = undecided about my future career. Higher scores on the SCQ indicate greater dissatisfaction with career choice. Slaney et al. (1981) reported average correlations of .43, .53 and .44 between the SCQ and other measures of career decidedness, including the OAQ, Vocational Decision Making Difficulty Scale, and the Career Decision Scale.
Student Data Sheet. A demographic form was used to collect basic information on each participant, including gender, age, ethnicity, major, grade level, and career decision state. The latter was measured with the OAQ and SCQ as measures of participants’ career decision state according to the level of career decidedness and satisfaction assessment. Preliminary t-tests indicated that the groups did not vary across age or gender, but Group 2, which received the YYC and EOF booklets, included more seniors and ethnic diversity.
Results
Several analyses were conducted to investigate participants’ ability to recall their SDS summary codes. For example, both 4 x 2 chi-square analyses and several 2 x 2 chi-square analyses were used to compare participants’ actual summary code and their recall of their summary code. Participants in Group 1 who received the computer-generated, individualized interpretive report were significantly more likely to accurately recall their overall summary code (84%), as compared with participants in Group 2 (YYC/EOF booklets) (61.5%, p < .05). Participants in Group 1 were significantly more likely to recall the first and third letters of their summary codes (84% and 88%), compared with participants in Group 2 (61.5% and 50%, p < .017), but there were no significant differences between the groups on the ability to recall the second letter of the summary code.
Using items from the SDS Feedback Form, a series of 3 x 2 chi-square analyses were conducted to determine participants’ ability to expand, elaborate, narrow or confirm their occupational alternatives. Participants in Group 1 (interpretive report) were significantly more likely to expand their career options than those in Group 2 (72% vs. 57.6%, p < .05). However, participants in the two groups showed no significant differences in their ability to elaborate, confirm or narrow their options.
A one-way multivariate analysis of variance (MANOVA) was performed to investigate group differences between two aspects of career decision state. The independent variable was treatment (Groups 1 and 2), and the dependent variable was career decision state, as measured by the OAQ level of decidedness and the response to the SCQ. There were no statistically significant differences noted between treatment groups with respect to career decision state (decidedness and satisfaction) following use of the SDS.
Finally, two independent-sampled t-test analyses were conducted to compare the impact of receiving different SDS:IR materials on the amount of time (calculated in minutes) participants spent with the interpretive materials and the number of times participants picked up the materials to review them. There were no significant differences between groups regarding either the amount of time or number of times spent reading the SDS interpretive materials. Although there were no differences between the two groups on the amount of time spent reading the interpretive materials, participants in both groups spent about 30 minutes reviewing the materials, with a range of 5 to 90 minutes and a standard deviation of nearly 20 minutes. The variability in the amount of time that participants spent examining the SDS interpretive materials is noteworthy and merits further research.
Discussion
The findings in this study indicate that individuals receiving the SDS:IR generated by the SDS software portfolio computer system were more likely to recall their SDS overall three-letter summary code, as well as the first and third letters of their code, than those receiving the YYC and the EOF booklets. Because the SDS software portfolio generates a more customized and individualized interpretive report, the findings of this study are consistent with Brown and Ryan Krane’s (2000) recommendations for providing individualized interpretations and feedback regarding career development interventions. These results are also consistent with previous research, which demonstrated that the SDS increases self-understanding (Zener & Schuelle, 1976). In addition, Tinsley and Chu (1999) indicated that the recall of test results is one method of assessing the effects of a test interpretation.
Findings reveal that the interpretive materials used in this study enabled students to expand their options, but no significant relationships (p < .05) were found regarding their ability to narrow or confirm their options. The SDS results examine matches between the users’ three-letter code and the codes of more than 1,300 occupations in the OF, so it is reasonable that the SDS helped individuals expand their options (Holland, 1997).
Several limitations in this study may affect the generalizability of these findings. First, the sample was small (n = 51) and limited to two sections of an undergraduate career class in one university. The sample was dominated by Social and Enterprising types (64%). Another study with a sample composed of more varied RIASEC types might produce different results. Finally, t-tests were conducted to investigate the homogeneity of groups, and differences were found between class level and ethnicity, with Group 2 (OF/EOF printed booklets) having more seniors and more ethnic diversity. Given that members of this group were less likely to recall their code, it is possible that some seniors were not as focused on exploring their occupational interests and may have been primarily focused on resume writing and job searching.
These limitations notwithstanding, this study adds to the very limited research literature reported by Whiston and James (2013) regarding how practitioners might best interpret the results of career assessments. This study used two different interpretation materials for the SDS, two printed booklets and a computer-based individualized report. The computer-based interpretive report enabled users to recall their results more accurately in a follow-up survey after completing the SDS, and it led users to consider a more expanded list of career options for further consideration. Future studies might examine the efficacy of new interpretive materials now being developed for the SDS, as well as interpretive materials in varied media formats for other career assessments. Such studies would alleviate the paucity of research in this area noted by Whiston and James (2013).
References
Brown, S. D., & Ryan Krane, N. E. (2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In S. D. Brown, & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 740–766). New York, NY: Wiley.
Holland, J. L. (1994a). Self-Directed Search. Odessa, FL: Psychological Assessment Resources.
Holland, J. L. (1994b). You and your career. Odessa, FL: Psychological Assessment Resources.
Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Odessa, FL: Psychological Assessment Resources.
Holland, J. L. (2000). The occupations finder. Odessa, FL: Psychological Assessment Resources.
Holland, J. L., Fritzsche, B., & Powell, A. (1994). The self-directed search: Technical manual. Odessa, FL: Psychological Assessment Resources.
Holland, J., Gottfredson, G., & Nafziger, D. (1975). Testing the validity of some theoretical signs of vocational decision-making ability. Journal of Counseling Psychology, 22, 411–422.
Peterson, G. W., Ryan-Jones, R. E., Sampson, J. P., & Reardon, R. C. (1988). Computer-assisted career guidance evaluation form. Tallahassee, FL: Florida State University.
Psychological Assessment Resources. (PAR; July 2009). Creating connections, changing lives. PAR catalog of professional testing resources, 32(3).
Reardon, R. (1987). Development of the computer-version of the Self-Directed Search. Measurement and Evaluation in Counseling and Development, 20, 62–67.
Reardon, R. C., & Lenz, J. G. (1998). The Self-Directed Search and related Holland career materials: A practitioner’s guide. Odessa, FL: Psychological Assessment Resources.
Reardon, R., & Psychological Assessment Resources Staff (2001). Self-Directed Search Software Portfolio (SDS-SP) for Windows[Computer program]. Odessa, FL: Psychological Assessment Resources.
Rosen, D., Holmberg, K., & Holland, J. (1997). Educational opportunities finder. Odessa, FL: Psychological Assessment Resources.
Ruff, E. A., Reardon, R. C., & Bertoch, S. C. (2008, June). Holland’s RIASEC theory and applications: Exploring a comprehensive bibliography. Career Convergence. Retrieved September 25, 2011 from http://ncda.org/aws/NCDA/pt/sd/news_article/5483/_self/layout_ccmsearch/false
Slaney, R. (1980). Expressed vocational choice and vocational indecision. Journal of Counseling Psychology, 27, 122–129.
Slaney, R., Stafford, M., & Russell, J. (1981). Career indecision in adult women: A comparative and descriptive study. Journal of Vocational Behavior, 19, 335–345.
Tinsley, H. E. A., & Chu, S. (1999). Research on test and interest inventory interpretation outcomes. In M. L. Savickas & A. R. Spokane (Eds.), Vocational interests: Meaning, measurement, and counseling use (pp. 257–276). Palo Alto, California: Davies-Black.
Walker, J. V., & Peterson, G.W. (2012). Career thoughts, indecision, and depression: Implications for mental health assessment in career counseling. Journal of Career Assessment, 20(4), 497–506.
Whiston, S. C., & James, B. N. (2013). Promotion of career choices. In S. D. Brown & R. W. Lent (Eds.), Career development and counseling: Putting theory and research to work (2nd ed., pp. 565–594). Hoboken, NJ: Wiley.
Whiston, S. C., Sexton, T. L., & Lasoff, D. L. (1998). Career intervention and outcome: A replication and extension of Oliver and Spokane (1988). Journal of Counseling Psychology, 45, 150–165. doi:10-1037/0022-0167.45.2.150
Zener, T., & Schnuelle, L. (1972). An evaluation of the Self-Directed Search. (Research Rep. 124). Baltimore, MD: Johns Hopkins University, Center for Social Organization of Schools.
Zener, T., & Schnuelle, L. (1976). Effects of the Self-Directed Search on high school students. Journal of Counseling Psychology, 23, 353–359.
V. Casey Dozier, NCC, is a post-doctoral fellow at the About Family and Youth Institute in Thomasville, Georgia. James P. Sampson, NCC, NCCC, is the Mode L. Stone Distinguished Professor of Counseling and Career Development, Associate Dean for Faculty Development and Administration, and Co-Director, Center for the Study of Technology in Counseling and Career Development, College of Education, Florida State University. Robert C. Reardon, NCC, is Professor Emeritus at Florida State University. Correspondence can be addressed to Robert C. Reardon, FSU Career Center, P.O. Box 3064162, 100 South Woodward Avenue, Tallahassee, FL 32306-4162,
rreardon@fsu.edu.
Author Note: Robert C. Reardon is an author of the SDS software report used in this research.
