TPC Journal V7, Issue 1-FULL ISSUE

92 The Professional Counselor | Volume 7, Issue 1 errata and advertisements excluded) during the specified time period to identify relevant articles and provide the total number of articles published for each journal. Journals not indexed within these electronic databases were searched by reading the electronic version of each issue’s table of contents and article abstracts and keywords. Following this process, the first author met with the graduate assistants in order to reconcile any differences between their lists of applicable texts and total number of journal articles found. A preliminary list of 226 articles was identified and reviewed by each author to determine suitability for the study. Fifteen articles were removed from the analysis because they did not discuss addictions or addictions treatment; for example, two articles included the keyword “substance abuse” in the abstract, but not in the article itself. To maintain independence, one article was removed because it had been published twice. The authors developed a content analysis protocol that included definitions and categories for each coding variable. To address research questions 1–3, the following variables were developed: (a) addiction-related content topic, (b) addictive behavior and (c) type of research article. In order to assess research question 4, (d) data analytic technique and (e) research population also were included as variables. Categories of each variable were initially developed by the authors based on their knowledge of the addictions literature as well as past content analysis research of counseling journals. The authors then pretested the protocol by coding 40 randomly selected articles within the sample (approximately 20%) to purify the coding scheme and conduct a preliminary assessment of coder agreement. Following this process, the authors met to refine existent category definitions, agree on the inclusion of additional variable categories and determine which variables would be single versus multiple classification. High inter-rater agreement (85% or higher) across all five study variables was observed among the three coders during the pilot phase. A pattern was not observed in the disagreements among the coders, suggesting that the framework possessed acceptable construct validity (Insch, Moore, & Murphy, 1997). Once the protocol was refined, all articles were coded independently by two members of the research team (first, second, and third authors). Krippendorff’s alpha (Krippendorff, 2013), with a minimum acceptable value set at α = .80, was utilized to assess agreement among the coders. This coefficient was selected to capture inter-rater reliability because it estimates error in observed agreement attributable to chance and accounts for small sample sizes. Using a reliability measure designed for small samples was an important consideration because research question 4 relates only to a smaller subset of the sample used in the study. Further, the use of two coders for each article was to ensure that the total number of observations for each variable in this study exceeded the minimum number recommended by Krippendorff (2013) for an alpha value greater than .80 at the .01 level of significance (i.e., according to Krippendorff [2013] two coders would have to code at least 103 units). The “odd-man-out” procedure recommended by Insch et al. (1997), in which a third coder determined the final category when differences emerged, was used to reconcile disagreements between coders. Coding Variables Five variables were identified by coders in the study. All articles ( N = 210) were coded utilizing three of the variables: addiction-related content topic, addictive behavior and type of research. Two variables were used to code research articles identified within the sample. Percentage of agreement (observed agreement [OA]) and Krippendorff’s alpha ( α ) were calculated for each variable. Addiction-related content topic. The purpose of this variable was to identify the area of addictions counseling and research that the article addressed. Categories were initially drawn from content analyses by Ray et al. (2011) and Moro et al. (2016) and modified for the present study. Because the purpose of this variable was to identify the main content focus of the articles, a single classification