TPC-Journal-V6-Issue2

The Professional Counselor /Volume 6, Issue 2 130 than the kappa produced using unadjusted Rule 1 and 3 cut-offs (κ = .423). The adjusted process identified 161 of the 181 (sensitivity = .89) participants whom the counselors classified as having an SUD. However, this increased sensitivity came at the cost of decreased specificity. The adjusted process identified only 33 (specificity = .55) of those participants whom the counselors determined did not have an SUD. The false positive rate and the false negative rate for this adjusted process were, respectively .45 and .11. In sum, this process increased the number of true positives by 20, decreased the number of true negatives by 8, increased the number of false positives by 8, and decreased the number of false negatives by 20. As one might expect, lowering the cut scores on these two rules increased the instrument’s ability to detect the presence of problems, but did so at the cost of possibly overdiagnosing 8 (3%) additional participants while reducing the false negative classifications by 20 (8.3%). Discussion The DSM-5 section on SUDs includes significant changes. Chief among these changes is the movement away from an abuse/dependence dichotomy to an SUD continuum that includes all of the criteria previously unique to abuse and dependence disorders as well as the addition of a craving criterion. The present study examined the SASSI-3’s utility in predicting counselors’ diagnostic classifications using the new DSM-5 SUD criteria. The results provided a mixed picture. The SASSI- 3’s agreement with the counselors’ diagnoses was moderate. This finding prompted us to conduct a similar series of kappa analyses for each of the SASSI-3’s decision rules and ROC analyses for the first five SASSI-3 decision rules. The last four decision rules could not be analyzed with the ROC as they are each composed of more than one scale of the SASSI-3. The decision rules’ agreement with the counselors’ diagnoses varied considerably. The kappa values presented in Table 1 are below what would be expected based on previously published agreement statistics using previous versions of the DSM (Miller & Lazowski, 1999). The SASSI-3 and its decision rules’ false negative values suggested that the instrument’s modest agreement with the counselors may have been a consequence of unnecessarily high raw score cut-off points. Consistent with Clements’ (2002) findings related to adjusting cut scores, the ROC score analyses presented mixed results. The ROC analyses provided evidence that lowered FVA and SYM cut scores improved these scales’ respective sensitivity and specificity estimates. The FVOD scale’s current cut score produced high sensitivity and specificity and did not need to be improved. The OAT and SAT cut scores could not be adjusted without unwanted compromises to either scale’s associated decision rules’ sensitivity and specificity. The SASSI-3’s overall decision was recalculated using the lowered Rule 1 and Rule 3 cut scores. This process resulted in an improvement in sensitivity with a slight decrease in specificity. The net result was an improvement in the SASSI-3’s overall agreement with licensed counselors’ SUD determinations. Our FVOD scale’s sensitivity and specificity findings are consistent with those of First et al. (1997) and Lazowski et al. (1998), and suggest that the FVOD scale is useful in predicting DSM-IV-TR and DSM-5 non-alcohol SUDs. Our FVA scale findings are consistent with those of First et al. (1997) but differ from those of Lazowski et al. (1998). There are no other SASSI-3 ROC analyses available for comparison. These results elicit deliberation about whether SUD counselors would be better served by an SUD screening instrument that over- or under-predicts SUD diagnoses. In the case of a scoring method that produces higher sensitivity but lower specificity, resource allocation might be a concern. A counselor’s diagnostic time might be unnecessarily spent ruling out clients, and clients might be unnecessarily inconvenienced by participating in a full SUD assessment. Alternatively, counselors using a scoring method with lower sensitivity but higher specificity would have fewer clients unnecessarily inconvenienced and spend less time assessing persons who do not need SUD