TECHNICAL INFORMATION ::: clinically significant change
the 'gold standard' of addiction treatment outcome -
statistically sophisticated and yet easily understood
Clinically Significant Change
The calculation of Clinically Significant Change is straightforward and depends on scale developers providing two required values (see below) - treatment outcomes then fall into one of four categories:
No reliable change
Reliable and clinically significant improvement
The use of change scores, for example the difference between pre-treatment and post treatment scores, is an inadequate method of determining change because: i) it fails to take account of the level of pre-treatment scores – someone with a low score has less scope to change than someone with a high score; ii) it fails to take account of the measurement error of the instrument used to measure the change; iii) it fails to signify how meaningful the difference is.
Clinically significant change requires that two criteria are met in order to assert that meaningful change has occurred. These are:
that the magnitude of change has to be statistically reliable
that individuals end up in a range that renders them indistinguishable from well functioning people.
Of course many people with addiction problems also function reasonably well for much of the time. Clinically Significant Change is one of the criteria used in the Quality Framework assessment.
The calculation of statistically reliable change takes account of the reliability of the measurement instrument and thus avoids the pitfall of interpreting measurement error as change. The formula is:
RC = reliable change value
sd = standard deviation of pretreatment responses
r = test retest relaibility of the measurement instrument
Three methods for fulfilling the second criterion for clinically significant change, which have varying degrees of stringency, are proposed:
scores that fall outside the range of the dysfunctional behaviour where this is described as being two standard deviations in the direction of improvement
scores fall within the functional range where this is set at two standard deviations from the mean score for the normal population
scores indicating the level of functioning suggest the service user is statistically more likely to be in the functional than the dysfunctional population
Option 3 is the preferred method.
FP = cut-off value for functional population
M = mean score
sd = standard deviation
p = problem population (pre any intervention)
f = functional population
So, each individual must change by at least the reliable change value, and by ending up closer to the mean score for the functioning population than the dysfunctional population in order to have achieved clinically significant change. If a pre-treatment score is within the parameters for ‘well functioning people’ then only reliable change, not clinically significant change, can be achieved. For this reason these individuals should be excluded from estimates of clinically significant change.
Statistical descriptions of measuring clinically significant change:
Bauer S, Lambert MJ, and Nielsen SL (2004) Clinical Significance Methods: A Comparison of Statistical Techniques. Journal of Personality Assessment 82: 60-70 DOI: 10.1207/s15327752jpa8201_11
Wise, EA (2004). Methods for analyzing psychotherapy outcomes: A review of clinical significance, reliable change, and recommendations for future directions. Journal of Personality Assessment 82: 50-59 DOI: 10.1207/s15327752jpa8201_10
Jacobson NS, Roberts LJ, Berns SB and McClinchey JB (1999) Methods for defining and determining the clinical significance of treatment effects: Description, application and alternatives. Journal of Consulting and Clinical Psychology 67: 300-307 PMID: 10369050
Jacobson NS and Truax P (1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59: 12-19 PMID: 2002127
Addiction examples with real data:
Raistrick DS, Tober GW, Sweetman J, Unsworth S, Crosby H, & Evans T (2014) Measuring clinically significant outcomes – LDQ, CORE-10, and SSQ as dimension measures of addiction. The Psychiatrist 38: 112-115 DOI: 10.1192/pb.bp.112.041301
Tober GW (2000) The nature and measurement of change in substance dependence, University of Leeds, PhD Thesis