Technical information :: clinically significant change
The 'gold standard' of addiction treatment outcome: statistically sophisticated and yet easily understood
Why is clinically significant change important?
The use of change scores, for example the difference between pre-treatment and post treatment scores, is an inadequate method of determining change because: i) it fails to take account of the level of pre-treatment scores – someone with a low score has less scope to change than someone with a high score; ii) it fails to take account of the measurement error of the instrument used to measure the change; iii) it fails to signify how meaningful the difference is.
The calculation of clinically significant change allocates treatment outcomes into one of four categories:
No reliable change
Reliable and clinically significant improvement
Clinically significant change requires that two criteria are met in order to assert that meaningful change has occurred. These are:
that the magnitude of change has to be statistically reliable
that individuals end up in a range that renders them indistinguishable from well functioning people.
Of course many people with addiction problems also function reasonably well for much of the time. Published values for calculating clinically significant change is one of the criteria used in the Quality Framework assessment.
The calculation of statistically reliable change takes account of the reliability of the measurement instrument and thus avoids the pitfall of interpreting measurement error as change. The formula is:
RC = reliable change value
sd = standard deviation of pretreatment responses
r = test retest relaibility of the measurement instrument
Three methods for fulfilling the second criterion for clinically significant change, which have varying degrees of stringency, are proposed:
scores that fall outside the range of the dysfunctional behaviour where this is described as being two standard deviations in the direction of improvement
scores fall within the functional range where this is set at two standard deviations from the mean score for the normal population
scores indicating the level of functioning suggest the service user is statistically more likely to be in the functional than the dysfunctional population
Option 3 is the preferred method.
FP = cut-off value for functional population
M = mean score
sd = standard deviation
p = problem population (pre any intervention)
f = functional population
So, each individual must change by at least the reliable change value, and by ending up closer to the mean score for the functioning population than the dysfunctional population in order to have achieved clinically significant change. If a pre-treatment score is within the parameters for ‘well functioning people’ then only reliable change, not clinically significant change, can be achieved. For this reason these individuals should be excluded from estimates of clinically significant change.
If you are interested in technical information you may want to…
Scientific articles on clinically significant change
Wise, EA (2004). Methods for analyzing psychotherapy outcomes: A review of clinical significance, reliable change, and recommendations for future directions. Journal of Personality Assessment 82: 50-59
Jacobson NS, Roberts LJ, Berns SB and McClinchey JB (1999) Methods for defining and determining the clinical significance of treatment effects: Description, application and alternatives. Journal of Consulting and Clinical Psychology 67: 300-307
Jacobson NS and Truax P (1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59: 12-19