Clinically Significant Change
The use of simple change scores, for example the difference between pre-treatment and post treatment means, is an inadequate method of determining change because: i) it fails to take account of the level of pre-treatment scores – someone with a low score has less scope to change than someone with a high score; ii) it fails to take account of the measurement error of the questionnaire; iii) it fails to signify how meaningful the difference is.
What is clinically significant change and why is it important?
What does it deliver?
The calculation of clinically significant change allocates treatment outcomes into one of four categories:
◼︎Reliable deterioration
◼︎No reliable change
◼︎Reliable improvement
◼︎Reliable and clinically significant improvement
This provides an easily understood picture of otcomes from a statistically sophisticated methodology
The requirements
Clinically significant change requires that two criteria are met in order to assert that meaningful change has occurred. These are:
◼︎The magnitude of change has to be statistically reliable
◼︎Individuals end up in a range that renders them indistinguishable from well functioning people
People with addiction problems often function well in several areas of their lives
How is clinically significant change calculated?
Each individual must change by at least the reliable change value, and by ending up closer to the mean score for the functioning population than the problem population in order to have achieved clinically significant change. If a pre-treatment score is within the parameters for ‘well functioning people’ then only reliable change, not clinically significant change, can be achieved. For this reason these individuals should be excluded from estimates of clinically significant change.
Step1 - is there reliable change?
RC = reliable change value
This is the minimum change in score that must be achieved in order to say that the change is genuine.
sd = standard deviation of pretreatment responses
r = test retest reliability of the measurement instrument
Step 2 - is change meaningful?
FP = cut-off value for functional population
Three methods for fulfilling the second criterion for clinically significant change, which have varying degrees of stringency, are proposed:
scores that fall outside the range of the dysfunctional behaviour where this is described as being two standard deviations in the direction of improvement
scores fall within the functional range where this is set at two standard deviations from the mean score for the normal population
scores indicating the level of functioning suggest the service user is statistically more likely to be in the functional than the dysfunctional population - this is the preferred method
M = mean score
sd = standard deviation
p = problem population (pre any intervention)
f = functional population
Clinically significant change - scientific articles
Jacobson NS and Truax P (1991) Clinical Significance: A Statistical Approach to Defining Meaningful Change in Psychotherapy Research. Journal of Consulting and Clinical Psychology 59: 12-19
Lunnen KM and Ogles BM (1998) A Multiperspective, Multivariable Evaluation of Reliable Change. Consulting and Clinical Psychology 66: 400-410
Jacobson NS, Roberts LJ, Berns SB, and McGlinchey JB (1999) Methods for Defining and Determining the Clinical Significance of Treatment Effects: Description, Application, and Alternatives. Consulting and Clinical Psychology 67: 300-307
Baur S, Lambert MJ and Nielsen SL (2004) Clinical significance methods: A Comparison of Statistical Techniques. Journal of Personality Assessment 82: 60-70
Wise E (2010) Methods for Analyzing Psychotherapy Outcomes: A Review of Clinical Significance, Reliable Change, and Recommendations for Future Directions. Journal of Personality Assessment 82: 50-59
More pages about outcomes measurement…
Check out these measures of: substance use, dependence, recovery, wellbeing, mental health and governance