TECHNICAL INFORMATION ::: clinically significant change
the 'gold standard' of addiction treatment outcome -
statistically sophisticated and yet easily understood

Clinically Significant Change

The calculation of Clinically Significant Change is straightforward and depends on scale developers providing two required values (see below) - treatment outcomes then fall into one of four categories:

Clin sig change.png

Reliable deterioration

No reliable change

Reliable improvement

Reliable and clinically significant improvement

The use of change scores, for example the difference between pre-treatment and post treatment scores, is an inadequate method of determining change because: i) it fails to take account of the level of pre-treatment scores โ€“ someone with a low score has less scope to change than someone with a high score; ii) it fails to take account of the measurement error of the instrument used to measure the change; iii) it fails to signify how meaningful the difference is.

Clinically significant change requires that two criteria are met in order to assert that meaningful change has occurred.  These are:

  1. that the magnitude of change has to be statistically reliable

  2. that individuals end up in a range that renders them indistinguishable from well functioning people.

Of course many people with addiction problems also function reasonably well for much of the time. Clinically Significant Change is one of the criteria used in the Quality Framework assessment.

 

The Calculation

The calculation of statistically reliable change takes account of the reliability of the measurement instrument and thus avoids the pitfall of interpreting measurement error as change.  The formula is:

Formula RC 300px.png
  • RC = reliable change value

  • sd = standard deviation of pretreatment responses

  • r = test retest relaibility of the measurement instrument

Three methods for fulfilling the second criterion for clinically significant change, which have varying degrees of stringency, are proposed:

  1. scores that fall outside the range of the dysfunctional behaviour where this is described as being two standard deviations in the direction of improvement

  2. scores fall within the functional range where this is set at two standard deviations from the mean score for the normal population

  3. scores indicating the level of functioning suggest the service user is statistically more likely to be in the functional than the dysfunctional population 

Option 3 is the preferred method.  

Formula FP.png
  • FP = cut-off value for functional population

  • M = mean score

  • sd = standard deviation

  • p = problem population (pre any intervention)

  • f = functional population

So, each individual must change by at least the reliable change value, and by ending up closer to the mean score for the functioning population than the dysfunctional population in order to have achieved clinically significant change. If a pre-treatment score is within the parameters for โ€˜well functioning peopleโ€™ then only reliable change, not clinically significant change, can be achieved.  For this reason these individuals should be excluded from estimates of clinically significant change.

 
Clin sig change.png

Statistical descriptions of measuring clinically significant change:

Bauer S, Lambert MJ, and Nielsen SL (2004) Clinical Significance Methods:  A Comparison of Statistical Techniques. Journal of Personality Assessment 82: 60-70    DOI: 10.1207/s15327752jpa8201_11

Wise, EA (2004). Methods for analyzing psychotherapy outcomes: A review of clinical significance, reliable change, and recommendations for future directions. Journal of Personality Assessment 82: 50-59   DOI: 10.1207/s15327752jpa8201_10

Jacobson NS, Roberts LJ, Berns SB and McClinchey JB (1999) Methods for defining and determining the clinical significance of treatment effects: Description, application and alternatives. Journal of Consulting and Clinical Psychology 67: 300-307    PMID: 10369050

Jacobson NS and Truax P (1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59: 12-19    PMID: 2002127

Addiction examples with real data:

Raistrick DS, Tober GW, Sweetman J, Unsworth S, Crosby H, & Evans T (2014) Measuring clinically significant outcomes โ€“ LDQ, CORE-10, and SSQ as dimension measures of addictionThe Psychiatrist 38: 112-115   DOI: 10.1192/pb.bp.112.041301

Tober GW (2000) The nature and measurement of change in substance dependence, University of Leeds, PhD Thesis