check out drink and drugs

Significant Change


Technical information :: clinically significant change

The 'gold standard' of addiction treatment outcome: statistically sophisticated and yet easily understood


Why is clinically significant change important?

The use of change scores, for example the difference between pre-treatment and post treatment scores, is an inadequate method of determining change because: i) it fails to take account of the level of pre-treatment scores – someone with a low score has less scope to change than someone with a high score; ii) it fails to take account of the measurement error of the instrument used to measure the change; iii) it fails to signify how meaningful the difference is.

The calculation of clinically significant change allocates treatment outcomes into one of four categories:

  1. Reliable deterioration

  2. No reliable change

  3. Reliable improvement

  4. Reliable and clinically significant improvement

Clinically significant change requires that two criteria are met in order to assert that meaningful change has occurred.  These are:

  1. that the magnitude of change has to be statistically reliable

  2. that individuals end up in a range that renders them indistinguishable from well functioning people.

Of course many people with addiction problems also function reasonably well for much of the time. Published values for calculating clinically significant change is one of the criteria used in the Quality Framework assessment.

The calculation

The calculation of statistically reliable change takes account of the reliability of the measurement instrument and thus avoids the pitfall of interpreting measurement error as change.  The formula is:

Formula RC 300px.png
  • RC = reliable change value

  • sd = standard deviation of pretreatment responses

  • r = test retest relaibility of the measurement instrument

Three methods for fulfilling the second criterion for clinically significant change, which have varying degrees of stringency, are proposed:

  1. scores that fall outside the range of the dysfunctional behaviour where this is described as being two standard deviations in the direction of improvement

  2. scores fall within the functional range where this is set at two standard deviations from the mean score for the normal population

  3. scores indicating the level of functioning suggest the service user is statistically more likely to be in the functional than the dysfunctional population 

Option 3 is the preferred method.  

Formula FP.png
  • FP = cut-off value for functional population

  • M = mean score

  • sd = standard deviation

  • p = problem population (pre any intervention)

  • f = functional population

So, each individual must change by at least the reliable change value, and by ending up closer to the mean score for the functioning population than the dysfunctional population in order to have achieved clinically significant change. If a pre-treatment score is within the parameters for ‘well functioning people’ then only reliable change, not clinically significant change, can be achieved.  For this reason these individuals should be excluded from estimates of clinically significant change.

If you are interested in technical information you may want to…

Scientific articles on clinically significant change

J personality assessment.jpeg

Bauer S, Lambert MJ, and Nielsen SL (2004) Clinical Significance Methods:  A Comparison of Statistical Techniques. Journal of Personality Assessment 82: 60-70

DOI: 10.1207/s15327752jpa8201_11

J personality assessment.jpeg

Wise, EA (2004). Methods for analyzing psychotherapy outcomes: A review of clinical significance, reliable change, and recommendations for future directions. Journal of Personality Assessment 82: 50-59

DOI: 10.1207/s15327752jpa8201_10

J Con & Clin Psychol.gif

Jacobson NS, Roberts LJ, Berns SB and McClinchey JB (1999) Methods for defining and determining the clinical significance of treatment effects: Description, application and alternatives. Journal of Consulting and Clinical Psychology 67: 300-307

DOI: 10.1037//0022-006x.67.3.300

J Con & Clin Psychol.gif

Jacobson NS and Truax P (1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59: 12-19

DOI: 10.1037//0022-006x.59.1.12