Original ArticleThe minimal detectable change cannot reliably replace the minimal important difference
Introduction
The minimal important difference (MID) is the smallest change in a health-related outcome measure which, in association with minimal toxicity and cost, is large enough to trigger a change in management [1], [2]. The MID may be determined by the patients in quality-of-life measures, or by clinicians in disease-activity indices.
Investigators have used distribution-based and anchor-based methods to ascertain the MID [3]. Distribution-based approaches rely on statistical properties of the instrument without reference to an external standard. Anchor-based approaches use an external criterion (i.e., anchor) to interpret whether a particular magnitude of change is significant.
Crosby et al. summarized the statistical methods to establish the distribution- and anchor-based MID estimates, using a variety of cross-sectional and longitudinal anchors [4]. Distribution-based approaches include effect size statistics (including Cohen's effect size [5] and standardized response mean [6]), variations of the standard error of measurement (Wyrwich SEM [7] and Jacobson's reliable change index [8]), and Bland and Altman's 95% limits of agreement [9] (Table 1). Other proposed modifications of the reliable change index are not in common use [4].
The major role of the distribution-based strategies is in identifying the “minimum detectable change” (MDC, sometimes also termed the “smallest detectable difference”), which is the smallest change in score that can be detected beyond random error [22], [23], [24]. The MDC depends on the number of observers, the scoring methods, and the distribution of results in the population under study. Authors have suggested varying thresholds to characterize the MDC from the distribution-based methods (Table 1).
For the anchor-based approach, the most widely used external criterion is a global rating of change [25]. The original statistical strategy to ascertain the MID from a global rating of change, calculated the mean change of patients who rated themselves as having a small change [1]. More recently, the receiver-operating characteristic (ROC) curve analysis, instead of the mean change, is being used to determine the change score from the global rating of change [10], [26], [27], [28], [29], [30], [31], [32], [33].
The MID values may be derived from small change (e.g., 2–3 on a −7 to +7 point global rating of change [2], [25], [34], [35]) as originally proposed. However, investigators have also used a cutoff of moderate change to define the MID (e.g., 4–5 on −7 to +7 point global rating of change [30], [31], [36], [37] or 2 on a −3 to +3 point global rating of change [18], [29], [38], [39]). The optimal threshold is yet to be established. For this presentation, we have assumed that clinimetric indices constructed of only a few items, each representing a different aspect of the phenomenon, require a larger MID than psychometric instruments that are composed of a large number of closely related items. Indeed, this is how the original MID values were determined for the four included instruments in this study. A change in one or two items in psychometric scales would be diluted by the other many equally important items, thereby resulting in a smaller overall change as compared with the clinimetric index that each item (especially the highly weighted ones) contributed significantly to the overall score. Therefore, a small MID may prove significant in the psychometric scales but not necessarily in the clinimetric indices.
The major disadvantage of the distribution-based approaches in determining the MID is that they provide no guidance regarding the importance of the change [18]. Although the anchor-based approach is the preferred strategy to ascertain the MID, researchers may wish to use the MDC for estimating the MID in studies that did not use anchors. Here, we examine the relation between commonly used anchor- and distribution-based methods using four different health-related outcome measures (two disease-activity clinimetric indices, and two quality-of-life psychometric scales), to establish whether the MDC can reliably estimate the MID in the absence of external anchor of change.
Section snippets
Population
Four prospectively collected longitudinal data sets of the following well-established instruments were used: the Pediatric Ulcerative Colitis Activity Index (PUCAI) [38], the Pediatric Crohn's Disease Activity Index (PCDAI) [29], [40], the Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ) [41], and the Chronic Respiratory Questionnaire (CRQ) [2], [42].
On the 7-point global rating of change (−3 to +3) used in the clinimetric indices (PUCAI and PCDAI), “0” was referred to as “no change,”
Results
The correlations between change in the relevant instruments/domains and the global ratings were 0.50 for the RQLQ, 0.61 for the dyspnea domain of the CRQ, 0.50 for the fatigue domain of the CRQ, 0.53 for the emotions domain of the CRQ, 0.72 for the PCDAI, and 0.85 for the PUCAI. All included indices and domains also satisfied criteria for the correlation of the baseline and the follow-up scores with the global rating of change.
Within the anchor-based methods, the ROC approach yielded values
Discussion
This systematic comparison of distribution- and anchor-based strategies has demonstrated that although there is some relation between the two approaches, the relation is inconsistent.
In interpreting the results, it is important to recognize two major differences between the psychometric instruments (CRQ and RQLQ) and the clinimetric indices (PUCAI and PCDAI). First, clinimetric and psychometric instruments are constructed in a very different way [47]. Classic psychometric measures are composed
Conclusion
The values of MID and MDC measure different concepts; the former measures important apparent change and the latter statistical distribution of margins of error. No distribution-based approach bears a consistent relationship with the anchor-based MID that should be regarded as the ultimate strategy to ascertain the MID. One could argue that, as a result, one should eschew the use of any distribution-based estimate of the MID. This, however, limits attempts to enhance the interpretability of the
References (58)
- et al.
Measurement of health status. Ascertaining the minimal clinically important difference
Control Clin Trials
(1989) - et al.
Defining clinically meaningful change in health-related quality of life
J Clin Epidemiol
(2003) - et al.
Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life
J Clin Epidemiol
(1999) - et al.
Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance
J Chronic Dis
(1986) - et al.
On the debate over methods for estimating the clinically important difference
J Clin Epidemiol
(1996) - et al.
A taxonomy for responsiveness
J Clin Epidemiol
(2001) - et al.
Identification of clinically important changes in health status using receiver operating characteristic curves
J Clin Epidemiol
(2000) - et al.
The minimal clinically important difference of the Carpal Tunnel Syndrome Symptom Severity Scale
J Hand Surg [Am]
(2006) - et al.
Determining a minimal important change in a disease-specific Quality of Life Questionnaire
J Clin Epidemiol
(1994) - et al.
Responsiveness and minimally important change for the Manchester-Oxford foot questionnaire (MOXFQ) compared with AOFAS and SF-36 assessments following surgery for hallux valgus
Osteoarthritis Cartilage
(2007)
Psychometric properties of the Neck Disability Index and Numeric Pain Rating Scale in patients with mechanical neck pain
Arch Phys Med Rehabil
Development and evaluation of a Pediatric Ulcerative Colitis Activity Index (PUCAI): a prospective multicenter study
Gastroenterology
Responsiveness of functional status in low back pain: a comparison of different instruments
Pain
Interpretation of rhinoconjunctivitis quality of life questionnaire data
J Allergy Clin Immunol
A comparison of the original chronic respiratory questionnaire with a standardized version
Chest
Using the entire cohort in the receiver operating characteristic analysis maximizes precision of the minimal important difference
J Clin Epidemiol
A comparative contrast of clinimetric and psychometric methods for constructing indexes and rating scales
J Clin Epidemiol
Clinimetrics vs. psychometrics: an unnecessary distinction
J Clin Epidemiol
Methods to explain the clinical significance of health status measures
Mayo Clin Proc
What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) Questionnaire? Results from Eastern Cooperative Oncology Group (ECOG) Study 5592
J Clin Epidemiol
Measurement properties and interpretability of the Chronic Respiratory Disease Questionnaire (CRQ)
COPD
Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life
Med Care
Effect sizes for interpreting changes in health status
Med Care
Comparisons of five health status instruments for orthopedic evaluation
Med Care
Clinical significance: a statistical approach to defining meaningful change in psychotherapy research
J Consult Clin Psychol
Statistical methods for assessing agreement between two methods of clinical measurement
Lancet
Statistical power analysis for the behavioral sciences
Responsiveness and minimal important differences for patient reported outcomes
Health Qual Life Outcomes
Interpreting the significance of changes in health-related quality-of-life scores
J Clin Oncol
Cited by (274)
What's the clinical significance of VAS, AOFAS, and SF-36 in progressive collapsing foot deformity
2024, Foot and Ankle SurgeryTest–Retest Reliability and Validity of 1-Minute Sit-to-Stand Test in Patients With Chronic Heart Failure
2023, Heart Lung and CirculationImpact on productivity impairment of a digital care program for chronic low back pain: A prospective longitudinal cohort study
2023, Musculoskeletal Science and PracticeMeasurement Properties and Minimal Important Change of the World Health Organization Disability Assessment Schedule 2.0 in Persons With Low Back Pain: A Systematic Review
2023, Archives of Physical Medicine and Rehabilitation