Low compliance is problematic

To illustrate the problem of compliance, we used a data set from an RCT with 40-week follow-up investigating the efficacy of an Athletics Injury Prevention Programme (AIPP) to reduce injury complaints leading to restrictions in athletics participation (PREVATHLE: CPP Ouest II-Angers, number: 2017-A01980-53; ClinicalTrials.gov Identifier: NCT03307434). In this trial, track and field athletes (n=840) were allocated to either a control group (n=391, reference group), who were asked to continue their usual training, or an intervention group (n=449), who were asked to perform the specifically designed AIPP twice a week. Interestingly and thought-provokingly, only 31 of the 449 (6.9%) athletes in the intervention group complied with the AIPP (ie, twice a week) throughout the 40-week follow-up. Unfortunately, this compliance problem is well known as the low compliance of 6.9% is similar to findings in some other sports injury trials.1 Consequently, the results derived from the ITT analysis will likely not reveal any effect of the preventive programme. Alternative analytical approaches should be considered.1

Do different analytical approaches give different results?

To illustrate the comparative benefits and challenges of different analytical approaches, we applied a time-to-event approach3 and analysed data from the PREVATHLE trial in three different ways: (1) by using the ITT principle,4 2) by using the as-treated (AT) principle4 and (3) by using the IV analyses principle (table 1).

The ‘classical’ ITT analysis revealed 3.1% fewer injuries among athletes in the intervention group compared with the control group, whereas the AT analysis–on the contrary—revealed that 12.9% more athletes in the intervention group (and who actually did the AIPP) sustained injuries. These contradictory findings may be explained by confounding as those athletes who followed the injury prevention programme did it for a reason (arrow from ‘confounders’ to ‘following the intervention’ in figure 1), which could also influence the outcome (arrow from ‘confounders’ to ‘injury’ in figure 1). For instance, having a history of previous injury could be one such confounder, since this can influence athletes attend to prevention strategies and thus actually do the AIPP. In addition, previous injury has also been reported as risk factor of future injury. In this specific example, 95% of the compliant athletes reported a history of injury in the previous season. Among non-compliant athletes, 65% had had previous injury. Consequently, history of having previous injury could (alongside other variables) serve as a marker for a confounder.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Directed acyclic graph explaining the causal architecture of the instrumental variable analytical approach.7–9 For instrumental variable (IV) analysis, the allocation was the IV to control the confounding due to for instance non-compliance.2 3 6 Such assignment can be referred as an instrument because it meets three conditions: (1) it has a causal effect on the intervention, (2) it indirectly affects the outcome through intervention and (3) does not share common cause with the outcome.5 6 Extraneous variables, called confounders can affect both compliance and outcome, for instance, participants may have their own reasons for complying with the assigned intervention (eg, the perceived benefits of an injury prevention programme to an injury-prone participant) or not complying (eg, are free of injury and/or see no need to waste time in injury prevention exercises).

In contrast to ITT and AT analyses, the IV analysis revealed that 52.5% fewer athletes sustained injury among those who did the AIPP compared with those who did not. This approach clearly underpins that results from ITT and AT can suffer from bias and provide misleading results.

What can IV analysis add?

As a practical application, we believe the IV analysis can serve (1) as a tool to crosscheck the potential risk of bias in ITT or AT analyses and (2) to provide a more accurate estimate of the efficacy of the intervention in RCT with low compliance taken into account unmeasured confounding.1 3 5–7 Similar to others,1 we recommend that IV analysis should be included, at least, as a complementary analysis to ITT used for RCT analysis,2 and is also suitable for observational studies.5 6

To be considered as an IV, a variable must meet three conditions: (1) it has a causal effect on the intervention, (2) it indirectly affects the outcome through intervention and (3) does not share common cause with the outcome.5 6 The IV could thus be the assignment to an intervention (eg, an injury prevention programme).7 IV analysis allows one to estimate the effect of the intervention on sports injury even if there are unmeasured confounding of its effect.6 It, thus, allows correction for confounding (eg, non-compliance), which is of particular interest in low compliance RCTs.1 2 6 7 The directed acyclic graph in figure 1 shows how the ITT and AT analyses are vulnerable to measured and unmeasured confounding, and how the IV analysis can address using measured confounders.8 9

What are the challenges with IV analysis?

The main limitation of IV analysis is to find a good IV which meet the conditions.2 5 6 Another comes when the IV is weakly correlated with the exposure.5 6 Sample size also could be a limitation as it can make the model is questionable.2 3 5 In our example, the extremely wide (and unrealistic) 95% CI reported in table 1 illustrates the error that surface when the assumption regarding a strong correlation between the instrument and the exposure that was violated. Importantly, alternative methods, such as non-parametric bootstrapping, than the IV-method above can be used. If such methods had been used, it may have been possible to calculate appropriate confidence intervals.