Impact of Information Lost to Follow-Up in RCTs
Impact of Information Lost to Follow-Up in RCTs
We included 235 eligible reports, close to our target number (Figure). Agreement between reviewers during the screening process was high: κ 0.93 for title and abstract screening and 0.78 for full text screening. Authors of 107 reports responded to our request to verify abstracted data (46% response rate). The percentage accuracy across trials for the different abstracted items varied from 84% to 100%. Inaccuracies were related to unclear reporting. Table 1 and Table 2 , respectively, present the general and methodological characteristics of included trials.
(Enlarge Image)
Figure 1.
Identification of articles to include in study of effect of loss to follow-up
Table 3 shows the percentage of trials with different information regarding reporting of loss to follow-up. Of the 235 trials, 204 (87%) had either an explicit statement about loss to follow-up or a CONSORT flow diagram showing loss to follow-up, or both.
Of the included trials, 191 (81%) reported some loss to follow-up. Among these 191 trials, the medians and interquartile ranges for the reported percentage of loss to follow-up were 6% (2-14%) overall; 6% (1-14%) in the intervention group; and 7% (2-15%) in the control group (the difference between intervention and control groups was not significant). The medians and interquartile ranges for the “lost to follow-up to events ratio” were 0.26 (0.09-0.76); 0.30 (0.09-1.25) in the intervention group; and 0.23 (0.08-0.65) in the control group. A value of 0.26 means that one participant was lost to follow-up for every four participants experiencing the primary outcome (the difference between intervention and control groups was not significant).
In our regression analysis, a higher percentage of participants lost to follow-up was associated with inadequate concealment of allocation, a longer length of follow up, and a non-medical non-procedural intervention (see Appendix 3).
Table 4 shows the analytical methods used for handling loss to follow-up in the primary analysis of the 191 trials that reported some loss to follow-up. In about a fifth of trials, the method used was unclear.
Of the 191 trials reporting some loss to follow-up, results of 160 remained significant in a complete case analysis.
Percentage of trials losing significance—For the four common assumptions, the percentage of trials that lost significance varied from 0% (best case scenario) to 9% (none of the participants lost to follow-up had the event) to 17% (all participants lost to follow-up had the event) to 58% (worst case scenario). Table 5 shows the percentage of eligible trials that lost significance across a range of assumptions for the event incidence among those lost to follow-up relative to those followed-up. This percentage varied from 0% to 33% for the range of plausible assumptions. The percentages in the sensitivity analyses that included trials in which no loss to follow-up occurred were relatively lower across all assumptions by about 20% (see Appendix 4).
Mean ratio of relative risks—The mean ratio of relative risk across randomised controlled trials varied from 0.73 (best case scenario) to 1.51 (worst case scenario) (see Appendix 5). A ratio of 1.51 signifies a relative increase in relative risk by 51% (for example, if the relative risk is 0.8 with the complete case, it would be 1.2 in the worst case scenario). Appendix 5 also presents the findings for the assumptions that the event incidence among participants lost to follow-up decreased or increased relative to those followed-up. The ratio varied from 0.79 to 1.23 (that is, the relative risk increased by up to 23%).
Results
We included 235 eligible reports, close to our target number (Figure). Agreement between reviewers during the screening process was high: κ 0.93 for title and abstract screening and 0.78 for full text screening. Authors of 107 reports responded to our request to verify abstracted data (46% response rate). The percentage accuracy across trials for the different abstracted items varied from 84% to 100%. Inaccuracies were related to unclear reporting. Table 1 and Table 2 , respectively, present the general and methodological characteristics of included trials.
(Enlarge Image)
Figure 1.
Identification of articles to include in study of effect of loss to follow-up
Reporting of Loss to Follow-up
Table 3 shows the percentage of trials with different information regarding reporting of loss to follow-up. Of the 235 trials, 204 (87%) had either an explicit statement about loss to follow-up or a CONSORT flow diagram showing loss to follow-up, or both.
Extent of Loss to Follow-up
Of the included trials, 191 (81%) reported some loss to follow-up. Among these 191 trials, the medians and interquartile ranges for the reported percentage of loss to follow-up were 6% (2-14%) overall; 6% (1-14%) in the intervention group; and 7% (2-15%) in the control group (the difference between intervention and control groups was not significant). The medians and interquartile ranges for the “lost to follow-up to events ratio” were 0.26 (0.09-0.76); 0.30 (0.09-1.25) in the intervention group; and 0.23 (0.08-0.65) in the control group. A value of 0.26 means that one participant was lost to follow-up for every four participants experiencing the primary outcome (the difference between intervention and control groups was not significant).
In our regression analysis, a higher percentage of participants lost to follow-up was associated with inadequate concealment of allocation, a longer length of follow up, and a non-medical non-procedural intervention (see Appendix 3).
Handling of Loss to Follow-up
Table 4 shows the analytical methods used for handling loss to follow-up in the primary analysis of the 191 trials that reported some loss to follow-up. In about a fifth of trials, the method used was unclear.
Potential Impact of Loss to Follow-up
Of the 191 trials reporting some loss to follow-up, results of 160 remained significant in a complete case analysis.
Percentage of trials losing significance—For the four common assumptions, the percentage of trials that lost significance varied from 0% (best case scenario) to 9% (none of the participants lost to follow-up had the event) to 17% (all participants lost to follow-up had the event) to 58% (worst case scenario). Table 5 shows the percentage of eligible trials that lost significance across a range of assumptions for the event incidence among those lost to follow-up relative to those followed-up. This percentage varied from 0% to 33% for the range of plausible assumptions. The percentages in the sensitivity analyses that included trials in which no loss to follow-up occurred were relatively lower across all assumptions by about 20% (see Appendix 4).
Mean ratio of relative risks—The mean ratio of relative risk across randomised controlled trials varied from 0.73 (best case scenario) to 1.51 (worst case scenario) (see Appendix 5). A ratio of 1.51 signifies a relative increase in relative risk by 51% (for example, if the relative risk is 0.8 with the complete case, it would be 1.2 in the worst case scenario). Appendix 5 also presents the findings for the assumptions that the event incidence among participants lost to follow-up decreased or increased relative to those followed-up. The ratio varied from 0.79 to 1.23 (that is, the relative risk increased by up to 23%).
Source...