Indirect comparisons of effectiveness (ICE)
Doug Altman, Charlotte Sakarovitch, Roberto D’Amico, Jon Deeks and Mike Bradburn with Anne-Marie Glenny (Manchester), Fujian Song (Birmingham, East Anglia), Alison Eastwood (York) and the International Stroke Trial Collaborative Group
In many areas available RCTs may not have directly compared the specific treatments or regimens of interest. A common example is where there is a class of several drugs, each of which has been studied in placebo-controlled RCTs but there are no trials (or very few) in which the drugs have been directly compared to each other.
The aims of the project were:
1. To survey the frequency of use of indirect comparisons in systematic reviews and evaluate the methods used in their analysis and interpretation
2. To identify alternative statistical approaches for the analysis of indirect comparisons
3. To assess the properties of different statistical methods used for performing indirect comparisons
4. To carry out empirical work comparing direct and indirect estimates of the same effects within reviews.
These aims were met by a literature review and methodological and empirical investigations.
Of 327 systematic reviews identified through DARE that included meta-analyses of two or more RCTs, 31 (9.5%) included indirect comparisons. Few studies had carried out a formal analysis. Some reviews based analysis on the naïve addition of data from the treatment arms of interest. Interpretation of indirect comparisons was not always appropriate. Very few methodological papers were identified, of which only one suggested a simple method: Bucher et al proposed using the ratio of two separate odds ratios.
Simulation studies showed that the naïve method is liable to bias and also produces over-precise answers. Several methods provide correct answers if strong but unverifiable assumptions are fulfilled. Four times as many similar sized trials are needed for the indirect approach to have the same power as directly randomised comparisons.
When making indirect comparisons within a systematic review, an adjusted indirect comparison method should, ideally, be used using the random effects model. If both direct and indirect comparisons are possible within a review, it is recommended that these are done separately before considering whether to pool data.
Publications: 79, 156, 187
