Methodological study
Outcome reporting bias in randomised controlled trials
An-Wen Chan and Doug Altman with Asbjørn Hròbjartsson, Mette Haahr, Peter Gøtzsche (Copenhagen, Denmark), Karmela Krleža-Jerić, Isabelle Schmidt (Ottawa, Canada)
Selective publication of studies based on the nature and direction of their results has been well-documented. Much less work has been conducted in the area of outcome reporting bias, which can be defined as the selective reporting of measured outcomes within a published study based on their nature and/or direction, resulting in systematic differences between reported and unreported data. Although it is widely believed to exist, selective reporting of outcomes had not previously undergone comprehensive review across cohorts of randomised trials. Direct evidence of outcome reporting bias has been limited to case reports with limited generalisability.
Three separate studies were carried out. Two separate cohorts of trial protocols were identified, from an ethics committee in Denmark and the Canadian Institutes of Health Research. For each protocol published reports were sought. The two cohorts yielded 102 and 48 pairs of protocols and publications respectively. The third study identified all RCTs indexed on PubMed that had been published in a single month in 2000. These were scrutinised to compare the outcomes specified in the methods section with those for which results were reported. For all three cohorts a questionnaire was sent to the corresponding author asking for information about unreported outcomes.
All three studies showed clear evidence of selective reporting in favour of statistically significant results; together they provide reliable evidence of significant deficiencies in outcome reporting within published reports of trials. The reporting of outcomes was frequently found to be not only incomplete, but also biased and inconsistent with trial protocols. A median of up to a third of efficacy outcomes and half of safety outcomes per parallel group trial were found to be inadequately reported for meta-analysis, with a median of 4-5 efficacy and 2-3 safety outcomes per trial being omitted entirely from publications. Full reporting of outcomes was consistently biased to favour statistically significant results, with a greater than two-fold increase in odds for a fully reported outcome being significant relative to an incompletely reported outcome. Furthermore, discrepancies between primary outcomes specified in protocols and publications were observed in approximately one half of randomised trials, with some evidence of further bias in favour of significant results. These findings have serious implications for the conduct and reliability of systematic reviews.
Publications: 90, 91, 168, 169, 193, 210
