Methods for investigating publication bias and other sample size effects in studies of diagnostic test accuracy
Jon Deeks with Petra Macaskill and Les Irwig (Sydney)
Publication bias and other sample size effects are issues for meta-analyses of test accuracy as for randomised trials. We investigated limitations of standard funnel plots and tests when applied to meta-analyses of test accuracy and look for improved methods, comparing Type I and Type II error rates in simulated meta-analyses of test accuracy.
Type I error rates for the Begg, Egger and Macaskill tests were found to be inflated for typical diagnostic odds ratios (DOR), when the prevalence of the disease differed from 50% and when thresholds favoured sensitivity over specificity and vice versa.
We considered an alternative funnel plot and tests of asymmetry based on functions of effective sample size (ESS), and found that they were valid if occasionally conservative tests for sample size effects. Empirical evidence suggested that they have adequate power to be useful tests, but that when DORs were heterogeneous all tests of funnel plot asymmetry have low power.
We concluded that existing tests that utilize standard errors of odds ratios are likely to be seriously misleading if applied to meta-analyses of test accuracy. We recommend that the effective sample size funnel plot and associated regression test of asymmetry should be used to detect publication bias and other sample size related effects.
Publication: 184
