10 Year Celebration
"Medical Statistics: Making a Difference in Health Care"
20th September 2005, Magdalen College, Oxford
10.15-10.55 Paul Glasziou: Clinical trials to clinical decision-making
The use of p-values alone to analyse studies has been gradually replaced by estimation and confidence intervals. While this has improved the statistical interpretation of results within trials, it is still insufficient for decision-making that requires absolute benefits and harms to be made clear. This talk will look at interpreting the trials of acute otitis media, and the use of the methods of the Cochrane Applicability and Recommendations Group which aims to provide guidance to reviewers to answer the questions "To whom do the results of the systematic reviews apply?" and "What are the implications for patients and policy?" The preferred approach for applying evidence to individuals is based on a risk-benefit model. This talk discusses a 5 step process to examine the transferability of results and then apply them to different groups and individuals.
10.55-11.35 Julian Higgins: Embracing complexity in evidence synthesis
In the past, statisticians have tended to impose available methods onto available evidence. Now it is possible for available evidence to dictate suitable methods, largely through developments in statistical modelling software. Flexible modelling approaches are particularly valuable in the field of evidence synthesis. Evidence synthesis seeks answers to major medical and healthcare questions by combining findings across studies, and the statistician is obligated to make best use of all relevant information. I shall overview recent developments in evidence synthesis, outlining how methods are available to combine studies addressing different parts of a larger problem, to encompass variation and inconsistencies across studies, and to incorporate evidence with different degrees of relevance and validity. Among the illustrative examples I will use will be syntheses of clinical trials to determine the best mode of topical fluoride therapy to prevent dental caries, and of epidemiological studies seeking an interaction among two genes and smoking history in pre-disposition to bladder cancer. The additional freedom offered by these methods brings additional responsibilities to the statistician, of being transparent, undertaking sensitivity analyses and enabling others to replicate the analyses.
11.35-12.15 David Schriger: Problems in the analysis and reporting of clinical research:
Are graphics the solution?
This lecture will begin with an exploration of the assumptions that are commonly invoked when we conduct, analyze and report clinical research. I will argue that in current practice most of these assumptions remain covert and that the failure to explicitly consider them in concert with the mechanical application of classical statistical methods has produced a literature that is overly confident and optimistic. I will conclude by exploring how graphical analyses and presentations may remedy this situation.
13.15-13.55 Marion Campbell: Data monitoring of randomised controlled trials: medical statistics in
action
Randomised controlled trials generally recruit to a pre-specified sample size, primarily dictated by clinical and statistical considerations. The interests of trial participants will be best served, however, if recruitment is closed as soon as a clear answer is available. On the other hand, the interests of society will be best met if recruitment continues until there is a clear answer sufficient to lead to changes in clinical practice. In the early trials conducted in the UK, early stopping of recruitment was not routinely considered. With the advent of large multicentre trials (such as the ISIS trials in the 1980s), however, statisticians and trialists recognised the need for formal independent monitoring of accumulating trial data to allow the balance of interests between trial participants and the wider society to be addressed "“ with this, the concept of the data monitoring committee (DMC) was born. Since then, DMCs have become increasingly common with medical statistics playing a pivotal role in their development. In this talk, I will chart the development of DMCs and the contribution of medical statistics to them. I will draw on the findings of the recent DAMOCLES project (which included a number of medical statisticians), which reviewed best practice for DMCs.
13.55-14.35 Martin Bland: Measuring agreement between measurements
Measurement is a key part of clinical medicine and the development and evaluation of new methods of measurement is an important research activity. The misinterpretation of simple statistics is frequent in the analysis of such studies. This led to the development of the limits of agreement approach, which has been widely adopted. Many problems remain and I shall illustrate some of these using a recent example concerning the measurement of blood glucose.
14.35-15.15 Peter Gøtzsche: Medical statistics in practice: Dr Jekyll and Mr Hyde
Proper use of medical statistics and logic has been tremendously successful and has helped doctors save thousands of lives. However, it is also true that their improper use, often for commercial benefit, has lured doctors into killing thousands of patients in good faith. Mainland noted twenty years ago that the striving for significant P's serve as passports to publication. This is still the case. A survey of 520 papers from 2003 that contained the words "relative risk" or "odds ratio" in the abstracts showed that the first result in the abstract was reported to be statistically significant in 70% of randomized trials, 84% of cohort studies and 84% of case-control studies. Many of these significant results were derived from subgroup or secondary analyses, and most of the claimed P-values in the interval 0.04-P<0.05 for the randomized trials proved to be non-significant on recalculation. Comparisons of trial protocols with trial reports have shown a large scope for bias. In one survey, at least one primary outcome was changed in two-thirds of the trials, and ongoing work shows that protocols sometimes specify a large number of different statistical analyses and highly complex and varied ways of measuring what could have been rather simple outcomes. For observational studies, we know very little about what was is going on as we don't have access to the protocol, if there ever was one. We need less statistics, better statistics, and statistics done for the right reasons to reduce the epidemic of false cause-effect relationships.
15.40-16.20 Fiona Godlee: What can statisticians do for medical journals?
Medical research has become more and more dependent on statistics, and the statistics themselves have become more and more complex and sophisticated. Whereas in the past a medical statistician may have been a luxury item, affordable only by the richest journals, now no self-respecting medical editor would dare to proceed without one at her side. So what have statisticians done for journals, and what could they do in the future?
16.20-17.00 Doug Altman: How much confidence can we have in published medical research?
There is considerable evidence that the methodological quality of much medical research is unacceptable. In the first part of the talk I will highlight aspects of medical research studies that cause particular difficulty, with illustrative examples, and will consider whether the methodological quality of the medical literature is improving. I will discuss the importance of publication bias, including recent evidence of selective reporting within publications. Although the broad issues are relevant to all types of research, I will give particular attention to randomized trials. In the second part I will discuss efforts by journals and others to improve quality (use of statistical reviewers, checklists, guidelines etc.) and any evidence of effectiveness of these; and discuss efforts to improve the quality of reporting of research, such as the CONSORT statement. I will also stress the importance of letters to the editor and other forms of post-publication peer review.
