Recognizing and working with uncertainty in handling risk are part of both clinical practice and the policy-making process. The Department of Health recognizes the inherent challenges that uncertainty brings and employs a wide range of research mechanisms, drawing on a diverse set of disciplines, to provide an evidence base to inform both policy creation and implementation, and clinical practice. This paper describes a variety of situations and the Government response to them, each time highlighting the use of science to reduce unknowns and to support decision-making. It highlights the need for a strong research infrastructure to support the Department's requirements, and those of the NHS, both in real time to respond to emergencies and to establish a high standard of care. However, although science provides a powerful tool to reduce uncertainty, it will not always produce definitive answers and often provides the start point for a dialogue between decision makers and researchers.
The Department of Health has two key roles relating to science: that of using science and evidence to inform policy, its role as a funder of science to underpin policy development and its evaluation as well as healthcare and the public health. The field of health sciences is broad and many different types of clinical and medical research are undertaken in the NHS, ranging from work in scientific laboratories through to clinical trials to determine which treatments are most effective. In addition, the Department calls on and funds many other types of research, including social sciences, which have an impact on the health and well being of the population—including research into public health, health services and social care. These employ a wide range of research designs and methods to investigate issues such as reducing inequalities in health and care, the organization and delivery of health and care services, workforce issues, funding allocations and the quality and outcomes of care. The Department and the NHS also draw on inter disciplinary work from widely different fields; such as that produced by the National Meteorological Office, where, for example, climate predictions are used to inform NHS personnel planning for winter pressures.
This paper uses case studies to demonstrate how, through dialogue, the science and evidence inform policy-making and practice development in the Department of Health and explore the experiences of both practitioners and policy-makers who have to work with uncertainty in the evidence base.
2. Evidence-based medicine to evidence-informed policy
Professor Archie Cochrane, a Scottish epidemiologist, is widely recognized as being the father of the modern concepts behind evidence-based practice (A. Cochrane; Effectiveness and efficiency: random reflections on health services 1972). The explicit methodologies used to determine ‘best evidence’ were largely established by Professor David Sackett and Gordon Guyatt at McMaster University .
3. Case study: the role of high-dose steroids for severe brain injury
In the 1970s and 1980s, high-dose steroids were routinely used in casualty departments for the management of severe brain injury, such as that associated with major trauma.
This practice was based on a body of evidence demonstrating, in animal models, that the brain oedema, or swelling, that occurs following major trauma could be reduced by intravenous treatment with high-dose steroids. However, in general, the historical studies that suggested this treatment resulted in fewer deaths were small and underpowered. Nevertheless, the practice continued until the early part of this century when an English group recognized that there was equipoise and they undertook a larger study that demonstrated conclusively that there were more deaths in the treatment group than in the group receiving the placebo . Since the publication of that paper, high-dose steroids for the treatment of severe brain injury have not been recommended as a routine treatment.
4. Using science to reduce uncertainty
This case history demonstrates that science is not always accurate in its predictions and can be incomplete in its understanding of the issue at hand. It also demonstrates that systematic reviews and meta-analysis of existing evidence can increase our confidence in predicting outcomes and demonstrate where there are real uncertainties.
The Department of Health last year spent £16 million on systematic reviews and the infrastructure for supporting them. Through the policy research programme (PRP) it funds a public health reviews facility at the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), and through the National Institute for Health Research (NIHR) supports the UK Cochrane Centre and associated Cochrane review groups as well as the NIHR Centre for Reviews and Dissemination.
Recognizing and working with uncertainty in handling risk are part of both clinical practice and the policy-making process. Evidence-based care, however, might be thought to imply certainty and that for individual patients there is a right and a wrong answer. Instead, most practitioners now recognize the need to know and use the evidence base to tailor the approach for an individual in discussion with that person, an approach called evidence-informed medicine. Successful evidence-informed policy-making takes a similar approach.
There is a widespread recognition of the need to use evidence in modern policy-making and comments from all political parties indicate their support for this approach . Science can be seen as a dialogue and scientists need to help policy-makers understand the uncertainties inherent in much of the evidence base. Scientists also need to recognize that there are inputs other than evidence that inform policy-making: experience and expertise, judgement, values, resources, habits and traditions. The National Audit Office has examined why knowledge is not as successfully transferred into practice as it could be and has summarized the issues as :
— research results are not easily accessible;
— poor understanding of policy questions by the researchers;
— poor communication of the research results by the researchers;
— poor understanding of the research results by the policy-makers;
— no direct, short-term relevance of research results to policy; and
— lack of resources for dissemination activity.
It is well recognized that researchers and policy-makers have different concepts of evidence. Linkage and exchange mechanisms need to be developed between the policy-making and research communities so that the approach is not just knowledge-push, but also knowledge-pull and on occasions, co-production of evidence.
5. Department of Health role as a funder of science
The Department of Health has two main routes for commissioning research.
Firstly, through the PRP1 to strengthen the evidence base for policy-making; and secondly, through the NIHR2 to strengthen the evidence base in clinical practice and public health. Through these funding mechanisms, the Department commissions research to resolve uncertainties, as shown in the case studies below, and provides a source of responsive funding for clinical and applied research. Additionally, the Department seeks input from external experts and Scientific Advisory Committees to gain the very widest understanding of the issues, gaps and uncertainties that drive the commissioning of science.
6. Policy research programme
The Department of Health operates a unique research programme that provides a direct link between the generation of scientific evidence and the creation and implementation of policy—the PRP. The PRP comprises a team of research specialists (Liaison Officers) who act as a single point of contact for policy teams across the Department, across Government and, in some cases, other public or private sector organizations. Liaison Officers work closely with policy teams to understand new and emerging policy areas in order to provide timely evidence for current policy needs and secure the evidence base for future policy-making. They work with policy-makers from the earliest phase of policy development, through policy implementation to the evaluation of policy interventions.
The PRP is the main engine for research in the Department and, as the following case studies demonstrate, commonly the work it addresses operates on a cross-Government and wider basis to address national needs.
7. Case study: Government response to the Foresight obesity report
It is now well recognized that the world is facing an epidemic of obesity. The report on tackling obesities produced by the Government Office for Science Foresight group in 2007 presented an interesting challenge to Government and research councils in terms of developing the evidence base. The report mapped out the complex set of determinants and relationships underpinning the obesity challenge. Possible policy responses are numerous and are influenced by:
— societal influences;
— food environment;
— physical activity and environment;
— individual psychology;
— food consumption; and
— underlying biology and genetics.
The complexity of the system illustrates the breadth of knowledge needed to understand obesity. It is necessary to draw on the results of biological studies as well as multi-disciplinary studies of obesity prevention interventions aimed at the individual and populations level—quite frequently these interventions are implemented outside of the NHS. There is increasing evidence in many of these areas, but the strengths of that evidence base are variable and patchy. However, national policy is being planned and implemented now so the public cannot wait for science to present a complete picture of the evidence across every domain of action. It is clear that as scientists, we need to remain supportive, and in addition act as a critical friend to policy-makers and practitioners while managing the uncertainties.
The initial Government research response was to bring together all Government Departments responsible for policies that might influence food and exercise alongside the Research Councils to develop a cross-Government strategy for research and surveillance to support England (Healthy Weight, Healthy Lives: a cross-Government research and surveillance plan for England, 2008). In the short term, the Department of Health has invested in systematic reviews of the current evidence and innovative approaches to capture other types of evidence, including, for example, on process and implementation that may be useful for policy-makers and practitioners—always bearing in mind the limitations of each approach.
The Department has funded early reviews and literature maps on issues as varied as bariatric surgery, the views of children on weight and shape, the relationship between educational attainment and obesity and the links between sedentary behaviour and weight. While none of these are likely to give a clear-cut view of ‘what works for whom’, these mapping and review exercises can give an early indication of what might not work, reducing any current level of uncertainty.
In the medium term, the Department is building up the research infrastructure to ensure funding can be in the right place at the right time. For example, the new NIHR Pubic Health Research programme has been set up to fund evaluation of interventions implemented outside of the NHS. An early example is a project looking at the health impact of a new bus scheme in an urban area.
While it is tempting for the Departments to look only to their immediate need for evidence, part of the management of uncertainty in science is planning for the long-term needs, keeping in mind the bigger picture and using experience to guide investment where there is no immediate demand. In the case of obesity, the Department is working closely with research councils to try to cover as many ‘evidence bases’ as possible in a coordinated approach to ensure that evidence is prepared for when it is needed.
8. Case study: climate change and health
There are many uncertainties in dealing in both mitigation and adaptation to the impact of climate change on health, even in the UK, and we are at an early stage of dealing with many of these issues. A broad range of scientific disciplines are needed to ensure a comprehensive approach to apparently unrelated issues, including, for instance, the impact on mental health of flooding and defining the vulnerable sub-groups.
Among the many issues for the Department of Health are:
— How the NHS and public health planners will use the information from the UK climate projections.
— Modelling the impact of policy interventions for mitigation and adaptation. The strategies here include examining the various impacts of heatwaves on the healthcare sector, identifying appropriate indicators to measure the effectiveness of the NHS heatwave plan and an evaluation of the effectiveness of public health messages on changing behaviour to limit the health impact of heatwaves.
In recognition of the intersectoral complexity of climate change, the UK research councils, led by the Natural Environment Research Council, have established the ‘Living With Environmental Change (LWEC)’ initiative to facilitate collaboration and coordination across all research funders and stakeholders, including key agencies such as the Meteorological Office.
9. Case study: building the evidence base for funding long-term care
A major area of social and economic certainty, which faces all developed societies, is how to meet the cost of long-term care for the ageing population. This ‘demographic time bomb’ is well advanced: in 2002, we passed the economically pivotal point where the proportion of people over 60 years was higher than those under 16 years.
While many people are living healthily in their old age, many are living with chronic debilitating conditions. Major uncertainties remain about the magnitude of the growing demand for health and care services, how our society can continue to afford high-quality care and how a smaller working population can support the increasing size of the ageing population.
The PRP funds a major programme of work on long-term care finance at the Personal Social Services Research Unit (PSSRU) at Kent University. As part of this, PSSRU are modelling projections for demand for long-term care up to 2041 using a dynamic microsimulation model to assess a range of future funding scenarios. These scenarios explore the impact of different assumptions about major ‘external drivers’, such as demographic or economic pressures, of the supply of and demand for long-term care. This work will help address the crucial ‘what if’ questions about the impact of different policy options for care finance. More recently, the PSSRU analysed the care funding options contained in the 2009 Green Paper on shaping the future of care together (http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_102338).3 Using actual individuals sampled in the British Household panel survey, the team simulated how their wealth, levels of need, rates of disability, health, household position, etc., would change in the future. By embodying a range of behavioural rules, drawn from a range of datasets, the model has made predictions about how each of the policy options for long-term care funding will affect the lives of older people.
10. National Institute for Health Research
As mentioned above, another way for the Government to reduce uncertainty and provide evidence for clinical practice and policy-making is by commissioning research projects and programmes through the NIHR. The following two case studies illustrate how NIHR funding has been successfully mobilized to address two quite different scenarios: first to increase certainty in the treatment of a common illness (Bell's palsy); and second to deal with uncertainties in science and health in an emergency (pandemic flu).
11. Case study: resolving uncertainty and saving money in clinical practice
Uncertainty in clinical practice has the potential to lead not only to the provision of sub-optimal or potentially harmful treatment, but also to disproportionate resource burden for the NHS. This arises both from the direct consequence of patient harm, but can also arise from incorrectly prescribed expensive and ineffectual treatments.
Bell's palsy is a common illness causing facial paralysis when a cranial nerve (the facial nerve) does not function properly. The traditional treatment has been to use steroids to speed the recovery, but 30 per cent of patients always had a poor result. Increasingly, an expensive antiviral, acyclovir, has been used. Prednisolone therapy was prioritized as a topic in 2002 by the NIHR and a randomized control trial comparing acyclovir with prednisolone on its own and in combination with prednisolone was commissioned by the NIHR Health Technology assessment programme in 2003.
This study of 551 patients was reported in 2007  demonstrating no advantage in treatment with acyclovir over prednisolone alone. This was the largest study on this topic ever done and, through providing a more definitive level of evidence than the studies that preceded it, has now led to changes in practice. In addition, this paper was awarded the BMJ research paper of the year in 2008.
12. Case study: pandemic flu in 2009
In the twentieth century, there were three pandemics: in 1918 with what the World Health Organization (WHO) estimates as between 40 and 50 million deaths worldwide, 1957 with 2 million deaths and in 1968 with 1 million deaths. The peak age of death varied from pandemic to pandemic. With the emergence of the H5N1 influenza virus in avian species and the demonstration of its transmission, although rarely to humans, Government had been preparing and planning for an influenza pandemic for some time. In the event, the influenza virus that led to the 2009 pandemic (sometimes referred to as ‘swine flu’) was an H1N1 influenza virus strain that was first recognized in April 2009 in Mexico. It quickly became evident that there had been an ongoing epidemic and associated deaths for some months previously. Britain is at the centre of global transport networks so it was not surprising that the first UK case was recognized early in the development of the pandemic, on 26 April 2009. WHO declared the outbreak as a pandemic in June 2009. It was therefore essential for the Government to receive the best scientific advice available for its planning and response to the pandemic.
The Department of Health as the ‘lead Government department’ established the Scientific Advisory Group for Emergencies (SAGE) co-chaired by the Government's Chief Scientific Adviser, Professor John Beddington and the Chair of the Scientific Pandemic Influenza Advisory Committee, Professor Sir Gordon Duff, to provide scientific advice to the Government during its emergency response to swine flu. This group was made up of relevant experts from inside and outside Government and the membership drew on the Scientific Pandemic Influenza Advisory Committee (SPI) set up in 2008. Throughout most of the pandemic, SAGE met weekly and received papers and reports and reviewed modelling from three independent groups of mathematical modellers. Such modelling was needed for the UK to develop planning assumptions for a reasonable worst case scenario: how bad the impact could be (in terms of cases, hospitalizations and deaths) and, when this became possible to predict in real time the likely numbers of cases, as well as to advise on when peak demand for services was likely to happen and thus to inform logistics planning.
SAGE drew together information on the epidemiology and impact of swine flu as it evolved. It reviewed the latest surveillance information from the four UK jurisdictions and obtained reports from South America and the USA, where the disease first emerged, and then from the Southern Hemisphere more widely as they experienced swine flu in their winter where it was likely to have a more severe impact than during summer in the UK. Information from the European Centre for Disease Prevention and Control and the WHO allowed SAGE to monitor the evolving situation globally, but the data had to be assessed carefully in order to decide what comparisons might be valid, as not all countries have the same levels of surveillance.
At each meeting, SAGE received reports on the international situation, a summary of the UK situation including the clinical severity of UK cases, results of virological surveillance and primary care clinical surveillance, as well as the latest mathematical modelling results. Initially mathematical modelling was difficult, because its accuracy is dependent on the accuracy of the parameters used in the model, for example, the number of cases infected by a single infected individual (used to calculate the basic reproduction number) and the average time from an individual being infected to that individual infecting others (the generation time), neither of which were known with accuracy. Many indicative scenarios were developed based on different possible generation times and reproduction numbers. These were based on analysis and modelling of data from both inside and outside the UK. Models based on ‘reasonable worst case’ values allowed robust planning assumptions to be developed to prepare for ‘worst case’ scenarios, but they were never intended as a prediction on how the pandemic would develop.
SAGE advised and reviewed advice on many other key issues in the swine flu outbreak such as:
— the appropriate definition of ‘at risk groups’ for antiviral drugs and vaccines;
— the potential effects of school closures on the pandemic; and
— priorities for research.
An important part of SAGE's role was to communicate the uncertainties, particularly from the mathematical modelling, to ministers and to present potential scenarios of how the pandemic might develop (while not making predictions) in order to ensure that the UK was fully prepared to meet any likely eventuality. The planning assumptions were revised as appropriate over time as further UK and international surveillance data emerged as well as the development of greater clinical understanding of the impact of the disease.
The cause of swine flu was a novel strain of influenza virus, leading to considerable uncertainty (particularly in the initial phases of the outbreak) about many aspects of its potential impact on individuals and on the UK as a whole, but by developing as comprehensive a picture as possible and bringing a wide range of scientific perspectives to bear on it, the best advice possible was provided to ministers.
The NIHR was swift to respond to the research challenges presented by swine flu through the commissioning in June 2009 of five projects based on SAGE priorities and, additionally, nine responsive projects in early August. The Government's processes for public health emergencies as well as the NIHR Research Networks were activated in order to ensure speedy delivery of these 14 studies.
In retrospect, it is not clear that the 2009 H1N1 pandemic would have been recognized as a pandemic in the first two-thirds of the twentieth century when the level of surveillance was not as high as exists today, but there is no doubt that SAGE and associated processes ensured that the best scientific advice, including the development of models for planning and prediction, helped to ensure that the UK response was as effective and proportionate as possible. The importance of the scientific advice to the response was highlighted in the independent review of the UK response to the 2009 influenza pandemic carried out by Dame Deirdre  that was published in July 2010.
This case study demonstrates that a Government must be ready to deal with uncertainties in science and health impact in an emergency. It must have in place the necessary structures and processes to capture efficiently and effectively information about the situation, identify clear areas of uncertainty and begin to resolve these using a diverse range of methodologies from a broad discipline base. In the case of swine flu, this was further complicated by the need to operate on local, national and international scales, introducing yet more potential uncertainty and contradiction that was resolved through seeking expert advice and guidance to inform ministers and enable effective decision-making.
13. Communicating uncertainty in public health
From a Government perspective, uncertainties in science can have very damaging consequences if they begin to undermine the trust the general public have in the health service or the advice being given by the Government, for example, from the Department of Health.
It is important to recognize that risk is an integral part of public policy decisions. It is vital that Government communicates well about risks in a clear proactive and carefully planned manner based on best-available evidence. Nevertheless, it is also important that people understand the messages they receive and these may need to be tailored for the public or health practitioners. The Government has to take responsibility for communicating public health messages based on the ‘best’ information available, but it is a fact that this is often in the face of uncertainty.
14. Case study: MMR vaccine
In 1995, The Lancet published a paper, led by Andrew Wakefield that claimed a link between Crohn's disease and the measles vaccine . The study looked at the prevalence of Crohn's disease, ulcerative colitis, coeliac disease and peptic ulceration in over 3500 people who received a live measles vaccine in 1964 as part of a measles vaccine trial. A longitudinal birth cohort of over 11 000 people and over 2500 of the partners of the vaccinated individuals acted as controls. The authors opined that there was an increase in the relative risk of developing inflammatory bowel disease after vaccination. The subsequent media attention resulted in a rapid decline in vaccination uptake at 16 months as a result of a loss of confidence in the vaccine. It was clear that the general public and many clinicians had forgotten the unpleasant, although rare, serious morbidity and mortality associated with measles.
In 1998, Andrew Wakefield et al. reported in The Lancet a study of 12 children that claimed that nine children displayed autistic symptoms and that for eight of these children a link could be shown to the MMR vaccine . Successive negative media reporting reduced the confidence of parents and the public in the MMR vaccine, such that in early 2003 over 35 per cent of mothers were not confident in MMR safety and the uptake had fallen to only 65 per cent of infants. The inevitable consequence followed of a 100-fold increase in confirmed cases of measles.
Many doctors and scientists who read these two articles were concerned by the quality of the science and believed that the case of a link between either Crohn's disease or autism with MMR vaccination was not convincingly made. All subsequent peer-reviewed studies, including some large ones, have failed to show any association between the MMR vaccine and autism.
A number of bodies including the Centres for Disease Control and Prevention, the Institute of Medicine, the NHS and a Cochrane review published in 2005 all concluded that there was no evidence of a link between the MMR vaccine and autism. Yet, by 2008, there were 1348 cases of measles in the UK with two confirmed deaths raising important questions about the peer-review process and editorial responsibility. In 2010, Wakefield was found by the General Medical Council (GMC) to have acted irresponsibly and was removed from the general medical register.
This case history demonstrates that as part of the risk landscape in developing policy it is important to recognize the role of ‘risk mongers’—those are the people and groups who conjure up or exaggerate risks, sometimes well intentioned but misguidedly, and often because of their own agenda or interest . Their arguments need to be explicitly challenged with robust science.
This case also demonstrates that it is important for scientists and journals to think about their communication with the public and practitioners, the impact of their claims and how they can improve the understanding of the scientific methods that underpin their research. It is important that when new, and possibly controversial, research findings are published or reported in the media, the Government engages in public risk communication and that dialogue with the public and the media is built on a better understanding of scientific methodologies.
15. An uncertain future?
As the case studies show, policy-making, clinical practice and the science underpinning them rarely, if ever, start from a base of certainty. Governments accept that while it would be ideal to possess clear and accurate evidence to use in decision-making, this is not often the case and therefore they create and support mechanisms to help bring clarity to situations that are often complex and confused.
As this paper has shown, it is essential that the necessary evidence base is built in areas of high priority. This can often take place through the review of existing knowledge, bringing together the required information to allow future policy directions to be mapped out, as seen in situations like the response to obesity. Sometimes, however, we have to respond to situations such as pandemic influenza where, while it is still possible to review current information, it is also necessary to create new evidence and reduce uncertainties in real time.
In both situations, there is generally an increased complexity brought about from the need to gather data from a very diverse set of sources and this in turn can, at least in the first instance, increase uncertainty. Policy-makers and clinicians have to weigh up the robustness of the science and sometimes take decisions based on partial evidence. Similarly, Government takes responsibility for communicating health messages on the ‘best’ information available.
It is clear from the case studies above that science, through research, plays an integral part in reducing uncertainty. It is therefore important to recognize the need for an embedded research strategy, such as Best Research for Best Health, alongside a central research capacity that not only builds the evidence base to reduce the high-priority uncertainties of the present, but also prepares us for areas of the future need.
One contribution of 15 to a Discussion Meeting Issue ‘Handling uncertainty in science’.
↵1 NIHR—the Department of Health set up the NIHR in April 2006 to provide a framework through which the research, research staff and research infrastructure of the NHS in England can be positioned, managed and maintained (http://www.nihr.ac.uk/Pages/default.aspx). NIHR plays a pivotal role in reducing uncertainties and strengthening the evidence base in clinical practice. The budget for 2010–2011 is £1 billion.
↵2 PRP—the Department of Health uses its PRP to commission high quality, research-based evidence across the full remit of the Department of Health including public health, social care and the NHS (http://www.dh.gov.uk/en/Aboutus/Researchanddevelopment/Policyresearchprogramme/DH_533).
- This journal is © 2011 The Royal Society