How the CfWI put research into practice

How the CfWI put research into practice

The R&D team has made changes to the research methods used in a number of workforce projects over the last four years. Together, these changes have helped to create a robust workforce planning framework as discussed in depth in the CfWI’s technical paper series.

This section of the research review outlines the ways in which improvements to individual stages of the workforce planning framework have directly impacted the quality of advice that the CfWI provides to decision-makers.

The two tables below present a list of all published (or upcoming) workforce reviews and technical papers supported and/or carried out by the R&D team since 2013.


Publication date

Horizon Scanning - A strategic review of the future healthcare workforce: Informing the maternity workforce

June 2013

Future nursing workforce projections – starting the discussion

June 2013

Future midwifery workforce projections – starting the discussion

June 2013

Horizon Scanning - A strategic review of the future healthcare workforce: Informing the nursing workforce

August 2013

A strategic review of the future pharmacist workforce

September 2013

A strategic review of the future dentistry workforce

December 2013

Securing the future workforce supply: Medical ophthalmology stocktake

February 2014

Horizon 2035: International responses to big picture challenges

May 2014

In depth review of the general practitioner workforce

July 2014

Horizon 2035: Health and care workforce futures progress update

July 2014

Innovative research and development at the CfWI

August 2014

Securing the future workforce supply: trauma and orthopaedic surgery stocktake

September 2014

Securing the future workforce supply: Dental care professionals stocktake

October 2014

In-depth review of the psychiatrist workforce

November 2014

Securing the future workforce supply: Speech and language therapy stocktake

December 2014

Understanding how the CfWI works

February 2015

The public health knowledge and intelligence workforce: A study

March 2015

Horizon 2035 - Future demand for skills: initial results

August 2015

Forecasting the adult social care workforce

October 2015



Technical Papers

Publication date

Robust workforce planning framework: An introduction

March 2014

Robust workforce planning: Examples and best practice

April 2014

Horizon scanning: Analysis of forces and factors

April 2014

Robust workforce planning: Medical model technical description

April 2014

Developing robust system-dynamics based workforce models: A best-practice approach

April 2014

Robust workforce planning framework: Update from practice

August 2014

Scenario generation: Enhancing scenario generation and quantification

August 2014

Policy analysis: Applying robust decision-making to the workforce planning framework

August 2014

Elicitation methods: Applying elicitation methods to robust workforce planning

February 2015

Policy analysis update

February 2015

Modelling the effect of multimorbidity on the demand for health services in England – Technical paper

March 2015

Long-term NHS expenditure trends and affordability constraints

July 2015

Modelling supply, demand and need: a literature review

October 2015

Elicitation methods: updated approaches to elicitation

November 2015

Improvements to each stage of the robust workforce planning framework over the last four years are discussed in more detail below:

Horizon scanning

The horizon scanning stage was originally a separate exercise largely conducted by interviews and used to inform the scenario generation stage. We now view this stage as a way of understanding the past, present and future of the system under investigation. The stage uses systems thinking methods to investigate the current system and the forces and factors driving the behaviour. Systems thinking provides a way of analysing and better understanding a system by taking into account the fundamental cause-and-effect relationships that drive system behaviour (Meadows, D.H. (2008). Thinking in systems: A primer. London: Earthscan). To support this we have undertaken the following:

  1. We have developed a consistent set of terms and definition for use in the horizon scanning stage and the wider framework.  These terms are presented as a visual taxonomy. Definitions have been published in the CfWI technical paper series. This ensures a consistent understanding of the terms being used.
  2. Introduced the concept of ‘ideas’, defined by the CfWI in our taxonomy as stories or fragmented narratives about the future. Ideas can be collected by interview, or more effectively using a crowdsourcing approach based on the horizon scanning hub (the hub located at Participants are asked to think about how the future might evolve, and the impact on a particular profession or aspect of workforce planning. They provide this as a brief story or narrative fragment. However, while they are not restricted in how they express an idea in the narrative, they are additionally asked to provide a quantitative interpretation of the significance of their idea. For example, this might be whether they consider the idea as having a low or high impact on the workforce, how likely it is, or which factors in the system are related to the idea. The hub provides an opportunity to increase greatly the number of ideas collected, and to provide a rich source of material for analysis. There are 398 Ideas available in the Horizon Scanning Ideas Bank ( The Ideas are categorised by sector, the workforces impacted, other relevant projects, and the factor themes represented within the idea. Each idea can be related to multiple elements within these categories. The ideas bank has not been consistently promoted so these findings need to be treated with care. We have conducted many more projects in health than in public health or social care, so the ideas are skewed towards health. It is proposed that these ideas are further categorised by the Horizon 2035 taxonomy for skills and competences. Horizon 2035 is examining future skill pressures across the whole of the health and care system ( This will support searching for narratives that discuss a particular issue of interest in the system.
  3. Improved how we analyse the forces and factors in the system, by developing a method for mapping the system using causal loop diagrams to explore the cause-and-effect relationships. Analysis of the maps provides a simple way to categorise the factors, and helps to decide which are potential policy levers or measures of performance. Vester’s influence matrix analysis is simple and fast to conduct, and offers insight into the nature of the factors ( The matrix describes the strength of the influence between all connected factors to focus attention on the critical aspects of a system – those factors that have the greatest influence over system evolution. This approach has proved highly promising, and areas for further research have been identified. For example, it would be interesting to get stakeholders involved in both building the map and the scoring the matrix. It is recommended that this be investigated further.
  4. A set of thematic categories has been developed to support the above analysis. The TEEPSE framework (technology, economy, environment, politics, society and ethics) was initially used in generating factors to check nothing important had been missed. It was expanded in 2014 to include a wider, range of themes (initially 14 themes were identified and then these were condensed into 11 factor themes). The decision to expand the set of thematic categories aligned with research and development advances that have been made to the CfWI Robust Workforce Planning Framework. The wider themes offer additional levels of detail to allow more options to section the data for user preference and during analysis. The categorisation of factors by a more sophisticated set of themes provides greater distinction between external and internal factors.

Scenario generation

The CfWI have produced over 64 scenarios, including 123 clusters and 44 scenario dimensions as of September 2015. Scenarios are constructed either from clusters or scenario dimensions. Each scenario provides a rich narrative of how the health and care system may plausibly evolve in the future.

The CfWI first used scenarios in the Medical and Dental Student Intake (MDSI) project (DH, 2012). They are a critical part of most of our workforce studies. The table below lists all CfWI workforce studies that use scenarios, and provides links to their scenario narratives. It should be noted that not all scenario narratives are available in the public domain.

Health and social care scenarios developed by the CfWI


Number of scenarios

Source for the scenario narrative

A strategic review of the future healthcare workforce - Informed medical and dental student intakes


Anaesthetics and Intensive care medicine


Forecasting the adult social care workforce to 2035


Unpublished work – event held in March 2014.

Future pharmacist workforce


Future psychiatrist workforce


General practitioner (GP) in-depth review


Horizon 2035: health and care workforce futures

8 describes the six scenarios in the public domain.

Public health specialist stocktake


Unpublished work – event held in July 2015

The Future Acute Medicine, General Internal Medicine and Acute Geriatric Medicine Workforce


The future shape of the healthcare science workforce


Event held in November 2012

Our scenario method has been updated over the years. We initially used the ‘classical’ intuitive logics approach, creating scenarios across two dimensions of uncertainty, described in a 2 x 2 matrix. This method focused on a single workshop to generate four narrative futures that were quantified for modelling. While the workshop is still an important component of this, the emphasis today is on producing a broad set of challenging but consistent futures for modelling and simulation. Several new approaches have been successfully trialled and tested on the Horizon 2035 project. We now use an approach developed by the Global Business Network (National Parks Service, 2013) allowing four dimensions and sixteen combinations to be considered. Once inconsistencies are removed this typically results in five or six scenarios.

Tighter integration has been achieved with the horizon scanning stage, following a systems thinking approach. Causal loop diagrams are analysed in the horizon scanning stage to provide a thematic set of factors as input to the scenario workshops. The factors are then consolidated into clusters which can be associated with multiple factor themes and multiple big picture challenges. Scenario dimensions replaced clusters in 2014. Each scenario dimension includes a narrative description and at least two extreme outcomes.

All in all, we have undertaken the following steps over the years in alignment with research and development advances:

  1. A formalised approach to workshops has been developed using a scripted technique derived from the group model building corpus (Hovmand, P.S., Andersen, D.F., Rouwette, E., Richardson, G.P., Rux, K. and Calhoun, A. (2012). Group model-building ‘scripts’ as a collaborative planning tool. Systems Research and Behavioural Science, Vol.29, pp.179-193). The workshops are fully defined and scripted to capture the tacit knowledge of the presenters, and improve quality and repeatability.
  2. A wider range of scenarios can now be produced. The 2 x 2 matrix method has been replaced and scenarios are now generated across four axes of uncertainty. Workshop participants qualitatively check potential factor combinations for consistency[1]before proceeding to create detailed narrative scenarios. This approach allows additional scenarios to be generated for situations where uncertainty is high. It is important that the scenarios span the full range of this uncertainty.
    1. Quantitative methods have also been introduced. The cross-impact balance method (Weimer-Jehle, W. (2006) Cross-impact balances: A system-theoretical approach to cross-impact analysis. Technological Forecasting and Social Change, Vo. 73, No.4, pp.334-361) is used to confirm the consistency of workshop scenarios, and to generate additional scenarios for modelling using a larger number of factors. This provides a greater depth and richness for subsequent quantification and modelling.
    2. Scenario quantification has been further improved by working with Professor Tony O’Hagan on the use of the SHeffield ELicitation Framework (SHELF) ( Elicitation is the process of obtaining knowledge from one or more experts for uncertain futures. Initially, the CfWI used the Delphi method as an elicitation process. It is an iterative process where experts provide their individual estimates for the parameters under consideration, together with their reasoning. We decided to reappraise the use of the Delphi in 2014-15 in order to improve the robustness of the elicited values. We selected SHELF as the ideal way to quantify uncertainty around parameter values in CfWI analysis. It was created by Professor Tony O’Hagan and Professor Jeremy Oakley at the University of Sheffield. In contrast to the Delphi method, it is designed for eliciting the knowledge of a group of experts in a face-to-face workshop, and capturing a probability distribution to represent their judgements. The SHELF method involves two rounds of debate on a particular question that is pertinent to an area under consideration. Experts are asked for a series of values on the question and these are later used to obtain a probability distribution curve for all experts. The SHELF form of elicitation has a well-defined protocol to correct likely biases, and supports knowledge sharing without allowing the group to be dominated by any individual. However, it is not practical to use the SHELF method for all workshops. We therefore adopt a framework in which three different levels of intensity are applied in three layers – for further information, please see elicitation methods technical paper.

We have discussed further enhancements with the Department of Health, such as a method for reusing a consistent set of scenarios across a related project. It is recommended that this be investigated further.

For further information see:

Workforce modelling

A core element of the framework is the use of system dynamics (SD) modelling to calculate workforce demand and supply. The models constructed by the CfWI vary greatly in size and complexity. The medical model developed for the medical and dental student intakes project (MDSI) contained 15 separate influence diagrams, had 997 distinct variables and was initialised with 903,525 items of data. Other models, for example dentistry, pharmacy, psychiatry and acute medicine, are of similar complexity. This places them firmly in the category of industrial-scale models, and thus there is a clear need to use a formalised, software engineering approach.

Since the development of the first SD workforce model, we have gained considerable practical experience. Decision Analysis Services Ltd[2] (DAS) has worked with the CfWI to formalise and document our approach. The benefits of this include models that are better designed, easier to use, more focussed, and more efficient. Applying a rigorous formal approach also results in increased stakeholder confidence in model outputs.

The approach is composed of four steps: model scoping, model construction, model documentation and model testing. Each step is described in detail and is supported by best practice guidance. The CfWI’s approach collates best practices from the many excellent references addressing SD model development and is informed by practitioner experience at the CfWI. Our approach provides the following:

  1. There is a clear description of how the SD modelling fits within the wider robust workforce modelling framework.
  2. Detailed guidance on SD modelling best practices is provided, along with examples drawn from our tested workforce models and checklists covering model scoping and testing.
  3. Workforce SD models are typically data-intensive, so particular attention is given to the critical activities of data gathering, loading, and testing.

Policy analysis

Policy analysis is the process of determining which workforce planning decisions are the most robust in the face of an uncertain future. The scale of the health and social care workforce, and the costs of training and employment, mean that at national level these are typically senior government decisions. However, decision-making can be made at lower levels of scale, across small workforce groups, geographic regions or individual enterprises.

Policy analysis is the final and most challenging part of the framework. It is here where information has to be critically assessed, options prioritised, and then presented to the decision-makers. Decisions are not always clear-cut. The future is, as we know, uncertain. Data and model limitations mean the outputs will always have uncertainty.

The policy analysis stage has been the subject of much research and development activity at the CfWI. This is ongoing but progress has been made in the following areas:

  1. We are understanding how policy analysis has been conducted across other disciplines as well as the NHS and workforce planning. This has been informed by a comprehensive literature review.
  2. How policy analysis links to other stages of the framework is another area being investigated. In particular we are examining the specific activities to frame and scope the nature of the policy interventions, define prospective policy levers for modelling, and agree measures of policy effectiveness.
  3. The nature of the decision-making process is another area of interest, along with how to present the impact of different policy options to decision-makers. This requires consideration of uncertainty, in addition to presenting the range of outputs across the examined scenarios.

We have noted areas of further investigation to this stage, including the development of a policy analysis tool to support robust decision-making.

For further information see:

[1] Each factor in a scenario and its future outcome or projection needs to consistent with the projections of all other factors. To give an example, a future where the economy is not doing well but health research is very well funded may not be fully consistent if these are the main factors being considered

[2] DAS provide technical consultancy to support the analysis of strategic challenges facing government and industry decision-makers using systems modelling and simulation methods.