Elicitation methods: updated approaches to elicitation
This technical paper addresses improvements to the elicitation process used by the CfWI. As part of the ongoing improvement and refinement to the Robust Workforce Planning Framework (RWPF), we have identified five potential areas of the elicitation process that could be improved.
As part of the ongoing improvement and refinement of the RWPF, this paper considers the following improvements to elicitation methodologies used by the CfWI:
- If an alternative to the quartiles method in the SHELF methodology should be used: Would this yield usable judgements?
- Correlation: If some parameters, such as life expectancy and prevalence of long-term conditions, should clearly be correlated.
- The three-layer model: If we move away from the quartiles method in a SHELF workshop, and/or if we bring in elicitation of correlations, how will this impact on the three-layer model? How do we complete the third layer with assessments of uncertainty?
- Communicating uncertainty: The CfWI analysis produces complex projections over time; how can they most clearly and most usefully convey the uncertainties around those projections?
- Training the experts: To reduce the time that experts have to spend training in face-to-face elicitation workshops, can we develop some training that they can take before the workshop?
It concludes that the CfWI carry out the following:
- Considering that there is more accuracy associated with eliciting tertiles, it is unlikely that we will continue eliciting quartiles. Nor do we feel that it would be necessary to investigate the roulette method in further detail, simply because it’s not as easily adaptable to the three-layer model or eliciting correlation.
- an internal workshop to test whether eliciting correlated quantities and/or tertiles is practical, considering the level of resource intensity required in a SHELF workshop alone.
- The CfWI suggests that either facilitators ask for the minimum and maximum bounds as well, or they use a suitable comparable quantity whose distribution has been elicited by either SHELF or the EFSA Delphi methodology.