The CfWI has developed a simple set of metrics to help users select workforce planning models, and to help model developers improve model quality.
In workforce planning, a model is a mathematical representation of the essential aspects of the system that is under investigation – for example, workforce supply or demand. These models can take many forms, from spreadsheet-based calculations to stochastic models running on specialist software.
Whatever the form, all models are simplified and limited representations of the real world. Consequently it is not possible to perfectly validate or verify a model since they all differ from reality in some way.
However, workforce planners need to know whether a model is a sound basis for decision-making. Our aim is to enable planners to choose the best model available for the purpose in hand, ensuring they are informed of the limitations so that they do not misuse the model.
Consequently, we have developed a simple set of metrics to help users select workforce planning models, and to help model developers improve model quality. Understanding a model’s limitations allows it to be improved. Ultimately it will enable us to provide or recommend the best available models.
Model users need to understand the overall suitability of a model for their purpose. They need to establish whether a model is well-defined, credible in terms of the structure, robustness and sensitivity, and is easily usable.
Most users do not have the time or experience to examine each potential model and determine if it is suitable. To help them we have defined a simple set of measures called workforce model maturity measures and applied them to real-life models.
Selected workforce models have been tested to see if they are:
- well-defined, meaning if the expected capabilities of the intended users are known, and the purpose and boundaries of the model are clearly defined
- credible, so the model does what it says it will do, and so the physical and decision-making structure of the model is appropriate to the intended purpose.
- usable, in that the model is practical, easy to use, well documented and produces useful outputs.
Three levels of maturity have been defined:
- Level 1 is the base or minimum acceptable level. Models at level 1 will be reliable and easy to use.
- Level 2 models are at an improved level of maturity, better defined and more robust.
- Level 3 models are the most optimised level, rigorously defined, tested and reviewed.
The Workforce Model Maturity measures are listed in table 1.
Table 1: Workforce Model Maturity measures
|Area||Level 1: Acceptable||Level 2: Improved||Level 3: Optimised|
These measures allow the user of a potential model to assess risk.
A model at Level 1 may be a better model for a particular purpose than one at Level 3, but one at Level 3 may be less risky for an inexperienced user as it must be fully documented and peer reviewed.
A model must meet or exceed the maturity measures of a particular level in order to achieve that level. For example, a model that achieves Level 2 for ‘definition’ and ‘credibility’, but is only at Level 1 for ‘usability’, is still considered to be at Level 1.
Workforce model assessment
The CfWI and our partners have assessed over 50 workforce models against the workforce maturity metrics measures, and identified those which have achieved a level of 1 or higher.
The models selected for assessment were those that CfWI and partners were most familiar with. Due to time constraints, this assessment has excluded models from other countries. We intend to extend the model assessment over time to include a wider range of UK and non-UK models.
Table 2 provides a brief description of the scope and purpose of each model. Contact us if you would like more information about these models.
Table 2: Workforce Model Maturity
|Generic supply modelling tool for medical and non-medical specialities||CfWI||Modelling of the estimated future supply of workforce for any given staff group.|
|High-level demand model||CfWI||Creation of demand scenarios for the whole of the health and social care workforce, using the eight Darzi pathways.|
|Patient pathway model||CfWI||Investigation of the workforce implications for a given patient pathway episode, based on an audiology model used in West Midlands SHA.|
|System dynamics workforce supply models||DH Workforce Directorate||System dynamics modelling of medical workforce supply, which supports analysis of options around training numbers and policy.|
|Frameworks for workforce demand||DH Workforce Directorate||Modelling of medical and non-medical workforce demand, which supports analysis of how supply needs to change to meet expected demand.
|Workforce supply models||DH Workforce Directorate||Modelling of medical and non-medical workforce supply, which supports analysis of options around training numbers and policy.
|Workforce provision inequality models||DH Workforce Directorate||Modelling how inequalities in healthcare provision will develop with time, which supports the allocations of training posts to under-doctored areas.
|Speciality training recruitment model||DH Workforce Directorate||Modelling of which cohorts of doctors or other specialities are most likely to fill training posts, and which are at risk of being over- or under-filled.
|Children’s Workforce Development Council (CWDC) Educational Psychologists workforce model||CfWI||Modelling the estimated future supply of workforce for educational psychologists.
|East Midlands care pathway models||East Midlands SHA||
System dynamics models of GP supply and care pathways, including long-term conditions, maternity and neonatal.
|Scenario generator||NHS Institute for Innovation and Improvement||Discrete event simulation model of a whole healthcare delivery system, represented by a care pathway.
The CfWI intends to publish updates to this list as existing models are refined and further models developed. We are also working on a more comprehensive approach to evaluating model quality, and will be providing guidance to model authors on how to achieve a higher maturity level.