Development and validation of the new ICNARC model for prediction of acute hospital mortality in adult critical care

Abstract

Purpose

To develop and validate an improved risk model to predict acute hospital mortality for admissions to adult critical care units in the UK.

Materials and methods

155,239 admissions to 232 adult critical care units in England, Wales and Northern Ireland between January and December 2012 were used to develop a risk model from a set of 38 candidate predictors. The model was validated using 90,017 admissions between January and September 2013.

Results

The final model incorporated 15 physiological predictors (modelled with continuous nonlinear models), age, dependency prior to hospital admission, chronic liver disease, metastatic disease, haematological malignancy, CPR prior to admission, location prior to admission/urgency of admission, primary reason for admission and interaction terms. The model was well calibrated and outperformed the current ICNARC model on measures of discrimination (area under the receiver operating characteristic curve 0.885 versus 0.869) and model fit (Brier's score 0.108 versus 0.115). On average, the new model reclassified patients into more appropriate risk categories (net reclassification improvement 19.9; P < 0.0001). The model performed well across patient subgroups and in specialist critical care units.

Conclusions

The risk model developed in this study showed excellent discrimination and calibration and when validated on a different period of time and across different types of critical care unit. This in turn allows improved accuracy of comparisons between UK critical care providers.

Introduction

Risk prediction models for critical care take information from early in a patient's critical illness to make a prediction about the patient's likely outcome. They are predominantly aimed at the health care provider level – allowing fair comparisons between different providers, adjusted for case mix. In the UK, the risk prediction model used to underpin the national comparative benchmarking programme for adult critical care is the Intensive Care National Audit & Research Centre (ICNARC) model [1].

As critical care populations change and new diagnostic, therapeutic and prognostic techniques become available, risk prediction models need to be updated and improved [2]. Many of the existing risk prediction models for critical care, including the ICNARC model, are constructed around a numeric severity score. While this approach has the advantage that the score can be used as a simple description of severity of illness – either for a population or an individual patient – such an approach restricts the statistical techniques that can be used for risk prediction. It is well-recognised that dividing continuous predictors, such as physiology, into categories is not the best approach to modelling [3].

The aim of our study was to develop and validate a new predictive model to improve the accuracy of comparisons between UK critical care providers. To achieve this, we decided to leave behind the paradigm of the severity score, allowing us to employ up-to-date statistical techniques for handling missing data and for continuous nonlinear modelling of physiological parameters.