Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

Background and aimsPrognostic models provide evidence-based predictions and estimates of future outcomes, facilitating decision-making, patient care, and research. A few of these models have been externally validated, leading to uncertain reliability and generalizability. This study aims to externally validate four models to assess their transferability and usefulness in clinical practice. The models include the respiratory index of severity in children (RISC)-Malawi model and three other models by Lowlavaar et al.MethodsThe study used data from the Clinical Information Network (CIN) to validate the four models where the primary outcome was in-hospital mortality. 163,329 patients met eligibility criteria. Missing data were imputed, and the logistic function was used to compute predicted risk of in-hospital mortality. Models' discriminatory ability and calibration were determined using area under the curve (AUC), calibration slope, and intercept.ResultsThe RISC-Malawi model had 50,669 pneumonia patients who met the eligibility criteria, of which the case-fatality ratio was 4406 (8.7%). Its AUC was 0.77 (95% CI: 0.77-0.78), whereas the calibration slope was 1.04 (95% CI: 1.00 -1.06), and calibration intercept was 0.81 (95% CI: 0.77-0.84). Regarding the external validation of Lowlavaar et al. models, 10,782 eligible patients  were included, with an in-hospital mortality rate of 5.3%. The primary model's AUC was 0.75 (95% CI: 0.72-0.77), the calibration slope was 0.78 (95% CI: 0.71-0.84), and the calibration intercept was 0.37 (95% CI: 0.28-0.46). All models markedly underestimated the risk of mortality.ConclusionAll externally validated models exhibited either underestimation or overestimation of the risk as judged from calibration statistics. Hence, applying these models with confidence in settings other than their original development context may not be advisable. Our findings strongly suggest the need for recalibrating these model to enhance their generalizability.

Original publication

DOI

10.1002/hsr2.1433

Type

Journal

Health science reports

Publication Date

08/2023

Volume

6

Addresses

Department of Mathematics University of Nairobi Nairobi Kenya.