Risk Prediction Models for Contrast Induced Nephropathy
Risk Prediction Models for Contrast Induced Nephropathy
We performed this systematic review in accordance with the preferred reporting items for systematic reviews and meta-analyses guidelines. Our objective was to systematically review prediction models for contrast induced nephropathy.
We used a strategy developed with a health informatics specialist to search Ovid Medline (1946 to 9 March 2015), Embase (1947 to week 10 in 2015), and CINAHL (cumulative index to nursing and allied health literature; 1993 to March 2015). We reviewed the bibliographies of identified articles to locate further eligible studies. The web appendix shows the search strategies performed.
Studies published in the English language were eligible for inclusion if they evaluated the characteristics of a predictive model for identifying patients at risk of contrast induced nephropathy among adults undergoing a diagnostic or interventional procedure that used conventional, iodinated radiocontrast (media used for computed tomography (CT) or angiography, and not gadolinium based contrast). Because a set of predictive factors derived in only one population could lack validity and applicability, we only included studies in which both development and validation of the prediction model was conducted. We did not prespecify the method of validation, nor did we exclude studies where the derivation and validation cohorts were drawn from the same population. We excluded unpublished conference abstracts.
Owing to the anticipated heterogeneity in the criteria for contrast induced nephropathy between studies and the well described association between even mild elevations of serum creatinine levels and adverse outcomes, we accepted each study's definition of the disorder. These included relative or absolute increases in serum creatinine after contrast exposure.
Two authors (ZH and SAS) scanned titles and abstracts for initial selection. Selected articles were reviewed in full and independently assessed for eligibility by the same two reviewers. Discrepancies were resolved by consensus.
From each study, we abstracted data on baseline patient characteristics, procedural characteristics, criteria to define contrast induced nephropathy, the number of events, predictor variables included in the risk model, internal and external validation, measures of discrimination, measures of calibration, and methodological features indicative of study quality. To facilitate a comparison of predictor variables, we grouped final model variables into six categories: demographic data, anthropometric data, medical history, physical examination and clinical presentation, procedural characteristics, and laboratory values.
We evaluated the internal validity of each model by examining model discrimination, calibration, and reclassification. The concordance (the C statistic) of the prediction tool was used as a measure of discrimination; however, other performance statistics such as sensitivity and specificity were included if the C statistic was not reported. The C statistic is equivalent to the area under the curve, and represents the model's ability to distinguish patients who will develop contrast induced nephropathy from those who will not. C statistic values range from 0.5 (no discrimination, no better than chance) to 1.0 (perfect discrimination). A C statistic of 0.7–0.8 indicates modest discriminative ability, while a C statistic greater than 0.8 indicates good discriminative ability.
Model calibration was measured by the Hosmer-Lemeshow statistic, which refers to the concordance between observed and predicted risks. A Hosmer-Lemeshow statistic with a small P value indicates poor calibration. If the Hosmer-Lemeshow statistic was not reported, we reported the range of observed rates of contrast induced nephropathy from the lowest to highest predicted risk groupings. Reclassification was evaluated by net reclassification improvement. Net reclassification improvement refers to the proportion of individuals who, after incorporating the prediction tool, are reclassified to a risk stratum that is a better reflection of their actual outcome. The net reclassification improvement indicates the frequency with which appropriate reclassification occurs compared to inappropriate reclassification with use of the new model. For this test, a value of P<0.05 suggests that a significantly greater number of patients are being reclassified appropriately than are being reclassified inappropriately.
We assessed study quality using a modification of the criteria recommended by Hayden and colleagues. The criteria involve assessment of seven categories related to study participation (sampling bias), study attrition (attrition bias), prognostic factor selection, prognostic factor measurement, outcome measurement (ascertainment bias), statistical analysis, and model performance (discrimination, calibration). These criteria are explained further in the web appendix.
Similar to a previous systematic review, we also assessed the clinical usefulness of each study, which was defined as the combination of clinical utility and usability. For clinical utility (the effect on a clinical decision linked to a risk category or threshold), we assessed whether authors linked their models to specific risk categories and discussed how the risk categories would aid diagnostic evaluations. For usability (the availability of a clinical decision aid), we noted whether authors included a calculator or risk score that would facilitate knowledge translation and use at the bedside. These criteria are explained further in the web appendix. We also evaluated the generalisability of each prediction model by determining whether it had been externally validated in an independent patient population, either in the original or a subsequent publication.
We qualitatively synthesised results focusing on the populations in which the risk score had been tested, the types of variables contained within the prediction models, model discrimination, external validation and practical aspects of model implementation. We did not perform meta-analyses because the included studies were too heterogeneous.
No patients were involved in setting the research question or the outcome measures, nor were they involved in the design and implementation of the study. There are no plans to involve patients in dissemination.
Methods
We performed this systematic review in accordance with the preferred reporting items for systematic reviews and meta-analyses guidelines. Our objective was to systematically review prediction models for contrast induced nephropathy.
Data Sources and Searches
We used a strategy developed with a health informatics specialist to search Ovid Medline (1946 to 9 March 2015), Embase (1947 to week 10 in 2015), and CINAHL (cumulative index to nursing and allied health literature; 1993 to March 2015). We reviewed the bibliographies of identified articles to locate further eligible studies. The web appendix shows the search strategies performed.
Study Selection
Studies published in the English language were eligible for inclusion if they evaluated the characteristics of a predictive model for identifying patients at risk of contrast induced nephropathy among adults undergoing a diagnostic or interventional procedure that used conventional, iodinated radiocontrast (media used for computed tomography (CT) or angiography, and not gadolinium based contrast). Because a set of predictive factors derived in only one population could lack validity and applicability, we only included studies in which both development and validation of the prediction model was conducted. We did not prespecify the method of validation, nor did we exclude studies where the derivation and validation cohorts were drawn from the same population. We excluded unpublished conference abstracts.
Owing to the anticipated heterogeneity in the criteria for contrast induced nephropathy between studies and the well described association between even mild elevations of serum creatinine levels and adverse outcomes, we accepted each study's definition of the disorder. These included relative or absolute increases in serum creatinine after contrast exposure.
Two authors (ZH and SAS) scanned titles and abstracts for initial selection. Selected articles were reviewed in full and independently assessed for eligibility by the same two reviewers. Discrepancies were resolved by consensus.
Data Extraction
From each study, we abstracted data on baseline patient characteristics, procedural characteristics, criteria to define contrast induced nephropathy, the number of events, predictor variables included in the risk model, internal and external validation, measures of discrimination, measures of calibration, and methodological features indicative of study quality. To facilitate a comparison of predictor variables, we grouped final model variables into six categories: demographic data, anthropometric data, medical history, physical examination and clinical presentation, procedural characteristics, and laboratory values.
Model Performance
We evaluated the internal validity of each model by examining model discrimination, calibration, and reclassification. The concordance (the C statistic) of the prediction tool was used as a measure of discrimination; however, other performance statistics such as sensitivity and specificity were included if the C statistic was not reported. The C statistic is equivalent to the area under the curve, and represents the model's ability to distinguish patients who will develop contrast induced nephropathy from those who will not. C statistic values range from 0.5 (no discrimination, no better than chance) to 1.0 (perfect discrimination). A C statistic of 0.7–0.8 indicates modest discriminative ability, while a C statistic greater than 0.8 indicates good discriminative ability.
Model calibration was measured by the Hosmer-Lemeshow statistic, which refers to the concordance between observed and predicted risks. A Hosmer-Lemeshow statistic with a small P value indicates poor calibration. If the Hosmer-Lemeshow statistic was not reported, we reported the range of observed rates of contrast induced nephropathy from the lowest to highest predicted risk groupings. Reclassification was evaluated by net reclassification improvement. Net reclassification improvement refers to the proportion of individuals who, after incorporating the prediction tool, are reclassified to a risk stratum that is a better reflection of their actual outcome. The net reclassification improvement indicates the frequency with which appropriate reclassification occurs compared to inappropriate reclassification with use of the new model. For this test, a value of P<0.05 suggests that a significantly greater number of patients are being reclassified appropriately than are being reclassified inappropriately.
Quality Assessment, Clinical Usefulness, and External Validation
We assessed study quality using a modification of the criteria recommended by Hayden and colleagues. The criteria involve assessment of seven categories related to study participation (sampling bias), study attrition (attrition bias), prognostic factor selection, prognostic factor measurement, outcome measurement (ascertainment bias), statistical analysis, and model performance (discrimination, calibration). These criteria are explained further in the web appendix.
Similar to a previous systematic review, we also assessed the clinical usefulness of each study, which was defined as the combination of clinical utility and usability. For clinical utility (the effect on a clinical decision linked to a risk category or threshold), we assessed whether authors linked their models to specific risk categories and discussed how the risk categories would aid diagnostic evaluations. For usability (the availability of a clinical decision aid), we noted whether authors included a calculator or risk score that would facilitate knowledge translation and use at the bedside. These criteria are explained further in the web appendix. We also evaluated the generalisability of each prediction model by determining whether it had been externally validated in an independent patient population, either in the original or a subsequent publication.
Data Synthesis
We qualitatively synthesised results focusing on the populations in which the risk score had been tested, the types of variables contained within the prediction models, model discrimination, external validation and practical aspects of model implementation. We did not perform meta-analyses because the included studies were too heterogeneous.
Patient Involvement
No patients were involved in setting the research question or the outcome measures, nor were they involved in the design and implementation of the study. There are no plans to involve patients in dissemination.
