Prognostic Models for Stable Coronary Artery Disease
Prognostic Models for Stable Coronary Artery Disease
We report findings from the CALIBER (CArdiovascular disease research using Linked BEspoke studies and Electronic Health Records) collaboration where we linked population-based primary care data from the Clinical Practice Research Datalink (CPRD) to three further sources of electronic health records: the Myocardial Ischaemia National Audit Project registry (MINAP), discharge records from Hospital Episodes Statistics (HES), and cause-specific mortality from the Office for National Statistics (ONS), as previously described. Eligible patients were those with a diagnosis of stable angina, patients with history of MI, coronary artery bypass graft (CABG), or percutaneous coronary intervention (PCI) prior to the start of the study period (other CHD) and patients with a diagnosis of ACS within the study period (unstable angina or acute MI). Myocardial infarction was classified into ST-elevation MI (STEMI) and non-ST-elevation MI (NSTEMI) where MI type was recorded (Figure 1). Diagnoses were identified in CPRD, HES, or MINAP records according to definitions in the CALIBER data manual. Stable angina was defined by Read codes in CPRD for angina diagnosis, positive ischaemia tests, coronary angiogram results recorded or repeat prescriptions for nitrates, or hospitalizations with a primary spell diagnosis ICD10 code I20.1, I20.8, or I20.9. Unstable angina was defined by Read codes in CPRD or hospital admission with ICD10 code I20.0. ST-elevation MI and NSTEMI were defined according to the discharge diagnosis as recorded in MINAP. Acute MI not otherwise specified was defined by Read codes in CPRD or ICD10 I21–I22 as the primary diagnosis in HES. Further details on the diagnostic codes and definitions used are available at http://www.caliberresearch.org/portal/.
(Enlarge Image)
Figure 1.
Study flow diagram.
Patients with prior ACS were defined as stable if they had survived more than 6 months following the acute event, entering the cohort at this point. We chose 6 months (i) to differentiate long-term prognosis from the high-risk period that typically follows ACS or revascularization and (ii) because models validated for clinical use following ACS cover the first 6 months post-ACS (e.g. GRACE).
Candidate predictors were drawn based on recommendations in recent guidelines for the management of SCAD. We included demographic measures (age, sex, ethnicity, social deprivation), SCAD subtype (stable angina, other CHD, unstable angina, MI, STEMI, NSTEMI), use of short- and long-acting nitrates, whether CABG or PCI was performed in the 6 months following CAD diagnosis, previous MI, smoking, body mass index (BMI), blood pressure, diagnosis of hypertension, diabetes, lipids, family history of coronary heart disease, CVD comorbidities [heart failure, peripheral arterial disease (PAD), atrial fibrillation, stroke], non-CVD comorbidities (chronic renal disease, chronic obstructive pulmonary disease, thyroid disorders, peptic ulcer, rheumatoid arthritis, cancer, chronic liver disease), psychosocial characteristics (depression, anxiety), and clinically assessed biomarkers (heart rate, white cell count, haemoglobin, creatinine, liver enzymes, HbA1c). We defined as baseline the most recent measurements encompassing those made up to 6 months prior to cohort entry.
The endpoints were all-cause mortality and a composite of non-fatal MI or coronary death. Patients were censored at the earliest date among date of endpoint of interest, relocation to a new primary care practice, or study end date (25 March 2010).
Prognostic models were developed and evaluated following the checklist outlined in the Transparent Reporting of a model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines. The baseline hazard for both outcomes followed an exponential distribution; hence, we developed exponential proportional hazards models. These are similar to the Cox model in that they assume proportional hazards but the baseline hazard is parametrically estimated based on the exponential model. Log of hazard vs. log of time plots showed no significant violations of the proportional hazards assumption. Nonlinear associations were modelled using splines. Multiple imputation was used to replace missing values in prognostic factors that appeared to have a missing-at-random pattern (details in Supplementary material online). All candidate models a priori included age, sex, SCAD subtype, use of long-acting nitrates, CABG/PCI in the last 6 months, previous/recurrent MI, and CVD risk factors (smoking, hypertension, diabetes, total, and HDL cholesterol). Additional predictors were selected based on their recording coverage and multivariate effect size. Because our data set is large, to decide which variables to include in the models we defined as 'moderately significant' those with P-values <0.001, and 'highly significant' those with P-values <10. Thus, we examined the multiply imputed multivariate associations of candidate predictors, adjusted for all other candidate variables (see Supplementary material online, Tables S4 and S5). Predictors with nonlinear associations (e.g. BMI) were modelled using restricted cubic splines (three knot points were sufficient). Predictors with moderate significance after imputation in the multivariate context were retained in the models if the missing data did not exceed 50%. Predictors with high significance after imputation were included if their coverage was at least 20%. Subsequently, we examined interactions between age and sex with each of the modelled predictors. For simplicity, only the age–sex interaction which was highly significant was included in the final models.
Discrimination was assessed based on Harrell's C-index. The contribution of individual prognostic factors to the C-index of the full model was assessed by backward elimination of a different variable at each iteration. The ACCF/AHA guidelines classify low- and high-risk patients as those with <1 and >3% annual mortality, respectively. We extended these cut-offs to a 5-year time horizon (low <5%, intermediate 5–14%, high >15%) to assess the net reclassification improvement (NRI) upon addition of different prognostic factors to the models. Calibration of 5-year risk predictions was visually assessed by comparing predicted vs. observed (Kaplan–Meier) risk by splitting the data into 10 subgroups with equal numbers of patients.
We compared the likely clinical impact of using our prognostic models to guide medical decisions by estimating the incremental number of life years saved by using these models over alternative models, as previously described. Briefly, suppose that among a cohort of size N a prognostic model identified n patients as high risk. Suppose that upon follow-up the observed survival in these patients was S(t). If a risk management decision associated with benefit (hazard ratio) θ was applied to high-risk patients, the anticipated risk reduction over time t would be S(t)–S(t). Estimation of the number of life years saved by using the prognostic model takes into account the number of patients identified as high risk by the model and the expected benefit and cost (medical or other) incurred over time t if specific management decisions were applied to high-risk patients. For the current analysis, we defined as high-risk patients those with ≥15% 5-year risk (20% in sensitivity analysis). We assumed that the management decision (e.g. further testing or treatments) is associated with a hazard ratio of 0.8 (~20% risk reduction) [for context, antiplatelet therapy has been shown to reduce risk of fatal/non-fatal CVD by ~20% in patients with prior MI or confirmed CAD]. Further, we assumed that the cost associated with implementing the decision balances the benefit obtained in patients with 15% 5-year risk (i.e. treating people with higher or lower observed risk results in positive or negative net benefit, respectively).
We validated models internally (in CALIBER) and externally (different study and clinical setting). Within CALIBER, we estimated bootstrap standard errors (200 samples) to obtain optimism-corrected confidence intervals. For external validation, the models developed in CALIBER were applied to 4020 patients in the 'Appropriateness of Coronary Revascularisation' (ACRE) study, a prospective cohort of patients, with similar case-mix to patients in our data, who underwent coronary angiography at the London Chest Hospital during 1996/1997 and followed up until 2004. Details of this study have been previously described. Covariates in ACRE that were incompletely recorded (smoking, TCHOL, HDL, heart rate, blood pressure, creatinine, haemoglobin, white cell count) were multiply imputed as described for the development data set, but independently of the CALIBER data. The following candidate predictors considered in our model development had not been collected by the ACRE investigators: history of anxiety, depression, cancer, liver disease, and atrial fibrillation (all binary). To allow models containing these variables to be evaluated in ACRE, we set these to 0. Discrimination and calibration statistics for 5-year risks predictions in the external data set were assessed as in the main analysis.
All analyses were performed in the R statistical package, version 2.15.2 for Unix.
Methods
Patient Population
We report findings from the CALIBER (CArdiovascular disease research using Linked BEspoke studies and Electronic Health Records) collaboration where we linked population-based primary care data from the Clinical Practice Research Datalink (CPRD) to three further sources of electronic health records: the Myocardial Ischaemia National Audit Project registry (MINAP), discharge records from Hospital Episodes Statistics (HES), and cause-specific mortality from the Office for National Statistics (ONS), as previously described. Eligible patients were those with a diagnosis of stable angina, patients with history of MI, coronary artery bypass graft (CABG), or percutaneous coronary intervention (PCI) prior to the start of the study period (other CHD) and patients with a diagnosis of ACS within the study period (unstable angina or acute MI). Myocardial infarction was classified into ST-elevation MI (STEMI) and non-ST-elevation MI (NSTEMI) where MI type was recorded (Figure 1). Diagnoses were identified in CPRD, HES, or MINAP records according to definitions in the CALIBER data manual. Stable angina was defined by Read codes in CPRD for angina diagnosis, positive ischaemia tests, coronary angiogram results recorded or repeat prescriptions for nitrates, or hospitalizations with a primary spell diagnosis ICD10 code I20.1, I20.8, or I20.9. Unstable angina was defined by Read codes in CPRD or hospital admission with ICD10 code I20.0. ST-elevation MI and NSTEMI were defined according to the discharge diagnosis as recorded in MINAP. Acute MI not otherwise specified was defined by Read codes in CPRD or ICD10 I21–I22 as the primary diagnosis in HES. Further details on the diagnostic codes and definitions used are available at http://www.caliberresearch.org/portal/.
(Enlarge Image)
Figure 1.
Study flow diagram.
Patients with prior ACS were defined as stable if they had survived more than 6 months following the acute event, entering the cohort at this point. We chose 6 months (i) to differentiate long-term prognosis from the high-risk period that typically follows ACS or revascularization and (ii) because models validated for clinical use following ACS cover the first 6 months post-ACS (e.g. GRACE).
Prognostic Factors
Candidate predictors were drawn based on recommendations in recent guidelines for the management of SCAD. We included demographic measures (age, sex, ethnicity, social deprivation), SCAD subtype (stable angina, other CHD, unstable angina, MI, STEMI, NSTEMI), use of short- and long-acting nitrates, whether CABG or PCI was performed in the 6 months following CAD diagnosis, previous MI, smoking, body mass index (BMI), blood pressure, diagnosis of hypertension, diabetes, lipids, family history of coronary heart disease, CVD comorbidities [heart failure, peripheral arterial disease (PAD), atrial fibrillation, stroke], non-CVD comorbidities (chronic renal disease, chronic obstructive pulmonary disease, thyroid disorders, peptic ulcer, rheumatoid arthritis, cancer, chronic liver disease), psychosocial characteristics (depression, anxiety), and clinically assessed biomarkers (heart rate, white cell count, haemoglobin, creatinine, liver enzymes, HbA1c). We defined as baseline the most recent measurements encompassing those made up to 6 months prior to cohort entry.
Endpoints
The endpoints were all-cause mortality and a composite of non-fatal MI or coronary death. Patients were censored at the earliest date among date of endpoint of interest, relocation to a new primary care practice, or study end date (25 March 2010).
Model Development
Prognostic models were developed and evaluated following the checklist outlined in the Transparent Reporting of a model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines. The baseline hazard for both outcomes followed an exponential distribution; hence, we developed exponential proportional hazards models. These are similar to the Cox model in that they assume proportional hazards but the baseline hazard is parametrically estimated based on the exponential model. Log of hazard vs. log of time plots showed no significant violations of the proportional hazards assumption. Nonlinear associations were modelled using splines. Multiple imputation was used to replace missing values in prognostic factors that appeared to have a missing-at-random pattern (details in Supplementary material online). All candidate models a priori included age, sex, SCAD subtype, use of long-acting nitrates, CABG/PCI in the last 6 months, previous/recurrent MI, and CVD risk factors (smoking, hypertension, diabetes, total, and HDL cholesterol). Additional predictors were selected based on their recording coverage and multivariate effect size. Because our data set is large, to decide which variables to include in the models we defined as 'moderately significant' those with P-values <0.001, and 'highly significant' those with P-values <10. Thus, we examined the multiply imputed multivariate associations of candidate predictors, adjusted for all other candidate variables (see Supplementary material online, Tables S4 and S5). Predictors with nonlinear associations (e.g. BMI) were modelled using restricted cubic splines (three knot points were sufficient). Predictors with moderate significance after imputation in the multivariate context were retained in the models if the missing data did not exceed 50%. Predictors with high significance after imputation were included if their coverage was at least 20%. Subsequently, we examined interactions between age and sex with each of the modelled predictors. For simplicity, only the age–sex interaction which was highly significant was included in the final models.
Evaluation of Prediction Performance
Discrimination was assessed based on Harrell's C-index. The contribution of individual prognostic factors to the C-index of the full model was assessed by backward elimination of a different variable at each iteration. The ACCF/AHA guidelines classify low- and high-risk patients as those with <1 and >3% annual mortality, respectively. We extended these cut-offs to a 5-year time horizon (low <5%, intermediate 5–14%, high >15%) to assess the net reclassification improvement (NRI) upon addition of different prognostic factors to the models. Calibration of 5-year risk predictions was visually assessed by comparing predicted vs. observed (Kaplan–Meier) risk by splitting the data into 10 subgroups with equal numbers of patients.
Estimation of Life Years Saved
We compared the likely clinical impact of using our prognostic models to guide medical decisions by estimating the incremental number of life years saved by using these models over alternative models, as previously described. Briefly, suppose that among a cohort of size N a prognostic model identified n patients as high risk. Suppose that upon follow-up the observed survival in these patients was S(t). If a risk management decision associated with benefit (hazard ratio) θ was applied to high-risk patients, the anticipated risk reduction over time t would be S(t)–S(t). Estimation of the number of life years saved by using the prognostic model takes into account the number of patients identified as high risk by the model and the expected benefit and cost (medical or other) incurred over time t if specific management decisions were applied to high-risk patients. For the current analysis, we defined as high-risk patients those with ≥15% 5-year risk (20% in sensitivity analysis). We assumed that the management decision (e.g. further testing or treatments) is associated with a hazard ratio of 0.8 (~20% risk reduction) [for context, antiplatelet therapy has been shown to reduce risk of fatal/non-fatal CVD by ~20% in patients with prior MI or confirmed CAD]. Further, we assumed that the cost associated with implementing the decision balances the benefit obtained in patients with 15% 5-year risk (i.e. treating people with higher or lower observed risk results in positive or negative net benefit, respectively).
Validation
We validated models internally (in CALIBER) and externally (different study and clinical setting). Within CALIBER, we estimated bootstrap standard errors (200 samples) to obtain optimism-corrected confidence intervals. For external validation, the models developed in CALIBER were applied to 4020 patients in the 'Appropriateness of Coronary Revascularisation' (ACRE) study, a prospective cohort of patients, with similar case-mix to patients in our data, who underwent coronary angiography at the London Chest Hospital during 1996/1997 and followed up until 2004. Details of this study have been previously described. Covariates in ACRE that were incompletely recorded (smoking, TCHOL, HDL, heart rate, blood pressure, creatinine, haemoglobin, white cell count) were multiply imputed as described for the development data set, but independently of the CALIBER data. The following candidate predictors considered in our model development had not been collected by the ACRE investigators: history of anxiety, depression, cancer, liver disease, and atrial fibrillation (all binary). To allow models containing these variables to be evaluated in ACRE, we set these to 0. Discrimination and calibration statistics for 5-year risks predictions in the external data set were assessed as in the main analysis.
All analyses were performed in the R statistical package, version 2.15.2 for Unix.
