Appropriateness of Use of Coronary Revascularization Procedures
Appropriateness of Use of Coronary Revascularization Procedures
Background: Evidence from numerous studies of coronary angiography show differences between observers' assessments of 15% to 45%. The implication of this variation is serious: If readings are erroneous, some patients will undergo revascularization procedures unnecessarily and others will be denied an essential treatment. We evaluated the variation in interpretation of angiograms and its potential effect on appropriateness of use of revascularization procedures.
Methods and Results: Angiograms of 308 randomly selected patients previously studied for appropriateness of angiography, coronary artery bypass grafting (CABG), and percutaneous transluminal coronary angioplasty (PTCA) were interpreted by a blinded panel of 3 experienced angiographers and compared with the original interpretations. The potential effect on differences on the appropriateness of revascularization was assessed by use of the RAND criteria. Technical deficiencies were found in 52% of cases. Panel readings tended to show less significant disease (none in 16% of vessels previously read as showing significant disease), less severity of stenosis (43% lower, 6% higher), and lower extent of disease (23% less, 6% more). The classification of CABG changed from necessary/appropriate to uncertain/inappropriate for 17% to 33% of cases when individual ratings were replaced by panel readings.
Conclusions: The general level of technical quality of coronary angiography is unsatisfactory. Variation in the interpretation of angiograms was substantial in all measures and tended to be higher in individual than in panel readings. The effect was to lead to a potential overestimation of appropriateness of use of CABG by 17% and of PTCA by 10%. These findings indicate the need for increased attention to the technical quality of studies and an independent second reading for angiograms before recommending revascularization.
Recent studies of the appropriateness of the use of coronary artery bypass graft surgery (CABG) and percutaneous transluminal coronary angioplasty (PTCA) in New York State showed low rates of inappropriate use of these procedures. The validity of those assessments depends on the credibility of the data that were used to determine appropriateness. These data include clinical symptoms, response to medical therapy, results of laboratory studies such as the exercise stress test, findings on coronary angiography, and the presence of risk factors such as hypertension, diabetes, and peripheral vascular disease. Of those data, the most important for determining feasibility of revascularization is the extent of disease of the coronary arteries. The extent of coronary artery disease is measured by the number of vessels involved and the severity of their obstructions. Patients with significant left main disease or 3-vessel disease, for example, are often appropriate candidates for bypass surgery because controlled trials have shown CABG improves long-term survival in these patients. By contrast, CABG offers little survival advantage for most patients with single-vessel disease, who are often more appropriately treated medically or by angioplasty.
Both the number of vessels affected and the severity of their stenoses are determined by coronary angiography. The American College of Cardiology has defined significant coronary artery disease as evidence by angiography of at least 50% narrowing of the lumen of the artery. Clearly, much depends on the accuracy of the angiographic assessment of the extent of stenosis. Unfortunately, the reproducibility of interpretations of angiograms is far from perfect. Evidence from a number of studies indicates that differences between observers' assessments of the presence of significant disease range from 15% to 45%. When individual readings are compared with a "standard," such as interpretation by a group of experts or quantitative coronary angiography, 28% to 31% are found to be erroneous.
With the exception of the Coronary Artery Surgical Study (CASS), these studies of rater reliability have been characterized by small numbers of observers and small numbers of angiograms, often in a single institution. Although the CASS study included patients from many regions of the country, most were treated in academic medical centers. Thus none of the previous studies is truly representative of the standard of community practice.
The implication of variation in interpretation of angiograms is serious: If readings are erroneous, some patients will undergo revascularization procedures unnecessarily and others will be denied an essential treatment. If, in the previous study in New York State, variation in angiographic interpretation was substantial, the findings of little inappropriate use of revascularization must be reconsidered because the judgments depended so much on the number of vessels with significant disease.
Because of these concerns, we assessed the interpretation of angiograms in a sample of patients from our previous study of appropriateness of CABG, PTCA, and coronary angiography in New York by comparing the hospital cardiologists' interpretations with those of a blinded panel of cardiologists. We asked 3 questions: (1) What is the overall reproducibility of interpretation of coronary angiograms? (2) Is there evidence of upward bias in readings? (3) To the extent that variation is present, how would that alter the previous evaluations of appropriateness of use of CABG and PTCA?
Background: Evidence from numerous studies of coronary angiography show differences between observers' assessments of 15% to 45%. The implication of this variation is serious: If readings are erroneous, some patients will undergo revascularization procedures unnecessarily and others will be denied an essential treatment. We evaluated the variation in interpretation of angiograms and its potential effect on appropriateness of use of revascularization procedures.
Methods and Results: Angiograms of 308 randomly selected patients previously studied for appropriateness of angiography, coronary artery bypass grafting (CABG), and percutaneous transluminal coronary angioplasty (PTCA) were interpreted by a blinded panel of 3 experienced angiographers and compared with the original interpretations. The potential effect on differences on the appropriateness of revascularization was assessed by use of the RAND criteria. Technical deficiencies were found in 52% of cases. Panel readings tended to show less significant disease (none in 16% of vessels previously read as showing significant disease), less severity of stenosis (43% lower, 6% higher), and lower extent of disease (23% less, 6% more). The classification of CABG changed from necessary/appropriate to uncertain/inappropriate for 17% to 33% of cases when individual ratings were replaced by panel readings.
Conclusions: The general level of technical quality of coronary angiography is unsatisfactory. Variation in the interpretation of angiograms was substantial in all measures and tended to be higher in individual than in panel readings. The effect was to lead to a potential overestimation of appropriateness of use of CABG by 17% and of PTCA by 10%. These findings indicate the need for increased attention to the technical quality of studies and an independent second reading for angiograms before recommending revascularization.
Recent studies of the appropriateness of the use of coronary artery bypass graft surgery (CABG) and percutaneous transluminal coronary angioplasty (PTCA) in New York State showed low rates of inappropriate use of these procedures. The validity of those assessments depends on the credibility of the data that were used to determine appropriateness. These data include clinical symptoms, response to medical therapy, results of laboratory studies such as the exercise stress test, findings on coronary angiography, and the presence of risk factors such as hypertension, diabetes, and peripheral vascular disease. Of those data, the most important for determining feasibility of revascularization is the extent of disease of the coronary arteries. The extent of coronary artery disease is measured by the number of vessels involved and the severity of their obstructions. Patients with significant left main disease or 3-vessel disease, for example, are often appropriate candidates for bypass surgery because controlled trials have shown CABG improves long-term survival in these patients. By contrast, CABG offers little survival advantage for most patients with single-vessel disease, who are often more appropriately treated medically or by angioplasty.
Both the number of vessels affected and the severity of their stenoses are determined by coronary angiography. The American College of Cardiology has defined significant coronary artery disease as evidence by angiography of at least 50% narrowing of the lumen of the artery. Clearly, much depends on the accuracy of the angiographic assessment of the extent of stenosis. Unfortunately, the reproducibility of interpretations of angiograms is far from perfect. Evidence from a number of studies indicates that differences between observers' assessments of the presence of significant disease range from 15% to 45%. When individual readings are compared with a "standard," such as interpretation by a group of experts or quantitative coronary angiography, 28% to 31% are found to be erroneous.
With the exception of the Coronary Artery Surgical Study (CASS), these studies of rater reliability have been characterized by small numbers of observers and small numbers of angiograms, often in a single institution. Although the CASS study included patients from many regions of the country, most were treated in academic medical centers. Thus none of the previous studies is truly representative of the standard of community practice.
The implication of variation in interpretation of angiograms is serious: If readings are erroneous, some patients will undergo revascularization procedures unnecessarily and others will be denied an essential treatment. If, in the previous study in New York State, variation in angiographic interpretation was substantial, the findings of little inappropriate use of revascularization must be reconsidered because the judgments depended so much on the number of vessels with significant disease.
Because of these concerns, we assessed the interpretation of angiograms in a sample of patients from our previous study of appropriateness of CABG, PTCA, and coronary angiography in New York by comparing the hospital cardiologists' interpretations with those of a blinded panel of cardiologists. We asked 3 questions: (1) What is the overall reproducibility of interpretation of coronary angiograms? (2) Is there evidence of upward bias in readings? (3) To the extent that variation is present, how would that alter the previous evaluations of appropriateness of use of CABG and PTCA?
