Improving Assessment of Perioperative Risk in Children
Improving Assessment of Perioperative Risk in Children
Background. This study evaluated whether an objective tool would provide a more reliable and valid assessment of perioperative risk compared with the ASA-physical status (ASA-PS) in children.
Methods. A system-based risk assessment tool was developed using these categories: Neurological, Airway, Respiratory, Cardiovascular, and Other (NARCO) with a subcomponent grading surgical severity (SS). Anaesthesiologists reviewed the preoperative assessments and assigned NARCO, SS, and ASA-PS scores independently. Perioperative outcomes were recorded by trained observers. Validity and reliability of the tools were evaluated.
Results. NARCO correlated with ASA-PS (ρ=0.664; P<0.01) supporting its criterion validity. Inter-rater reliability of the measures was supported (intraclass correlation coefficients 0.71–0.96; κ 0.43–0.87) except for the Airway category. Measures of exact agreement were slightly better for NARCO compared with ASA-PS. NARCO, SS, and ASA-PS scores correlated significantly with perioperative escalation of care, adverse events (AE), hospital length of stay, and admission status. Correlations between NARCO and ASA-PS and outcomes improved when SS was factored into their coding. There were significant, but low, correlations between all measures and mortality. The odds of having escalation of care, AE, and mortality were 5–47 times greater among children with higher risk scores.
Conclusions. Findings suggest that all measures of outcome have acceptable to excellent reliability with a slight improvement in agreement for the NARCO compared with the ASA-PS. This study supports the validity of both the NARCO and the ASA-PS in predicting perioperative risk in children with a slight improvement in correlations when combined with the SS score.
Clinical scoring systems provide standardized, reproducible measures of a patient's condition or functional status. Such systems provide important measures to facilitate comparison across groups for quality improvement, research, and benchmarking, and also for health-care planning, allocation of resources, and clinical decision-making. The ASA-physical status (ASA-PS) is one such measure, developed in 1941 and later modified to its present format to describe the overall PS of patients before surgery. Owens and colleagues suggested that the ASA-PS was not intended to assess surgical risk since it does not consider the impact of the surgery on perioperative outcomes. However, the score has become widely used not only to assess the preoperative health status but also to assess perioperative risk, guide billing, and allocation of resources, adjust for case-mix differences, and to stratify subjects in clinical research. Although some studies in adults have found that the ASA-PS correlates well with perioperative morbidity and mortality, others have reported poor reliability with much inter-rater variability in the assignment of scores. The non-specific global nature of the ASA-PS and its lack of specific descriptors have led to imprecision and inconsistencies in scoring and have been cited as significant limitations of this measure. Despite these limitations, the ASA-PS is considered to be the gold standard of perioperative risk assessment against which other tools have been compared.
Data evaluating the usefulness of the ASA-PS in children are scant, yet it remains the only perioperative risk assessment tool used in children. Previous evaluations of the inter-rater reliability of the ASA-PS reported poor-to-modest agreement between anaesthesiologists who assigned scores to hypothetical paediatric case scenarios, calling into question its reliability in children. Paediatric anaesthesiologists have identified several limitations of the ASA-PS including difficulty defining 'functional limitation' in children, the lack of consideration of self-limiting illnesses or congenital abnormalities, non-specified timing of assessments, and perceptions about its reliability and validity. Despite these limitations, multiple studies support the validity of the ASA-PS as a measure of risk. Several studies reported an association between the number or severity of adverse events (AE) and ASA-PS scores in children, and others found associations between scores and morbidity or mortality after cardiac arrests. Although such studies show promise, additional reliability and validity data are needed to support the use of the ASA-PS as a measure of perioperative risk in children. Furthermore, several investigators have concluded that a new grading system designed specifically for use in children which includes acute illness and congenital malformations in its definitions is needed. Although several comprehensive perioperative risk assessment systems have been developed and tested, none addresses these limitations and none has been designed specifically for children.
The current prospective study was designed to determine whether an objective, specific, and comprehensive measure provides a more reliable and valid assessment of perioperative risk compared with the ASA-PS. The specific aims were to (i) develop an objective and specific perioperative risk classification system for children, (ii) evaluate the reliability and validity of the new system and the ASA-PS, and (iii) evaluate and compare the predictive validity of these assessment methods as measures of perioperative risk.
Abstract and Introduction
Abstract
Background. This study evaluated whether an objective tool would provide a more reliable and valid assessment of perioperative risk compared with the ASA-physical status (ASA-PS) in children.
Methods. A system-based risk assessment tool was developed using these categories: Neurological, Airway, Respiratory, Cardiovascular, and Other (NARCO) with a subcomponent grading surgical severity (SS). Anaesthesiologists reviewed the preoperative assessments and assigned NARCO, SS, and ASA-PS scores independently. Perioperative outcomes were recorded by trained observers. Validity and reliability of the tools were evaluated.
Results. NARCO correlated with ASA-PS (ρ=0.664; P<0.01) supporting its criterion validity. Inter-rater reliability of the measures was supported (intraclass correlation coefficients 0.71–0.96; κ 0.43–0.87) except for the Airway category. Measures of exact agreement were slightly better for NARCO compared with ASA-PS. NARCO, SS, and ASA-PS scores correlated significantly with perioperative escalation of care, adverse events (AE), hospital length of stay, and admission status. Correlations between NARCO and ASA-PS and outcomes improved when SS was factored into their coding. There were significant, but low, correlations between all measures and mortality. The odds of having escalation of care, AE, and mortality were 5–47 times greater among children with higher risk scores.
Conclusions. Findings suggest that all measures of outcome have acceptable to excellent reliability with a slight improvement in agreement for the NARCO compared with the ASA-PS. This study supports the validity of both the NARCO and the ASA-PS in predicting perioperative risk in children with a slight improvement in correlations when combined with the SS score.
Introduction
Clinical scoring systems provide standardized, reproducible measures of a patient's condition or functional status. Such systems provide important measures to facilitate comparison across groups for quality improvement, research, and benchmarking, and also for health-care planning, allocation of resources, and clinical decision-making. The ASA-physical status (ASA-PS) is one such measure, developed in 1941 and later modified to its present format to describe the overall PS of patients before surgery. Owens and colleagues suggested that the ASA-PS was not intended to assess surgical risk since it does not consider the impact of the surgery on perioperative outcomes. However, the score has become widely used not only to assess the preoperative health status but also to assess perioperative risk, guide billing, and allocation of resources, adjust for case-mix differences, and to stratify subjects in clinical research. Although some studies in adults have found that the ASA-PS correlates well with perioperative morbidity and mortality, others have reported poor reliability with much inter-rater variability in the assignment of scores. The non-specific global nature of the ASA-PS and its lack of specific descriptors have led to imprecision and inconsistencies in scoring and have been cited as significant limitations of this measure. Despite these limitations, the ASA-PS is considered to be the gold standard of perioperative risk assessment against which other tools have been compared.
Data evaluating the usefulness of the ASA-PS in children are scant, yet it remains the only perioperative risk assessment tool used in children. Previous evaluations of the inter-rater reliability of the ASA-PS reported poor-to-modest agreement between anaesthesiologists who assigned scores to hypothetical paediatric case scenarios, calling into question its reliability in children. Paediatric anaesthesiologists have identified several limitations of the ASA-PS including difficulty defining 'functional limitation' in children, the lack of consideration of self-limiting illnesses or congenital abnormalities, non-specified timing of assessments, and perceptions about its reliability and validity. Despite these limitations, multiple studies support the validity of the ASA-PS as a measure of risk. Several studies reported an association between the number or severity of adverse events (AE) and ASA-PS scores in children, and others found associations between scores and morbidity or mortality after cardiac arrests. Although such studies show promise, additional reliability and validity data are needed to support the use of the ASA-PS as a measure of perioperative risk in children. Furthermore, several investigators have concluded that a new grading system designed specifically for use in children which includes acute illness and congenital malformations in its definitions is needed. Although several comprehensive perioperative risk assessment systems have been developed and tested, none addresses these limitations and none has been designed specifically for children.
The current prospective study was designed to determine whether an objective, specific, and comprehensive measure provides a more reliable and valid assessment of perioperative risk compared with the ASA-PS. The specific aims were to (i) develop an objective and specific perioperative risk classification system for children, (ii) evaluate the reliability and validity of the new system and the ASA-PS, and (iii) evaluate and compare the predictive validity of these assessment methods as measures of perioperative risk.
