Nitrous Oxide Anaesthesia and Mortality or CV Morbidity
Nitrous Oxide Anaesthesia and Mortality or CV Morbidity
The purpose of this systematic review was to summarize the current evidence from randomized clinical trials associating nitrous oxide with serious cardiovascular complications. Our review presents results using both conventional meta-analysis and TSA. We were able to conduct meta-analyses for short-term mortality and long-term mortality. For the other four outcomes, there was no more than one trial with low risk of bias, and we therefore did not conduct meta-analyses.
In the case of short-term mortality, the conventional meta-analysis (with low risk of bias trials) gives a point estimate indicating increased risk in the nitrous oxide group (RR 1.38) but with very wide CIs (95% CI 0.22–8.71). There is huge imprecision in the estimate provided by this evidence. This imprecision, combined with the low control event proportion and the relatively small number of patients included, means that we cannot say that the two groups are different for this outcome. Nor can we say that they are similar. For TSA, the accrued information was such a small percentage of the estimated RIS (<2%) that the boundaries were unable to be constructed. That is, the quantity of the information was too small to even be analysed with TSA. We conclude, therefore, that there is currently insufficient evidence to assess whether nitrous oxide increases the risk of short-term mortality in high-risk groups having surgery.
For the meta-analysis for short-term mortality, events were sparse and there were zero-event groups in two of the three included trials. In this situation (sparse data and zero-event groups), the choice of continuity adjustment can have a large impact on the estimate of effect (RR) and on the calculated heterogeneity (I). I affects the decision about which meta-analytic model to use, which in turn can have a large impact on the estimate of effect when the data are sparse. Each combination of effect measure, continuity adjustment, and meta-analytic model represents a different way to model the situation. When data are sparse, it is difficult to judge which model is most appropriate. We, therefore, did sensitivity analyses using each different model. The Cochrane handbook suggests using the Peto odds ratio (with fixed-effect meta-analysis) for zero-event data, and we included this in the sensitivity analysis. An extensive variation in point estimates, calculated heterogeneity (and consequently estimated RIS) was demonstrated. This variation reflects the enormous amount of uncertainty, based on these data, of the effect of nitrous oxide on short-term mortality.
Long-term mortality provides an example of how TSA can assist with interpreting the results of meta-analysis. Using conventional techniques, the point estimate was close to 1 (RR 0.94) with rather narrow CIs (95% CI 0.80–1.10). One might be tempted to conclude, on the basis of these results, that there is no difference in long-term mortality with or without nitrous oxide for this population. For the TSA for this outcome, we used a control event proportion of 25% and a relative risk reduction of 6% for the effect size (derived from included trials with a low risk of bias). To test this model, the RIS was large (24 627). However, the number of randomized patients actually accrued in this meta-analysis was <10% (1873). So concluding no effect here would be similar to stopping a trial after only 10% of the sample size had been recruited, finding narrow CIs that crossed 1.0 and concluded that an intervention had no effect (despite being grossly under-powered). Indeed, the futility boundaries constructed with TSA were not crossed, and we conclude that there is insufficient evidence to assess whether nitrous oxide increases or decreases long-term mortality by an effect estimate of 6%.
When we tested this model using effect estimates of relative risk reductions of 10%, there was also insufficient evidence. For a relative risk reduction of 20%, however, the TSA just crossed the futility boundary. The population of patients included in this meta-analysis was patients having non-cardiac surgery expected to last longer than 2 h and patients with colon cancer having colon resection expected to last longer than 2 h. For this population, we can conclude, with a type II risk of 10%, that exposure to nitrous oxide does not alter long-term mortality by 20% (number needed to harm=5). Clearly, for mortality we are clinically interested in an effect size much smaller than this for an intervention as commonly used as nitrous oxide. We therefore conclude that, for long-term mortality, for a clinically relevant effect size, we have insufficient evidence.
The major strength of this review was its thorough search. We designed our approach to maximize sensitivity. With the amount of published trial work including nitrous oxide as an intervention, this task was large, but we feel confident that our systematic and transparent approach gave us a good chance to collect a complete summary of the evidence derived from current randomized controlled trials. This systematic review did not aim to investigate all the outcomes relevant to the use of nitrous oxide. Rather, it aimed to systematically collect and analyse what we know about nitrous oxide and mortality and major cardiovascular complications, before the completion of ENIGMA II.
Modelling the significance of the findings can be challenging. TSA allows the definition of a hypothesis, with regard to control event proportions and effect size estimate, in the same way as is done for a randomized controlled trial. Moreover, heterogeneity is included. As you change the results of these parameters, that is, as you change the assumptions in the model, you may change your conclusions. So the answers in the meta-analyses depend crucially on the questions being asked, and the assumptions being made. In this systematic review, we estimated control event proportions by pooling all the no-nitrous oxide groups included in the relevant meta-analysis, the effect size was estimated from the included trials with a low risk of bias and for an a priori anticipated relative risk reduction of 10 and 20%, and we used the D present in the included studies to adjust for heterogeneity. Consequently, our conclusions answer the questions that are defined by these parameters and depend on these assumptions.
As with any attempt to synchronize evidence and to model the significance of the findings, there are limitations. Of particular note, in this review, the potential for clinical heterogeneity was large because the inclusion criteria were broad. Estimates of heterogeneity in meta-analysis can be unreliable when numbers are small. Consequently, we may have under-estimated the heterogeneity, and this fact further reduces the confidence we can have in the existing evidence.
Nitrous oxide has been investigated extensively with regard to many different primary outcomes. From our search, we did not find any randomized clinical trials that handled our outcomes as primary outcomes. In most cases, we were extracting data that were reported incidentally. The disadvantage of this is that the study design would not have been intended for our outcomes. Moreover, because our outcomes were incidental for many of these trials, we retrieved some of our data from email correspondence with authors. There were 117 publications that were eligible based on trial type, population and intervention. Of these, we were unable to contact 63. It is possible that this approach to data collection introduces bias; perhaps those whom we could not contact represent a different population of studies.
Our systematic review demonstrated the high number of outcomes—both surrogate and clinical—that have been considered in randomized controlled trials investigating nitrous oxide. This list does not include outcomes examined in observational studies, such as environmental issues and staff exposure, which would extend its size further. The length of this list reflects how many different outcomes have been considered relevant to nitrous oxide. Assessing the merits of using this drug involves evaluating the strength of evidence for how nitrous oxide affects each relevant outcome. Advantageous outcomes can then be weighed up against the disadvantageous and a judgement can be made about the merits of using the drug.
While many outcomes can alter in their relative importance, depending on circumstances and individual judgement, mortality and cardiovascular complications are always important. In most situations, these outcomes would be considered more important than all others. If nitrous oxide does not affect mortality or cardiovascular complications, then the merits of its use lie with the balance of other outcomes. If nitrous oxide does affect these important outcomes, then this information should profoundly affect whether this drug is used. Our systematic review has demonstrated that we currently do not have robust evidence for how nitrous oxide affects mortality and cardiovascular complications. We can say neither that it does not have an effect nor that it does. This lack of evidence leaves a striking hole. We look forward with interest, therefore, to the results of ENIGMA II and to the continued debate on the role of nitrous oxide in modern anaesthetic practice.
Discussion
The purpose of this systematic review was to summarize the current evidence from randomized clinical trials associating nitrous oxide with serious cardiovascular complications. Our review presents results using both conventional meta-analysis and TSA. We were able to conduct meta-analyses for short-term mortality and long-term mortality. For the other four outcomes, there was no more than one trial with low risk of bias, and we therefore did not conduct meta-analyses.
In the case of short-term mortality, the conventional meta-analysis (with low risk of bias trials) gives a point estimate indicating increased risk in the nitrous oxide group (RR 1.38) but with very wide CIs (95% CI 0.22–8.71). There is huge imprecision in the estimate provided by this evidence. This imprecision, combined with the low control event proportion and the relatively small number of patients included, means that we cannot say that the two groups are different for this outcome. Nor can we say that they are similar. For TSA, the accrued information was such a small percentage of the estimated RIS (<2%) that the boundaries were unable to be constructed. That is, the quantity of the information was too small to even be analysed with TSA. We conclude, therefore, that there is currently insufficient evidence to assess whether nitrous oxide increases the risk of short-term mortality in high-risk groups having surgery.
For the meta-analysis for short-term mortality, events were sparse and there were zero-event groups in two of the three included trials. In this situation (sparse data and zero-event groups), the choice of continuity adjustment can have a large impact on the estimate of effect (RR) and on the calculated heterogeneity (I). I affects the decision about which meta-analytic model to use, which in turn can have a large impact on the estimate of effect when the data are sparse. Each combination of effect measure, continuity adjustment, and meta-analytic model represents a different way to model the situation. When data are sparse, it is difficult to judge which model is most appropriate. We, therefore, did sensitivity analyses using each different model. The Cochrane handbook suggests using the Peto odds ratio (with fixed-effect meta-analysis) for zero-event data, and we included this in the sensitivity analysis. An extensive variation in point estimates, calculated heterogeneity (and consequently estimated RIS) was demonstrated. This variation reflects the enormous amount of uncertainty, based on these data, of the effect of nitrous oxide on short-term mortality.
Long-term mortality provides an example of how TSA can assist with interpreting the results of meta-analysis. Using conventional techniques, the point estimate was close to 1 (RR 0.94) with rather narrow CIs (95% CI 0.80–1.10). One might be tempted to conclude, on the basis of these results, that there is no difference in long-term mortality with or without nitrous oxide for this population. For the TSA for this outcome, we used a control event proportion of 25% and a relative risk reduction of 6% for the effect size (derived from included trials with a low risk of bias). To test this model, the RIS was large (24 627). However, the number of randomized patients actually accrued in this meta-analysis was <10% (1873). So concluding no effect here would be similar to stopping a trial after only 10% of the sample size had been recruited, finding narrow CIs that crossed 1.0 and concluded that an intervention had no effect (despite being grossly under-powered). Indeed, the futility boundaries constructed with TSA were not crossed, and we conclude that there is insufficient evidence to assess whether nitrous oxide increases or decreases long-term mortality by an effect estimate of 6%.
When we tested this model using effect estimates of relative risk reductions of 10%, there was also insufficient evidence. For a relative risk reduction of 20%, however, the TSA just crossed the futility boundary. The population of patients included in this meta-analysis was patients having non-cardiac surgery expected to last longer than 2 h and patients with colon cancer having colon resection expected to last longer than 2 h. For this population, we can conclude, with a type II risk of 10%, that exposure to nitrous oxide does not alter long-term mortality by 20% (number needed to harm=5). Clearly, for mortality we are clinically interested in an effect size much smaller than this for an intervention as commonly used as nitrous oxide. We therefore conclude that, for long-term mortality, for a clinically relevant effect size, we have insufficient evidence.
The major strength of this review was its thorough search. We designed our approach to maximize sensitivity. With the amount of published trial work including nitrous oxide as an intervention, this task was large, but we feel confident that our systematic and transparent approach gave us a good chance to collect a complete summary of the evidence derived from current randomized controlled trials. This systematic review did not aim to investigate all the outcomes relevant to the use of nitrous oxide. Rather, it aimed to systematically collect and analyse what we know about nitrous oxide and mortality and major cardiovascular complications, before the completion of ENIGMA II.
Modelling the significance of the findings can be challenging. TSA allows the definition of a hypothesis, with regard to control event proportions and effect size estimate, in the same way as is done for a randomized controlled trial. Moreover, heterogeneity is included. As you change the results of these parameters, that is, as you change the assumptions in the model, you may change your conclusions. So the answers in the meta-analyses depend crucially on the questions being asked, and the assumptions being made. In this systematic review, we estimated control event proportions by pooling all the no-nitrous oxide groups included in the relevant meta-analysis, the effect size was estimated from the included trials with a low risk of bias and for an a priori anticipated relative risk reduction of 10 and 20%, and we used the D present in the included studies to adjust for heterogeneity. Consequently, our conclusions answer the questions that are defined by these parameters and depend on these assumptions.
As with any attempt to synchronize evidence and to model the significance of the findings, there are limitations. Of particular note, in this review, the potential for clinical heterogeneity was large because the inclusion criteria were broad. Estimates of heterogeneity in meta-analysis can be unreliable when numbers are small. Consequently, we may have under-estimated the heterogeneity, and this fact further reduces the confidence we can have in the existing evidence.
Nitrous oxide has been investigated extensively with regard to many different primary outcomes. From our search, we did not find any randomized clinical trials that handled our outcomes as primary outcomes. In most cases, we were extracting data that were reported incidentally. The disadvantage of this is that the study design would not have been intended for our outcomes. Moreover, because our outcomes were incidental for many of these trials, we retrieved some of our data from email correspondence with authors. There were 117 publications that were eligible based on trial type, population and intervention. Of these, we were unable to contact 63. It is possible that this approach to data collection introduces bias; perhaps those whom we could not contact represent a different population of studies.
Our systematic review demonstrated the high number of outcomes—both surrogate and clinical—that have been considered in randomized controlled trials investigating nitrous oxide. This list does not include outcomes examined in observational studies, such as environmental issues and staff exposure, which would extend its size further. The length of this list reflects how many different outcomes have been considered relevant to nitrous oxide. Assessing the merits of using this drug involves evaluating the strength of evidence for how nitrous oxide affects each relevant outcome. Advantageous outcomes can then be weighed up against the disadvantageous and a judgement can be made about the merits of using the drug.
While many outcomes can alter in their relative importance, depending on circumstances and individual judgement, mortality and cardiovascular complications are always important. In most situations, these outcomes would be considered more important than all others. If nitrous oxide does not affect mortality or cardiovascular complications, then the merits of its use lie with the balance of other outcomes. If nitrous oxide does affect these important outcomes, then this information should profoundly affect whether this drug is used. Our systematic review has demonstrated that we currently do not have robust evidence for how nitrous oxide affects mortality and cardiovascular complications. We can say neither that it does not have an effect nor that it does. This lack of evidence leaves a striking hole. We look forward with interest, therefore, to the results of ENIGMA II and to the continued debate on the role of nitrous oxide in modern anaesthetic practice.
