Heading for the Beach and the Intention to Treat
Heading for the Beach and the Intention to Treat
Hi I'm Dr. Henry Black, Clinical Professor of Internal Medicine at the New York University School of Medicine, and immediate past President of the American Society of Hypertension. I'm here with my friend and colleague Dr. Andrew Vickers from Sloan-Kettering talking about biostatistics in the 21 century. We're way beyond chi-squares now and to understand what to do, we need help from our biostatistical colleagues.
Andrew J. Vickers, MD, DPhil: Glad to be here.
Dr. Black: I've conducted some large clinical trials, Women's Health Initiatives, the SHEP [Systolic Hypertension in the Elderly Program] study, the ALLHAT [Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial], and CONVINCE [Controlled Onset Verapamil Investigation of Cardiovascular End Points]. We sometimes argue with people or they don't understand what we mean by "intention to treat" as opposed to "on protocol." People say, "How can you blame the drug for what happened when they never took it, or they only took 1 pill?" On the other hand, "How can you credit the drug when they never took it?" The statistical reasoning that I was taught was that the only time you could deal with bias was in a randomized controlled trial. Is that pretty much how you think, or has it gotten better than that?
Dr. Vickers: I'll use a very simple analogy. I live in Brooklyn, and we go out to the beach in Queens. We are sitting at home one day and we say, "Should we go to the beach today?" Being a statistician, I've kept good notes about when we went to the beach on a weekend and what happened; did we have a good time? A couple of things can happen. We can decide to stay at home and we can have a good day or a bad day. We can decide to go to the beach. We can end up at the beach and have a good day or a bad day; or what sometimes happens is we get stuck in traffic, the kids start to fight, it starts raining, and we turn around and go home again. So I'm looking at my data set, and let's imagine that we stayed at home 4 days and 50% of the time we had a good time. On 2 days we were fine and on 2 days we didn't have a good day. Let's say also that we had gone to the beach on 5 occasions, [and on] 4 of those occasions we had a great time and on the fifth occasion we got stuck in traffic, the kids started fighting, it started raining, we had a bad day and we came home. So what is the success rate for going to the beach? Would you say it is actually getting to the beach, so we have a 100% success rate if we got to the beach, or do you say it's 80% because there was that 1 day that we tried to get to the beach and we couldn't? I can't decide to be at the beach, I can only decide to try and go to the beach.
Dr. Black: That's your intention to go.
Dr. Vickers: That's my intention, so it's my intention to treat. My intention is to go to the beach, that's the only thing that I can affect. Most statisticians would say the best estimate of your success rate at the beach is 80%. It doesn't mean there is anything wrong with the beach, it doesn't mean the beach isn't a really nice place, it's just saying it can start raining or there can be bad traffic or something.
Dr. Black: That doesn't have anything to do with bias, however; that has to do with what happens to you once you have gone forward in a trial. You would try to control for things you don't understand and you don't know about that may affect the outcomes.
Dr. Vickers: No, that absolutely has to do with bias. If we take that day when the traffic was bad, now on some of the days when we stayed at home, whether the traffic was bad or not, we're going to ignore that, right?
Dr. Black: So that's your inclusion criteria. You listened to the weather report and it was going to be bad so you didn't go, you exclude bad days. Or, you heard there was a lot of traffic on the beltway so you don't go. That's a bad day, and your inclusion criteria for going was that the traffic was okay and the weather looked good. One of the things that I was taught about intention to treat (or could only use intention to treat and hope that results of "on treatment" were in the same direction) is that an intention to treat is the only time that you can be pretty sure that the things you can't measure or don't know about are balanced between group A and group B or between experimental and control.
Dr. Vickers: That's absolutely right too. The beach analogy is a very good way of thinking about intention to treat in terms of medical decision-making and policy. In terms of statistics and bias and so on, one of the examples I like to use is whether it's a partial or radical nephrectomy, whether you only take out part of the kidney as a treatment for kidney cancer or you take out the whole kidney. There are plenty of studies that seemed to show that partial nephrectomy was a very good idea and it improved cancer-specific survival, which of course is kind of impossible. You can't take out less tissue and cure more cancer. When we did a study on it, what we realized was that there were a bunch of patients in whom the doctor went in aiming to do a partial, but when in there, [the surgeon] saw that the tumor was really big and ugly and then changed to a radical nephrectomy.
Dr. Black: How would you assign that?
Dr. Vickers: In a randomized trial you would have to say that you intended to do a partial and you'd call it a partial. If you end up saying, "No, no, we're going to call it a radical because they did a radical," you're starting to move all of the bad tumors over onto the radical side, yet the point of randomization is a fair comparison between like groups. Now you've got unlike groups, you now have selection bias.
Dr. Black: A lot of this is coming up with respect to whether to do breast biopsies, whether to do radiation, whether to take out the nodes, and this has become a very hot issue in the breast cancer business, which I'm sure you have been following. How do you deal with that?
Dr. Vickers: Some people are now reporting intention-to-treat and per-protocol analyses and I have to say that generally makes me feel really uncomfortable. I think it's a fairly well-established principle that intention to treat is the way to go. If you see per-protocol analyses, there is going to have to be a very good reason to believe that those were sound.
Dr. Black: They ought to be in the same direction at least, maybe not in the same magnitude but in the same direction.
Dr. Vickers: Right, they will be in the same direction. There was a good example from the cancer literature in which there was some justification -- a study of prostate cancer screening. With prostate cancer screening, some people are randomly assigned to get screening but never show up, and some people who are told not to get screening go and get it anyway. The main results were published and they said, "Here are the results of the study, intention to treat," but they also said, "Let's see if everyone who was asked to turn up for screening did turn up for screening, and if none of the people who were asked not to did have screening, what would the results have been?" That seems to me a sensible analysis, to sort of investigate what the effects might be at a population level.
Dr. Black: Dr. Vickers thank you very much. I think I really enjoyed this conversation. I hope you have as well.
Dr. Vickers: Yes, thank you for having me.
Hi I'm Dr. Henry Black, Clinical Professor of Internal Medicine at the New York University School of Medicine, and immediate past President of the American Society of Hypertension. I'm here with my friend and colleague Dr. Andrew Vickers from Sloan-Kettering talking about biostatistics in the 21 century. We're way beyond chi-squares now and to understand what to do, we need help from our biostatistical colleagues.
Andrew J. Vickers, MD, DPhil: Glad to be here.
Dr. Black: I've conducted some large clinical trials, Women's Health Initiatives, the SHEP [Systolic Hypertension in the Elderly Program] study, the ALLHAT [Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial], and CONVINCE [Controlled Onset Verapamil Investigation of Cardiovascular End Points]. We sometimes argue with people or they don't understand what we mean by "intention to treat" as opposed to "on protocol." People say, "How can you blame the drug for what happened when they never took it, or they only took 1 pill?" On the other hand, "How can you credit the drug when they never took it?" The statistical reasoning that I was taught was that the only time you could deal with bias was in a randomized controlled trial. Is that pretty much how you think, or has it gotten better than that?
Dr. Vickers: I'll use a very simple analogy. I live in Brooklyn, and we go out to the beach in Queens. We are sitting at home one day and we say, "Should we go to the beach today?" Being a statistician, I've kept good notes about when we went to the beach on a weekend and what happened; did we have a good time? A couple of things can happen. We can decide to stay at home and we can have a good day or a bad day. We can decide to go to the beach. We can end up at the beach and have a good day or a bad day; or what sometimes happens is we get stuck in traffic, the kids start to fight, it starts raining, and we turn around and go home again. So I'm looking at my data set, and let's imagine that we stayed at home 4 days and 50% of the time we had a good time. On 2 days we were fine and on 2 days we didn't have a good day. Let's say also that we had gone to the beach on 5 occasions, [and on] 4 of those occasions we had a great time and on the fifth occasion we got stuck in traffic, the kids started fighting, it started raining, we had a bad day and we came home. So what is the success rate for going to the beach? Would you say it is actually getting to the beach, so we have a 100% success rate if we got to the beach, or do you say it's 80% because there was that 1 day that we tried to get to the beach and we couldn't? I can't decide to be at the beach, I can only decide to try and go to the beach.
Dr. Black: That's your intention to go.
Dr. Vickers: That's my intention, so it's my intention to treat. My intention is to go to the beach, that's the only thing that I can affect. Most statisticians would say the best estimate of your success rate at the beach is 80%. It doesn't mean there is anything wrong with the beach, it doesn't mean the beach isn't a really nice place, it's just saying it can start raining or there can be bad traffic or something.
Dr. Black: That doesn't have anything to do with bias, however; that has to do with what happens to you once you have gone forward in a trial. You would try to control for things you don't understand and you don't know about that may affect the outcomes.
Dr. Vickers: No, that absolutely has to do with bias. If we take that day when the traffic was bad, now on some of the days when we stayed at home, whether the traffic was bad or not, we're going to ignore that, right?
Dr. Black: So that's your inclusion criteria. You listened to the weather report and it was going to be bad so you didn't go, you exclude bad days. Or, you heard there was a lot of traffic on the beltway so you don't go. That's a bad day, and your inclusion criteria for going was that the traffic was okay and the weather looked good. One of the things that I was taught about intention to treat (or could only use intention to treat and hope that results of "on treatment" were in the same direction) is that an intention to treat is the only time that you can be pretty sure that the things you can't measure or don't know about are balanced between group A and group B or between experimental and control.
Dr. Vickers: That's absolutely right too. The beach analogy is a very good way of thinking about intention to treat in terms of medical decision-making and policy. In terms of statistics and bias and so on, one of the examples I like to use is whether it's a partial or radical nephrectomy, whether you only take out part of the kidney as a treatment for kidney cancer or you take out the whole kidney. There are plenty of studies that seemed to show that partial nephrectomy was a very good idea and it improved cancer-specific survival, which of course is kind of impossible. You can't take out less tissue and cure more cancer. When we did a study on it, what we realized was that there were a bunch of patients in whom the doctor went in aiming to do a partial, but when in there, [the surgeon] saw that the tumor was really big and ugly and then changed to a radical nephrectomy.
Dr. Black: How would you assign that?
Dr. Vickers: In a randomized trial you would have to say that you intended to do a partial and you'd call it a partial. If you end up saying, "No, no, we're going to call it a radical because they did a radical," you're starting to move all of the bad tumors over onto the radical side, yet the point of randomization is a fair comparison between like groups. Now you've got unlike groups, you now have selection bias.
Dr. Black: A lot of this is coming up with respect to whether to do breast biopsies, whether to do radiation, whether to take out the nodes, and this has become a very hot issue in the breast cancer business, which I'm sure you have been following. How do you deal with that?
Dr. Vickers: Some people are now reporting intention-to-treat and per-protocol analyses and I have to say that generally makes me feel really uncomfortable. I think it's a fairly well-established principle that intention to treat is the way to go. If you see per-protocol analyses, there is going to have to be a very good reason to believe that those were sound.
Dr. Black: They ought to be in the same direction at least, maybe not in the same magnitude but in the same direction.
Dr. Vickers: Right, they will be in the same direction. There was a good example from the cancer literature in which there was some justification -- a study of prostate cancer screening. With prostate cancer screening, some people are randomly assigned to get screening but never show up, and some people who are told not to get screening go and get it anyway. The main results were published and they said, "Here are the results of the study, intention to treat," but they also said, "Let's see if everyone who was asked to turn up for screening did turn up for screening, and if none of the people who were asked not to did have screening, what would the results have been?" That seems to me a sensible analysis, to sort of investigate what the effects might be at a population level.
Dr. Black: Dr. Vickers thank you very much. I think I really enjoyed this conversation. I hope you have as well.
Dr. Vickers: Yes, thank you for having me.
