Photo by Esther Dyson
New research suggests patient behavior can have a serious impact on the effectiveness of a treatment, and procedures used in double-blind randomized controlled trials (DBRCTs) may not allow researchers to assess the effects of behavior on treatment.
To solve this problem, a group of investigators has proposed a new trial design, called a two-by-two trial, that can account for behavior-treatment interactions.
They described this design in PLOS ONE.
Study rationale
The study authors pointed out that patient behaviors can directly relate to a trial. For example, a patient who believes in the drug might religiously stick to his or her treatment regimen, while someone more skeptical might skip a few doses.
Alternatively, patient behaviors may relate to how the person acts in general, such as preferences in diet, exercise, and social engagement. And in the design of today’s standard trials, these behaviors are not accounted for, said author Erik Snowberg, PhD, of the California Institute of Technology in Pasadena.
He noted that DBRCTs typically assign patients to an experimental group that receives the new treatment and a control group that does not. As the trial is double-blinded, neither the subjects nor their doctors know who falls into which group.
However, patients do know their probability of getting the treatment, and that 50% likelihood of getting the new treatment might not be enough to encourage a patient to change behaviors that could influence the efficacy of the drug, Dr Snowberg said.
For example, if a subject wants to lose weight and knows he has a high probability—a 70% chance—of being in the experimental group for a new weight loss drug, he may be more likely to take the drug as directed and to make other healthy lifestyle choices that can contribute to weight loss. As a result, he might lose more weight, boosting the apparent effectiveness of the drug.
However, if the subject knows he only has a 30% chance of being in the experimental group, he might be less motivated to both take the drug as directed and to make those other changes. As a result, he might lose less weight—even if he is in the treatment group—and the same drug would seem less effective.
“Most medical research just wants to know if a drug will work or not,” Dr Snowberg said. “We wanted to go a step further, designing new trials that would take into account the way people behave. As social scientists, we naturally turned to the mathematical tools of formal social science to do this.”
New trial design
Dr Snowberg and his colleagues found that, with the two-by-two trial, they can tease out the effects of behavior and the interaction of behavior and treatment, as well as the effects of treatment alone. The new trial design, which still randomizes treatment, also randomizes the probability of treatment, which can change a patient’s behavior.
In a two-by-two trial, instead of patients first being assigned to either the experimental or control groups, they are randomly assigned to either a “high probability of treatment” group or a “low probability of treatment” group.
The patients in the high probability group are then randomized to either the treatment or the control group, giving them a 70% chance of receiving the treatment. Patients in the low probability group are also randomized to treatment or control, and their likelihood of receiving the treatment is 30%. The patients are then informed of their probability of treatment.
By randomizing both the treatment and the probability of treatment, researchers can quantify the effects of treatment, the effects of behavior, and the effects of the interaction between treatment and behavior, Dr Snowberg said. And determining each is essential for understanding the overall efficacy of treatment.
“It’s a very small change to the design of the trial, but it’s important,” he said. “The effect of a treatment has these two constituent parts: pure treatment effect and the treatment-behavior interaction. Standard blind trials just randomize the likelihood of treatment, so you can’t see this interaction. Although you can’t just tell someone to randomize their behavior, we found a way that you can randomize the probability that a patient will get something that will change their behavior.”
Testing the design
Because it is difficult to implement new trial design changes in active trials, Dr Snowberg and his colleagues wanted to first test their idea with a meta-analysis of data from previous clinical trials. They devised a new mathematical formula that can be used to analyze DBRCT data.
The formula teases out the health outcomes resulting from treatment alone as well as outcomes resulting from an interaction between treatment and behavior.
The investigators used the formula to analyze 6 DBRCTs evaluating the antidepressants imipramine (a tricyclic antidepressant also known as Tofranil) and paroxetine (a selective serotonin reuptake inhibitor sold as Paxil).
First, the researchers wanted to see if there was evidence that patients behave differently when they have a high probability of treatment and when they have a low probability of treatment.
The trials recorded how many patients dropped out of the study, so this was the behavior Dr Snowberg and his colleagues analyzed. They found that, in trials where patients happened to have a relatively high probability of treatment—near 70%—the dropout rate was significantly lower than in trials where patients had a lower probability of treatment—around 50%.
Although the team did not have any specific behaviors to analyze, other than dropping out of the study, they also wanted to determine if behavior in general could have added to the effect of the treatments.
Using their statistical techniques, the investigators found that imipramine seemed to have a pure treatment effect but no effect from the interaction between treatment and behavior. That is, the drug seemed to work fine, regardless of any behavioral differences that may have been present.
The researchers also found that paroxetine seemed to have no effect from the treatment alone or behavior alone. However, an interaction between the treatment and behavior did effectively decrease depression.
Because this study was conducted in the past, the investigators could not determine which specific behavior was responsible for the interaction, but with the mathematical formula, they were able to tell that this behavior was necessary for the drug to be effective.
In their paper, Dr Snowberg and his colleagues speculate how a situation like this might come about.
“Maybe there is a drug, for instance, that makes people feel better in social situations, and if you’re in the high probability group, then maybe you’d be more willing to go out to parties to see if the drug helps you talk to people,” Dr Snowberg explained.
“Your behavior drives you to go to the party, and once you’re at the party, the drug helps you feel comfortable talking to people. That would be an example of an interaction effect. You couldn’t get that if people just took this drug alone at home.”
Although this specific example is just speculation, Dr Snowberg said the researchers’ actual results reveal that some behavior or set of behaviors interact with paroxetine to effectively treat depression. And, without this behavior, the drug appears to be ineffective.
“Normally, what you get when you run a standard blind trial is some sort of mishmash of the treatment effect and the treatment-behavior interaction effect,” Dr Snowberg said. “But knowing the full interaction effect is important.”
“Our work indicates that clinical trials underestimate the efficacy of a drug where behavior matters. It may be the case that 50% probability isn’t high enough for people to change any of their behaviors, especially if it’s a really uncertain new treatment. Then, it’s just going to look like the drug doesn’t work, and that isn’t the case.”
Because the meta-analysis supported the team’s hypothesis—that the interaction between treatment and behavior can have an effect on health outcomes—the next step is incorporating these new ideas into an active clinical trial.
Dr Snowberg said the best fit would be a drug trial for a condition, such as a mental health disorder or an addiction, that is known to be associated with behavior. At the very least, he hopes these results will lead the medical research community to a conversation about ways to improve the DBRCT.