MADRID – The efficacy results from new rheumatoid arthritis drugs tested in many phase 2 trials run over the past couple of decades have routinely overestimated the efficacy of many of the drugs tested when compared with how the same agents performed in subsequent phase 3 testing, according to an analysis of published results from 44 pairs of phase 2 and 3 trials.
MItchel L. Zoler/MDedge News
Dr. Andreas Kerschbaumer
Based on the percentage of rheumatoid arthritis (RA) patients who showed an American College of Rheumatology 20% improvement (ACR20) in their joint symptoms, the 44 phase 2 trials overestimated efficacy by an average of 39% when compared with the ACR20 responses seen in paired phase 3 trials, Andreas Kerschbaumer, MD, said at the European Congress of Rheumatology. The ACR50 results overstated efficacy by an average of 34% when compared with phase 3 results for the same drugs, and the ACR70 endpoint showed drug efficacy that averaged 39% better during phase 2 studies than it did in the phase 3 trials. All three between-group differences were statistically significant, said Dr. Kerschbaumer, a rheumatologist at the Medical University of Vienna.
“Active treatment arms of phase 2 studies systematically overestimated efficacy when compared with subsequent phase 3 studies,” he said.
“Many researchers had already seen this, and realized that phase 2 trials often overestimated [efficacy], but no one has ever shown this systematically,” Dr. Kerschbaumer said in an interview. The same problem also appears to affect trials of oncology drugs, he noted, but a unique feature of RA studies allowed him and his colleagues to examine the ubiquity and persistence of this phase 2 bias in rheumatology trials over time: ongoing reliance during more than 2 decades of experience on the ACR20 response as the primary endpoint of RA drug trials. Even with this advantage, which allowed inclusion of 44 pairs of studies, “it was very surprising to find a statistically significant effect in the meta-analysis,” he said.
He and his associates also ran a further analysis that looked for measured parameters that showed significant correlation with mismatch of the phase 2 and 3 results, and this identified two apparently causal factors: having a low number of swollen and tender joints as an inclusion criterion for patients and using a 28-joint count rather than a 66-joint count for assessing disease activity during the study. The analysis lacked enough information to provide clear evidence on why these two aspects of patient assessment could lead to misleading phase 2 results, but Dr. Kerschbaumer believed the findings were clear enough to influence future trial design.
Going forward, trialists “should be very careful of how you include patients [in studies]. This is what our study shows. And they should not use 28 joints but 66. The higher the number of swollen and tender joints detected for study inclusion, the less the possibility to overestimate [efficacy]. It’s easier to achieve ACR20 when you look at fewer joints.”
Until now, companies that sponsor drug trials had an incentive to use a lower minimum number of swollen and tender joints for enrolled patients because it made enrollment easier, he noted. The downside, in addition to overstating efficacy at the phase 2 stage, is subjecting patients to treatments in phase 3 trials with a reduced likelihood for success.
Dr. Kerschbaumer offered two examples of drugs that showed promising RA efficacy based on ACR20 responses in phase 2 trial results that were followed by neutral phase 3 trial outcomes. One episode involved tabalumab (Ann Rheum Dis. 2015 Aug;74[8]:1567-70), and a second was a trial of fostamatinib (Arthritis Rheumatol. 2014 Dec;66[12]:3255-64).
The systematic review he led identified 44 study pairs run since the late 1990s that met all the study criteria, which covered 19 different drugs tested in more than 17,000 RA patients.
What the results showed was “just an association, so we must be careful not to overstate the results, but we saw something that may explain what people have seen [anecdotally] over the past 20 years,” he said. “Some people get very excited by phase 2 results, but we need to be careful about interpreting these outcomes.” Validation of the finding would require analysis of patient-level data, something that would be hard to obtain for a large number of phase 2 and 3 trials, Dr. Kerschbaumer noted.
Dr. Kerschbaumer has been a speaker on behalf of Bristol-Myers Squibb, Celgene, Merck Sharp & Dohme, and Pfizer.
SOURCE: Kerschbaumer A et al. Ann Rheum Dis. Jun 2019;78(Suppl 2):191-2. Abstract OP0229, doi: 10.1136/annrheumdis-2019-eular.5161.