Clinicians interpreting this and other retrospective studies using large cancer registry data need to consider:
1) Whether treatment groups are comparable. For most retrospective studies, treatment groups are not comparable. Surgery versus RT is especially difficult to compare. In many studies in various cancers that have compared surgery alone versus surgery plus adjuvant therapy, the latter patients are inherently healthier. The ability or inability of investigators to address these confounding issues is central to assessing the quality of the evidence in a study.
(2) Limitations related to the data elements that are contained in the registry and data accuracy.
(3) Whether the results are believable clinically. If survival is the endpoint in a study, when do the survival curves separate – and is that consistent with the known clinical course of a particular disease? The main advantage of randomized trials is the ability to create comparable patient groups and minimize confounding, which is also the biggest limitation of retrospective studies. In clinical scenarios where multiple randomized trials have consistently confirmed a result (e.g., adding androgen deprivation therapy to RT improves survival for high-risk patients), repeated retrospective analyses asking these same questions that may show either the same or an opposite result are less useful.
Perhaps the biggest challenge in conducting studies using cancer registry data and interpreting their results as researchers and clinicians relates to the personal biases most of us harbor; these biases tempt us to analyze registry data in an attempt to refute clinical trials and to selectively believe studies that provide results supporting our own biases. For each clinician interpreting retrospective results along with or in absence of clinical trial data, recognizing our own biases by following the framework outlined previously to assess the quality and believability of each study can potentially remove the greatest confounder of all.
Ronald C. Chen, MD, MPH, is associate professor of radiation oncology at the University of North Carolina at Chapel Hill. He serves in a consulting or advisory role for Medivation/Astellas, Accuray, and Bayer, and has received research funding from Accuray. His remarks are adapted from an editorial (J Clin Oncol. 2018 Feb 28. doi: 0.1200/JCO.2017.77.5833).
