Hospital leaders report the most common aim of peer review (PR) is to improve quality and patient safety, thus it is a potentially powerful quality improvement (QI) driver.1 “When conducted systematically and credibly, peer review for quality management can result in both short-term and long-term improvements in patient care by revealing areas for improvement in the provision of care,” Veterans Health Administration (VHA) Directive 1190 states. “This ultimately contributes to organizational improvements.” At the same time, there are anecdotal concerns that PR may be used punitively and driven by case outcomes rather than by accepted best practices supporting QI.
Studies of the PR process suggest these concerns are valid. A key tenet of QI is standardization. PR is problematic in that regard; studies show poor interrater reliability for judgments on care, as well as hindsight bias—the fact that raters are strongly influenced by the outcome of care, not the process of care.2-5 There are concerns that case selection or review process when not standardized may be wielded as punitive too.6 In this study, we sought to identify the relationship between PR findings and subsequent institution quality metrics. If PR does lead to an improvement in quality, or if quality concerns are managed within the PR committee, it should be possible to identify a measurable relationship between the PR process and a facility’s subsequent quality measures.
A handful of studies describe the association between PR and quality of care. Itri and colleagues noted that random, not standardized PR in radiology does not achieve reductions in diagnostic error rate.7 However, adoption of just culture principles in PR resulted in a significant improvement in facility leaders’ self-reports of quality measures at surveyed institutions.8 The same author reported that increases in PR standardization and integration with performance improvement activities could explain up to 18% of objective quality measure variation.9
We sought to determine whether a specific aspect of the PR process, the PR committee judgment of quality of care by clinicians, was related to medical center quality in a cross-sectional study of 136 Veterans Health Administration (VHA) medical centers. The VHA is a good source of study because there are standardized PR processes and training for committee members and reviewers. Our hypothesis was that medical centers with a higher number of Level 2 (“most experienced and competent clinicians might have managed the case differently”) and Level 3 (“most experienced and competent providers would have managed the case differently”) PR findings would also have lower quality metric scores for processes and outcomes of care.
Methods
We used PR data from fiscal year 2018 and 2019. VHA PR data are available quarterly and are self-reported by each facility to the VHA Office of Clinical Risk Management. These data are broken down by facility. The following data, when available in both fiscal years 2018 and 2019, were used for this analysis: percent and number of PR that are ranked as level 1, 2, or 3; medical center group (MCG) acuity measure assigned by the VHA (1 is highest, 3 is lowest); and number of PR per 100,000 unique veteran encounters in 2019. Measures of facility quality are drawn from Strategic Analytics for Improvement and Learning (SAIL) data from 2019, which are available quarterly by facility and are rolling for 12 months. SAIL measures processes and outcomes of care. Table 1 indicates which measures are focused on outcomes vs quality processes.
SAS Version 9.2 was used to perform statistical analyses. We used Spearman correlation to estimate the PR and quality relationship.
Results
There were 136 facilities with 2 years of PR data available. The majority of these facilities (89) were highest complexity MCG 1 facilities; 19 were MCG 2, and 28 were MCG 3. Of 13,515 PRs, most of the 9555 PR findings were level 1 (70.7%). The between-facility range of level 2 and 3 findings was large, varying from 3.5% to nearly 70% in 2019 (Table 2). Findings were similar in 2018; facilities level 2 and 3 ratings ranged from 3.6% to 73.5% of all PR findings.
There was no correlation between most quality measures and facility PR findings (Table 3). The only exception was for Global Measures (GM90), an inpatient process of care measure. Unexpectedly, the correlation was positive—facilities with a higher percentage of level 2 and 3 PR findings had better inpatient processes of care SAIL score. The strongest correlation was between 2018 and 2019 PR findings.