Dr. Lagasse is a professor of anesthesiology and director of quality management at the Yale School of Medicine, New Haven, Conn. He is on the steering committee for the Surgical Care Improvement Project. He reported having no financial disclosures.
COUNTERPOINT: Studies have not shown effectiveness.
When it was created, SCIP did not reflect reality. SCIP started at the U.S. Department of Veterans Affairs, which conducted a 10-year study. They found a 25% relative risk reduction, but that was only a 0.8% absolute risk reduction for the incidence of complications, a drop from about 3.1% to about 2.3% (Arch. Surg. 2002;137:20-27).
It would be great to have randomized controlled trials on the effectiveness of SCIP, but it’s not happening. We’re going to have to go by high-fidelity observational trials, which according to a number of researchers in the field have the same impact as randomized controlled trials.
One study of 35,543 patients in 44 hospitals found a whopping 27% reduction in surgical site infections, but that was only a 0.6% absolute reduction, from about 2.5% to about 1.9% (Am. J. Surg. 2005;190:9-15). There was no significant difference between groups.
Another study showed improved compliance with SCIP measures, but no change in surgical site infection rate (Dis. Colon Rectum 2010;53:24-30). This is the theme in study after study after study.
Dr. Paul Barash
A 2008 study enrolled 9,195 patients undergoing colorectal, orthopedic, or vascular surgery and looked at SCIP compliance vs. surgical site infection. The SCIP rate correlated with the hospital case mix. If you look at the SCIP rate in terms of antibiotic timing, SCIP is not significant. The study basically showed that variables other than timely antibiotic administration are affecting surgical site infection rates (J. Am. Coll. Surg. 2008;206:814-19).
Hospital performance on process measures may not be a good marker of surgical site infection or the outcome we’re looking at, according to another study, which reported that unmeasured effects may have a larger impact than the measured effects (Health Serv. Res. 2008;43:1464-84).
There is a randomized controlled trial that randomized patients to strict control with the SCIP measures or routine treatment at the hospital. The SCIP-treated patients had nearly twice the incidence of surgical site infections as the patients receiving standard treatment. The authors concluded that combining each of the SCIP factors into one big category doesn’t necessarily work (Arch. Surg. 2011;146:263-9).
Should we be evaluating outcome measures with performance measures (e.g., percent timely antibiotic administration) to determine whether they work or not? One editorial evaluated eight articles with data on 31,448 patients, looking just at antibiotic administration within 1 hour of surgery, a SCIP measure. It found a higher infection rate if antibiotics were administered within 30 minutes of incision (JAMA 2010;303:2527-2528).
There was no significant difference in another study between standard of care and SCIP for venous thromboembolism (Am. J. Surg. 2012;204:591-97). The authors wrote that there is no convincing evidence that improvements in compliance are associated with better outcomes. We see this time and time again.
When people find that SCIP is not working, they turn to other measures to reduce surgical site infection. The Comprehensive Unit-based Safety Program (CUSP) is targeted at a specific problem that a specific hospital is having in managing infections. It’s not coming from Washington; it’s based at the hospital. One study showed that following CUSP, there was a significant reduction in surgical site infections despite the fact that previous to that there was 95% compliance with SCIP standards (J. Am. Coll. Surg. 2012;215:193-200). SCIP was working, but it wasn’t affecting outcome.
Dr. Lagasse and I interpret one key study very differently. He abstracts a sentence from a Limitations section of the study and makes a sweeping generalization out of context. But the study showed no relationship between facility adherence to SCIP and the surgical site infection rate. The authors concluded, "Policies regarding continued SCIP measurement and reporting should be reassessed" (Ann. Surg. 2011;254:494-99).
The largest SCIP study to date from a single entity involved 32,459 patients in the Veterans Affairs medical system. Overall, antibiotics were administered within 28 minutes of surgical incision. Once they adjusted for confounders, they found no significant relationship between surgical site infection and the SCIP measures (JAMA 2013;148:649-57). No one has proven that giving antibiotics within 60 minutes of surgical incision gives you a lower infection rate.
SCIP measures divert resources and divert clinical care. They obscure the nuances of care. They may harm the hospital and the provider, and they raise unnecessary legal risk if an antibiotic is not given within 60 minutes of incision.