Phase 3: Training and User Support
Training and support structures and tools were critical to the rollout of the inappropriate imaging alerts. Providers were reminded about clinical best practices and informed during staff meetings about the new CDS rules. In addition, various types of training and support were available to clinicians and staff during the rollout process. Dedicated time for end-user training provided an opportunity to identify and cultivate super-users. These super-users not only helped provide technical support to their colleagues, but also helped create excitement for the initiative. A centralized support desk provided telephone support for providers in facilities throughout the Banner Health system. Site managers were provided toolkits to support providers and staff throughout the implementation process. The toolkits included frequently asked questions and answers, and were maintained as ‘living documents’ that were updated based on emerging issues and questions.
To keep things on track, project managers from the central project management department were involved in the initiative to provide direct project management services to the initiative. They also worked to instill project management competencies throughout the organization, applying a train-the-trainer approach to disseminate best practices for enhancing communication among team members, implementing workflow changes, and monitoring the results.
Phase 4: Optimization
The optimization phase is continuous and continues to the present day. Notably, the success of the CDS rules depends on the availability of current clinical information for each patient, in addition to information about the treatment plan. For this initiative, Banner Health maintained aggregated clinical patient data in the data warehouse that aggregated data from disparate sources, including billing and EHR data from different care settings such as ambulatory offices, inpatient units, the emergency department, home care, and ambulatory surgery centers. The data warehouse is housed in a strategically chosen physical location to minimize the threat of natural disasters, and cloud-based backup is also used. A master patient index and provider credentialing system are integrated with the data warehouse. Query-based health information exchange is used, when possible, to collect information on care received by patients outside of the Banner Health system.
It is important to note that many CDS alerts are over-ridden without changes to clinical care [34]. Previous research indicates that alert fatigue from “false positives” can impede the effectiveness of alerts [35]. Banner Health monitors the rate at which CDS alerts are over-ridden. Figure 1 shows the percentage of all alerts for radiation exposure—including the alert related to using ultrasound as a diagnostic tool for pediatric appendicitis—that led to order cancellations. The percentage of CT orders that generated the alert and were cancelled fell from 18.9% in March 2011 to 13.6% in February 2012. The rate of order cancellations might have declined over time due to a change in provider behavior from the alert. That is, if inappropriate CT scan orders declined over time, then providers would cancel a decreasing percentage of the CT scan orders that prompted an alert.
Imaging Use
In Figure 2, data on the use of the 2 imaging procedures for the diagnosis of pediatric appendicitis is presented. During the fourth quarter of 2010, almost half of pediatric patients with suspected appendicitis received a CT scan and only about 1 in 7 received an ultrasound. After the clinical decision support alert was put in place to remind providers to perform an ultrasound as a first-line diagnostic tool, the use of ultrasounds increased sharply. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases.
Discussion
This case study discusses the application of CDS functionality in a health care improvement initiative to address inappropriate imaging in a large health system. Four main implementation lessons emerge for the field. First, it is important to involve all levels of staff in the planning process to ensure that health care improvement activities are prioritized correctly and to build buy-in for the priorities addressed with health care improvement activities. Second, it is necessary to allow time to design the alert or reminder, as well as testing it during the implementation process to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow and ensuring usability of the alert or reminder are important, and using the skills of trained engineers helps in this process. Ensuring the availability of trained ultrasound staff was particularly important to this initiative. Finally, the effectiveness of CDS depends on having complete data for each patient, as well as up-to-date information on the relevant evidence-based practice guidelines.