Staff included registered nurses, licensed practical nurses, medical assistants, and patient service representatives. Because nurses and medical assistants are responsible for medication refills, they received education specifically about this intervention. The remaining staff also received instruction, as they have personal interactions with patients at the provider visit, and thus their attitudes were important to measure. Participants completed surveys at the time of the educational sessions and again 9 months following implementation of the intervention. This was a one-year intervention, with 3 initial months of teaching; the study period therefore lasted 9 months. Since surveys were anonymous, we could not link results to specific individuals. However, we provided post-intervention surveys only to those who reported completing the initial survey.
Survey design and administration
The provider survey contained an attitude component and a knowledge component (TABLE 2). The attitude component consisted of 6 items taken from the Drug Problems Perceptions Questionnaire,13 to address role adequacy, support, and self-esteem, as well as job satisfaction (the words “drug users” were replaced with “patients on [chronic] opioids”). We created an additional 3 items to further explore these domains (items 1-3). Three additional items addressed provider access to EMR specific tools (items 10-12).
The knowledge survey consisted of multiple choice questions created by the study team, and it reflected best practice guidelines for opioid management for CNCP and knowledge of protocol elements. Items included the definition of chronic pain, opioid medications not included on the UDS, interpretation of UDS results, addiction risk, intervals for office visits for patients on chronic opioid therapy, and pain medication dose escalation.
The staff survey included similar attitude components and a modified knowledge portion regarding which patients should have a CMA, where to document a CMA in the EMR, addiction risk, intervals for office visits, and how to handle early prescription refill requests.
Evaluation and statistical analysis
To assess the impact of the intervention, we chose 2 measures of physician adherence with the protocol (UDS and chronic pain diagnosis) because of our ability to access these measures within our approved protocol.
Individual attitude survey questions were compared using paired t-tests. We averaged knowledge test scores, and also used the paired t-test to compare pre- and posttest averages. We used Stata 11.2 (StataCorp LP, 2009) to analyze survey data.
This study was sponsored by the Matthew Slap Research Award and approved by the University of Pennsylvania Institutional Review Board.
RESULTS
Practice demographics
The 3 practices are located within the same zip code, a few city blocks from one another. Despite geographic proximity of the practices, their populations differ racially and ethnically as well as in neighborhood income distributions (TABLE 3). In all 3 practices, the total number of patients prescribed >2 opioid medications declined during the year-long study period. Practice 3 had the sharpest decline in the number of patients prescribed chronic opioids, likely due to provider turnover during the study period. Practices 1 and 2 had the highest adherence to guidelines. The marked variability in adoption of guidelines likely reflects a number of factors: the difference in baseline opioid prescribing (highest in Practice 3), the presence of physician champions in Practices 1 and 2, and more intensive training of the primary care residents in Practice 1.
Protocol adherence
We measured provider adherence to the protocol by comparing data from the year before the intervention to the year following the start of the intervention for the number of UDSs ordered, the number of chronic pain diagnoses on patients’ EMR problem lists, and the number of office visits with CNCP patients. UDSs ordered increased by 145% across all 3 practices, with the largest improvement seen in Practice 1 (430%; P<.05). Documentation of a chronic pain diagnosis in the EMR problem list increased by 424% across practices, with the largest improvement seen again in Practice 1 (918%, P<.05) (TABLE 4). Based on this performance, 24 of 27 (89%) full time physicians qualified for the financial incentive. We chose not to include the third measure (number of office visits) for analysis, as we discovered that >90% of patients were seen at least every 6 months before the intervention.
Survey results
Before the protocol training, we surveyed 26 providers and 33 staff members. Nine months after the initiation of the protocol, 25 providers and 26 staff were again surveyed. Surveys were anonymous so we were unable to link knowledge gains to individuals.