Implementing Patient-Reported Outcome Measures in Your Practice: Pearls and Pitfalls

Take-Home Points

• Systematic use of PROMs allows physicians to review data on pain, physical function, and psychological status to aid in clinical decision-making and best practices.
• PROMs should include both general outcome measures (VAS, SF-36, or EQ-5D) and reliable, valid, and responsive disease specific measures.
• PROM questionnaires should collect pertinent information while limiting the length to maximize patient compliance and reliability.
• PROMIS has been developed to standardize questionnaires, but generality for specific orthopedic procedures may result in less effective measures.
• PROMs can also be used for predictive modeling, which has the potential to help develop more cost-effective care and predict expected outcomes and recovery trajectories for individual patients.

Owing to their unique ability to recognize patients as stakeholders in their own healthcare, patient-reported outcome measures (PROMs) are becoming increasingly popular in the assessment of medical and surgical outcomes. ¹ PROMs are an outcome measures subset in which patients complete questionnaires about their perceptions of their overall health status and specific health limitations. By systematically using PROMs before and after a clearly defined episode of care, clinicians can collect data on perceived pain level, physical function, and psychological status and use the data to validate use of surgical procedures and shape clinical decisions about best practices. ^2-4 Although mortality and morbidity rates and other traditional measures are valuable in assessing outcomes, they do not represent or communicate the larger impact of an episode of care. As many orthopedic procedures are elective, and some are low-risk, the evaluation of changes in quality of life and self-reported functional improvement is an important addition to morbidity and mortality rates in capturing the true impact of a surgical procedure and recovery. The patient’s preoperative and postoperative perspectives on his or her health status have become important as well; our healthcare system has been placing more emphasis on patient-centered quality care. ^2,5

Although PROMs have many benefits, implementation in an orthopedic surgery practice has its challenges. With so many PROMs available, selecting those that fit the patient population for a specialized orthopedic surgery practice can be difficult. In addition, although PROM data are essential for research and for measuring individual or institutional recovery trajectories for surgical procedures, in a busy practice getting patients to provide these data can be difficult.

PROMs are heavily used for outcomes assessment in the orthopedics literature, but there are few resources for orthopedic surgeons who want to implement PROMs in their practices. In this article, we review the literature on the challenges of effectively implementing PROMs in an orthopedic surgery practice.

PROM Selection Considerations

PROMs can be categorized as either generic or disease-specific, ⁴ but together they are used to adequately capture the impact, both broad and local, of an orthopedic condition.

Generic Outcome Measures

Generic outcome measures apply to a range of subspecialties or anatomical regions, allowing for evaluation of a patient’s overall health or quality of life. The most widely accepted measure of pain is the visual analog scale (VAS). The VAS for pain quantifies the level of pain a patient experiences at a given time on a graphic sliding scale from 0 (no pain) to 10 (worst possible pain). The VAS is used in clinical evaluation of pain and in reported outcomes literature. ^6,7

Many generic PROMs assess mental health status in addition to physical limitations. Poor preoperative mental health status has been recognized as a predictor of worse outcomes across a variety of orthopedic procedures. ^8,9 Therefore, to assess the overall influence of an orthopedic condition, it is important to include at least 1 generic PROM that assesses mental health status before and after an episode of care. Generic PROMs commonly used in orthopedic surgery include the 36-Item Short Form Health Survey (SF-36), the shorter SF-12, the Veterans RAND 12-Item Health Survey (VR-12), the World Health Organization Disability Assessment Schedule (WHODAS), the European Quality of Life-5 Dimensions (EQ-5D) index, and the 10-item Patient-Reported Outcomes Measurement Information System Global Health (PROMIS-10) scale. ^10-14

Some generic outcome measures (eg, the EQ-5D index) offer the “utility” calculation, which represents a preference for a patient’s desired health status. Such utilities allow for a measurement of quality of life, represented by quality-adjusted life years (QALY), which is a standardized measure of disease burden. Calculated QALY from measures such as the EQ-5D can be used in cost-effectiveness analyses of surgical interventions and have been used to validate use of procedures, particularly in arthroplasty. ^15-17

Disease-Specific Outcome Measures

Likewise, there is a range of disease-specific PROMs validated for use in orthopedic surgery, and providers select PROMs that fit their scope of practice. In anatomical regions such as the knee, hip, and shoulder, disease-specific outcome measures vary significantly by subspecialty and patient population. When selecting disease-specific PROMs, providers must consider tools such as reliability, validity, responsiveness, and available population norms. One study used Evaluating Measures of Patient-Reported Outcomes (EMPRO) to assess the quality of a PROM in shoulders and concluded that the American Shoulder and Elbow Surgeons (ASES) index, the Simple Shoulder Test (SST), and the Oxford Shoulder Score (OSS) were all supported for use in practice. ¹⁸ It is important to note that reliability, validity, and responsiveness of a PROM may vary with the diagnosis or the patient population studied. For example, the SST was found to be responsive in assessing rotator cuff injury but not as useful in assessing shoulder instability or arthritis. ¹⁹ Variable responsiveness highlights the need for a diagnosis-based level of PROM customization. For example, patients who undergo a surgical intervention for shoulder instability are given a customized survey, which includes PROMs specific to their condition, such as the Western Ontario Shoulder Instability (WOSI) index. ²⁰ For patients with knee instability, similar considerations apply; specific measures such as the Lysholm score and the Tenger Activity Scale capture the impact of injury in physically demanding activities. ²¹ When selecting disease-specific PROMs, providers should consult articles like those by Davidson and Keating ²² and Bent and colleagues, ²³ who present provider-friendly tools that can be used to examine the effectiveness of a PROM, and provide additional background information on selecting disease-specific PROMs. For hip and knee arthroplasty subspecialties, the International Society of Arthroplasty Registries (ISAR) created a working group that determines best practices for PROM collection and identifies PROMs most commonly reported in arthroplasty. ²⁴

Questionnaire Length Considerations

When PROMs are used in a practice, a balance must be struck between gathering enough information to determine functionality and limiting the patient burden of questionnaire length. A decision to use several PROMs all at once, at a single data collection point, can lengthen the questionnaire significantly. One study found that, with use of longer questionnaires, patients may lose interest, resulting in decreased reliability and compliance. ²⁵ For example, providers who use the long (42-item) Knee Injury and Osteoarthritis Outcome Score (KOOS) questionnaire to assess knee function are often limited in what other PROMs they may administer at the same time. Efforts to shorten this questionnaire while still capturing necessary information led to the development of the 7-item KOOS Jr, which was validated for use in knee arthroplasty and had its 7 items drawn from the original 42. ²⁶ Similarly, the 40-item Hip Disability and Osteoarthritis Outcome Score (HOOS) questionnaire was shortened to the 6-item HOOS Jr, which was validated for use in hip arthroplasty, ²⁷ and the generic SF-36 was shortened to the SF-12. ¹¹ Providers trying to build an outcomes database while minimizing patient burden should consider using the shorter versions of these questionnaires but should also consider their validity, as KOOS Jr and HOOS Jr have been validated for use only in knee and hip arthroplasty and not in other knee and hip conditions.

PROM Data Collection Considerations

Comprehensive collection of longitudinal PROM data poses many challenges for providers and patients. For providers, the greatest challenges are infrastructure, technology, and the personnel needed to administer and store paper or electronic surveys. For patients, the most common survey completion barriers are questionnaire length, confusing or irrelevant content, and, in the case of some older adults, inability to complete surveys electronically. ²⁵

Identifying a nonresponsive or noncompliant patient population is an important issue in collecting PROM data for research or other purposes. A study of factors associated with higher nonresponse rates in elective surgery patients (N = 135,474) found that noncompliance was higher for males, patients under age 55 years, nonwhites, patients in the lowest socioeconomic quintile, patients living alone, patients needing assistance in completing questionnaires, and patients who previously underwent surgery for their condition. ²⁸ In a systematic review of methods that increased the response rates of postal and electronic surveys, Edwards and colleagues ²⁹ found significantly higher odds of response for patients who were prenotified of the survey, given shorter questionnaires, or given a deadline for survey completion. Of note, response rates were lower when the word survey was used in the subject line of an email.

PROM distribution has evolved with the rise of technological advances that allow for electronic survey distribution and data capture. Several studies have found that electronically administered PROMs have high response rates. ^3,30,31 In a study of patients who underwent total hip arthroplasty, Rolfson and colleagues ³² found that response rates were significantly higher for those who were surveyed on paper than for those surveyed over the internet. A randomized controlled study found that, compared with paper surveys, digital tablet surveys effectively and reliably collected PROM data; in addition, digital tablets provided instant data storage, and improved survey completion by requiring that all questions be answered before the survey could be submitted.33 However, age, race/ethnicity, and income disparities in technology use must be considered when administering internet-based follow-up surveys and analyzing data collected with web-based methods. ³⁴ A study of total joint arthroplasty candidates found that several groups were less likely to complete electronic PROM questionnaires: patients over age 75 years, Hispanic or black patients, patients with Medicare or Medicaid, patients who previously underwent orthopedic surgery, patients undergoing revision total joint arthroplasty, patients with other comorbidities, and patients whose primary language was not English. ³⁵ Providers interested in implementing PROMs must consider their patient population when selecting a method for survey distribution and follow-up. A study found that a majority of PROMs were written at a level many patients may not have understood, because of their literacy level or age; this lack of understanding created a barrier to compliance in many patient populations. ³⁶

PROM Limitations and PROMIS Use

Use of PROMs has its limitations. The large variety of PROMs available for use in orthopedic surgery has led to several standardization initiatives. The National Institutes of Health funded the development of PROMIS, a person-centered measures database that evaluates and monitors the physical, social, and emotional health of adults and children. ³⁷ The goal of PROMIS is to develop a standardized method of selecting PROMs, so that all medical disciplines and subspecialties can choose an applicable set of questions from the PROMIS question bank and use it in practice. Orthopedic surgery can use questions pertaining to physical functioning of the lower and upper extremities as well as quality of life and mental health. PROMIS physical function questions have been validated for use in several areas of orthopedic surgery. ^38-40 A disadvantage of PROMIS is the overgenerality of its questions, which may not be as effective in capturing the implications of specific diagnoses. For example, it is difficult to use generalized questions to determine the implications of a diagnosis such as shoulder instability, which may affect only higher functioning activities or sports. More research on best PROM selection practices is needed in order to either standardize PROMs or move toward use of a single database such as PROMIS.

Future Directions in PROM Applications

PROMs are being used for research and patient engagement, but there are many other applications on the horizon. As already mentioned, predictive modeling is of particular interest. The existence of vast collaborative PROM databases that capture a diverse patient population introduces the possibility of creating models capable of predicting a patient outcome and enhancing shared decision-making. ³ Predicting good or excellent patient outcomes for specific patient populations may allow elimination of certain postoperative visits, thereby creating more cost-effective care and reducing the burden of unnecessary clinic visits for both patients and physicians.

As with other healthcare areas, PROM data collection technology is rapidly advancing. Not only has electronic technology almost entirely replaced paper-and-pencil collection methods, but a new method of outcome data collection has been developed: computerized adaptive testing (CAT). CAT uses item-response theory to minimize the number of questions patients must answer in order for validated and reliable outcome scores to be calculated. According to multiple studies, CAT used across several questionnaires has reliably assessed PROMs while minimizing floor and ceiling effects, eliminating irrelevant questions, and shortening survey completion time. ^41-43

Besides becoming more patient-friendly and accessible across multiple interfaces (mobile devices and computers), PROMs are also beginning to be integrated into the electronic medical record, allowing easier access to information during chart reviews. Use of statistical and predictive modeling, as described by Chang, ³ could give PROMs a role in clinical decision-making. Informing patients of their expected outcome and recovery trajectory—based on demographics, comorbidities, preoperative functional status, and other factors—could influence their decision to undergo surgical intervention. As Halawi and colleagues ⁴⁴ pointed out, it is important to discuss patient expectations before surgery, as unrealistic ones can negatively affect outcomes and lead to dissatisfaction. With clinicians having ready access to statistics and models in patient charts, we may see a transformation in clinical practices and surgical decision-making.

Conclusion

PROMs offer many ways to improve research and clinical care in orthopedic surgery. However, implementing PROMs in practice is not without challenges. Interested orthopedic surgeons should select the PROMs that are most appropriate—reliable, validated, and responsive to their patient population. Electronic distribution of PROM questionnaires is effective and allows data to be stored on entry, but orthopedic surgeons must consider their patient population to ensure accurate data capture and compliance in longitudinal surveys. Proper implementation of PROMs in a practice can allow clinicians to formulate expectations for postoperative recovery and set reasonable postoperative goals while engaging patients in improving quality of care.