Triangulation
We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.33 The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.
Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.
Nested Sampling Design
The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.
A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.
IRB Approval
This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.
Data Collection
Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.
All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.
Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.
On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.34 Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.
Participant/Practice Observation
On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.