Conducting The Direct Observation of Primary Care Study Insights from the Process of Conducting Multimethod Transdisciplinary Research in Community Practice
The DOPC Writing Group Submitted, revised, February 16, 2001. From the Departments of Family Medicine, Epidemiology and Biostatistics, and Sociology, Case Western Reserve University, Ireland Comprehensive Center, University Hospitals of Cleveland and Case Western University Center for Research in Family Practice and Primary Care. Additional information for the DOPC Writing Group is provided in the acknowledgments. Reprint requests should be addressed to Kurt C. Stange, MD, PhD, Department of Family Medicine, Case Western Reserve University, 10900 Euclid Ave, Cleveland, OH 44106.
References
Planning and Conduct of Fieldwork
In 1994, more than 3 years after the idea was conceived, the board of directors of the 138-member research network (RAPP) was formally activated. The board’s 14 network volunteer members, several of whom helped develop the study and the network, were active in planning the practical implementation and refinement of the study protocol. When a board member suggested that they review the details of the direct observation measures before the study, the board concluded that as study participants they should not be involved in planning study measures, to avoid biasing their behavior during direct observation.
With funding, 2 logistic aspects of the study gained importance. First, it had been 2 years since the research network was formed. Whether physicians had retained their commitment to participate was unknown. That concern was laid to rest, however, when the vast majority of physicians expressed continued interest in the study. In retrospect, the 2-year struggle to obtain funding helped bond the network and create a sense of ownership and allegiance to the project.
The second major logistic issue was the need to recruit 8 research nurses. Job requirements included excellent interpersonal skills, sensitivity to the demands of real world community family practice, attention to detail in collecting reliable and valid quantitative data from multiple measures, an open-minded observational ability to simultaneously collect qualitative data, willingness to drive to multiple and sometimes distant sites, and interest in a 1-year job at a university salary. Hiring 8 nurses who could meet these requirements and start the 2-month training process on the same date seemed unrealistic at best. Yet, because of word of mouth advertising, the excitement generated by the study, the recent termination of another research project at the university, and the excellent reputation of the department, 8 highly qualified research nurses were found.
During their 8-week training, the research nurses were enlisted as true partners. They helped refine the research protocol and instruments, and items were added to the measures to reflect their interests. Using videotaped encounters, Dr Callahan instructed the nurses in applying the DOC, and they took the lead in adapting it for the study. As the immensity of the quantitative data collection requirements grew, Drs Miller and Crabtree scaled back the qualitative data collection protocol. They trained the research nurses in observational techniques and in how to dictate ethnographic field notes to record unanticipated findings, provide rich descriptions of quantitatively measured variables, and critique the study methods’ accuracy in capturing the phenomena under study.50
Details of the data collection procedure have been reported elsewhere.1,2 Briefly, teams of 2 research nurses spent 1 day observing patient care by the 138 participating RAPP members. One nurse obtained verbal informed consent from patients in the waiting room and distributed patient exit questionnaires. The other nurse accompanied the physician, directly observing consecutive visits by consenting patients and recording observations using the DOC and a direct observation checklist. Typically, the nurses exchanged roles after lunch. They returned on a subsequent day to perform medical record reviews for each other’s observed visits and to collect billing data. On the basis of observation and brief interviews with key informants, the research nurses completed a practice environment checklist. They dictated ethnographic field notes immediately after leaving the practice.10
During the course of the fieldwork, research team meetings were held every other week to coordinate logistics and assess and recalibrate inter-rater reliability using videotaped visits and medical records that were not part of the larger study. The high degree of inter-rater reliability achieved with this approach has been reported previously.1
After data were collected from each physician, the board of directors met to review study progress and reassess the study protocol. The academic research team, including all consultants, also met to refine the protocol and plan the second round of physician visits. Initial plans called for ongoing analysis of the ethnographic field notes, but this proved to be infeasible because of their large volume and the study demands. However, at the study midpoint, Drs Crabtree and Miller independently analyzed the field notes using an immersion crystallization technique.51 Based on the richness of the information, they developed a template52 for gathering field notes during the second round of physician visits.
Data collection procedures were repeated, and each physician was visited a second time. The 4 months (on average) between visits helped assure that seasonal variations in health problems did not unduly affect the characterization of patient care. After the second data collection visit, physicians completed a detailed questionnaire.
