Experience, Expertise, or Specialty? Uses and Misuses of a Reference

,

Original Research

Experience, Expertise, or Specialty? Uses and Misuses of a Reference

J Fam Pract. 2002 February;51(2):1

By

Darius A Rastegar, MD

PDF Download

References

1. McLellan MF, Case D, Barnett MC. Trust, but verify. The accuracy of in four anesthesia journals. Anesthesiology 1992;77:185-8.

2. Orlin W, Pehling J, Pogrell MA. Do authors check their ? A survey of 500 from the Journal of Oral and Maxillofacial Surgery. J Oral Maxillofac Surg 1996;54:200-2.

3. DeLacey G, Record C, Wade J. How accurate are quotations and in medical journals? BMJ 1985;291:884-6.

4. Evans JT, Nadjari HI, Burchell SA. Quotational and reference accuracy in surgical journals. JAMA 1990;263:1353-4.

5. Kitahata MM, Koepsell TD, Deyo RA, Maxwell CL, Dodge WT, Wagner EH. Physicians’ experience with the acquired immunodeficiency syndrome as a factor in patient’s survival. N Engl J Med 1996;334:701-6.

6. Crystal S, Sambamoorthi U. Functional impairment trajectories among persons with HIV disease: a hierarchical linear models approach. Health Serv Res 1996;31:469-88.

7. Lange JMA. Current problems and the future of antiretroviral drug trials. Science 1997;276:548-50.

8. Pollack S. A role for academic medical centers in the era of managed care: immediate, interactive, free information. Acad Med 1998;73:357-9.

9. Saver BG, Doescher MP. The rising tide of hospitalism: evidence-based or anecdote-based medicine? J Fam Pract 1998;46:465-8.

10. Donohoe MT. Comparing generalist and specialty care-discrepancies, deficiencies, and excesses. Arch Intern Med 1998;158:1596-608.

11. Schroeder SA, Schapiro R. The hospitalist: new boon for internal medicine or retreat from primary care? Ann Intern Med 1999;130:382-7.

12. Goldman L. The impact of hospitalists on medical education and the academic health system. Ann Intern Med 1999;130:364-7.

13. Wachter RM. An introduction to the hospitalist model. Ann Intern Med 1999;130:338-42.

14. Go AS, Rao RK, Dauterman KW, Massie BM. A systematic review of the effects of physician specialty on the treatment of coronary disease and heart failure in the United States. Am J Med 2000;108:216-26.

15. Golomb BA, Pyne JM, Wright B, Lohr JB, Bozette SA. The role of psychiatrists in primary care of patients with severe mental illness. Psychiatr Serv 2000;51:766-73.

16. Kitahata MM, Holmes EK, Wagner EH, Gooding TD. Caring for persons with HIV infection in a managed care environment. Am J Med 1998;104:511-5.

17. Grace CJ, Richardson K, Kutzko D, Alston WK, Ramundo M. Service delivery for patients with HIV in a rural state: the Vermont model. AIDS Patient Care Stds 1999;13:659-66.

18. Turner BJ, Newschaffer CJ, Cocroft J, Fanning TR, Marcus S, Hauck WW. Improved birth outcomes among HIV-infected women with enhanced medicaid prenatal care. Am J Public Health 2000;90:85-91.

19. Gallant JE. Strategies for long-term success in the treatment of HIV infection. JAMA 2000;283:1329-34.

ABSTRACT

OBJECTIVES: We wanted to analyze systematically the manner in which the results of a published study are presented in subsequent publications that refer to it.

STUDY DESIGN: We identified a convenience sample of 121 scientific papers that referred to an often-cited 1996 study by Kitahata and colleagues. This study reported that greater primary care physician experience with AIDS was associated with lower mortality among their patients with AIDS.

OUTCOMES MEASURED: We determined the manner in which the results of the Kitahata and coworkers study were presented, the type of article, and whether its focus was on HIV care.

RESULTS: Most of the articles reviewed (78%) appropriately referred to the study as evidence of improved outcomes with increasing provider experience. However, 8% of the articles reviewed referred to the study as evidence of improved outcomes with specialty care and 3% referred to it as evidence of the benefits of expert care. Articles that referred to the study as evidence of improved outcomes with specialty care were more likely to be review articles and articles with a non-HIV focus.

CONCLUSIONS: This study demonstrates that misrepresentation of the findings of published studies is not uncommon. More needs to be done to ensure the accuracy of references in scientific publications.

References provide the foundation for scientific publications, particularly for review articles and editorials. Readers must rely on the honesty and integrity of the authors or go through the painstaking work of finding and verifying the references themselves. Previous research indicates that errors in references are common but usually minor and do not necessarily detract from the authors’ argument or conclusions.^1,2 More serious errors have been found, however, in the form of misleading or erroneous quotations.^3,4

In 1996, Kitahata and colleagues published a study “to determine whether more experience with the management of AIDS on the part of primary care physicians is associated with increased survival among their patients with AIDS.”⁵ They found that patients with AIDS cared for by physicians who had the least experience with AIDS survived for significantly less time than did those cared for by physicians with the most experience. The least-experienced physicians were defined as those who had only 1 patient with AIDS and a low level of residency experience with AIDS. The most-experienced physicians had either 6 or more patients with AIDS or 2 to 5 such patients and a high level of residency experience. This study was not a comparison of generalists with specialists; most of the physicians (85%) were general or family practitioners and the remainder were internists.

Since then, the study has been frequently cited in a variety of contexts, from discussions of HIV and AIDS care to more general discussions on the relationship between physician experience and patient outcomes. We undertook the current study after observing that some articles referred to the Kitahata study as evidence of the benefits of specialty care. The goal of this study was to review systematically the articles that refer to this publication and to analyze the conclusions that authors draw from the paper.

Methods

We identified articles that referred to the Kitahata et al study through the Scientific Citation Index (maintained by the Institute for Scientific Information). We reviewed a convenience sample of articles-those in journals available in 1 of 2 major local health sciences libraries (including online links). Letters to the editor were excluded. Each article was reviewed by the authors and the following information collected: (1) type of article (original research, review, editorial, or other); (2) whether the focus of the article was on HIV care or another topic; and (3) the passage in which the Kitahata et al article was first mentioned. Each passage was independently assessed by the authors and classified by the assertion made; namely, whether patient outcomes are related to experience, expertise, specialty, or none of the above. If there was initial disagreement on the classification of the passage, the final decision was made by consensus. The relationship between the type and focus of the article and the assertion made was investigated using Fisher’s exact test.

Results

As of July 31, 2000, 142 articles were listed on the Scientific Citation Index that had referred to the paper by Kitahata and coworkers. Twelve (8%) were in journals not accessible through either of 2 major local health sciences libraries. Nine letters were excluded from the analysis. A total of 121 articles were reviewed (85% of total); the results are summarized in Table 1. Ninety-four of the articles reviewed (78%) were focused on HIV-related topics. Sixty-three (52%) of the articles were original research papers; 35 (29%) were review articles; 15 (12%) were editorials; and 8 (7%) were other types (4 program descriptions, 1 program proposal, 1 conference report, and 2 commentaries).