Experience, Expertise, or Specialty? Uses and Misuses of a Reference

,

Original Research

Experience, Expertise, or Specialty? Uses and Misuses of a Reference

J Fam Pract. 2002 February;51(2):1

PDF Download

References

1. McLellan MF, Case D, Barnett MC. Trust, but verify. The accuracy of in four anesthesia journals. Anesthesiology 1992;77:185-8.

2. Orlin W, Pehling J, Pogrell MA. Do authors check their ? A survey of 500 from the Journal of Oral and Maxillofacial Surgery. J Oral Maxillofac Surg 1996;54:200-2.

3. DeLacey G, Record C, Wade J. How accurate are quotations and in medical journals? BMJ 1985;291:884-6.

4. Evans JT, Nadjari HI, Burchell SA. Quotational and reference accuracy in surgical journals. JAMA 1990;263:1353-4.

5. Kitahata MM, Koepsell TD, Deyo RA, Maxwell CL, Dodge WT, Wagner EH. Physicians’ experience with the acquired immunodeficiency syndrome as a factor in patient’s survival. N Engl J Med 1996;334:701-6.

6. Crystal S, Sambamoorthi U. Functional impairment trajectories among persons with HIV disease: a hierarchical linear models approach. Health Serv Res 1996;31:469-88.

7. Lange JMA. Current problems and the future of antiretroviral drug trials. Science 1997;276:548-50.

8. Pollack S. A role for academic medical centers in the era of managed care: immediate, interactive, free information. Acad Med 1998;73:357-9.

9. Saver BG, Doescher MP. The rising tide of hospitalism: evidence-based or anecdote-based medicine? J Fam Pract 1998;46:465-8.

10. Donohoe MT. Comparing generalist and specialty care-discrepancies, deficiencies, and excesses. Arch Intern Med 1998;158:1596-608.

11. Schroeder SA, Schapiro R. The hospitalist: new boon for internal medicine or retreat from primary care? Ann Intern Med 1999;130:382-7.

12. Goldman L. The impact of hospitalists on medical education and the academic health system. Ann Intern Med 1999;130:364-7.

13. Wachter RM. An introduction to the hospitalist model. Ann Intern Med 1999;130:338-42.

14. Go AS, Rao RK, Dauterman KW, Massie BM. A systematic review of the effects of physician specialty on the treatment of coronary disease and heart failure in the United States. Am J Med 2000;108:216-26.

15. Golomb BA, Pyne JM, Wright B, Lohr JB, Bozette SA. The role of psychiatrists in primary care of patients with severe mental illness. Psychiatr Serv 2000;51:766-73.

16. Kitahata MM, Holmes EK, Wagner EH, Gooding TD. Caring for persons with HIV infection in a managed care environment. Am J Med 1998;104:511-5.

17. Grace CJ, Richardson K, Kutzko D, Alston WK, Ramundo M. Service delivery for patients with HIV in a rural state: the Vermont model. AIDS Patient Care Stds 1999;13:659-66.

18. Turner BJ, Newschaffer CJ, Cocroft J, Fanning TR, Marcus S, Hauck WW. Improved birth outcomes among HIV-infected women with enhanced medicaid prenatal care. Am J Public Health 2000;90:85-91.

19. Gallant JE. Strategies for long-term success in the treatment of HIV infection. JAMA 2000;283:1329-34.

Ninety-four of the papers reviewed (78%) referred to the Kitahata et al study as evidence of the association between experience and patient outcomes (“experience articles”). Ten of the papers (8%) referred to the study as evidence of the benefits of specialty or specialized care (“specialty articles”); quotations from those papers are shown in Table 2.^6-15 Four of the papers reviewed (3%) referred to the study as evidence of the benefits of expertise or expert care (“expert articles”); these quotations are shown in Table 3.^16-19 Thirteen of the articles reviewed (11%) listed the study as a general reference or cited it for reasons other than making an assertion about the relationship between health care provider characteristics and patient outcomes. On the initial review, the authors’ classification of the passage differed for only 5 (4%) of the 121 articles reviewed; all of these were ultimately classified as “experience” or “other” articles.

“Specialty articles” were more likely to be non-HIV related (80%) than HIV related, while “experience articles” were more likely to be HIV related (81%); this difference was statistically significant (P <.001). “Specialty articles” were also more likely to be reviews or editorials (80%); most of the “experience articles” were original research (59%); the P value for this difference was 0.02 by Fisher’s exact test.

Discussion

This study illustrates the various ways in which the results of a single study are interpreted and conveyed to readers of scientific papers. The study by Kitahata and colleagues was a comparison of generalists with varying levels of experience. It was not a study of specialists or specialized care; nevertheless, it was presented as such in 8% of the articles reviewed. Another 3% cited the study as evidence of the benefits of “expert care”; this is perhaps justifiable, but the study did not measure expertise, and the leap from experience to expertise is questionable, at best.

There are a few possible explanations for the authors’ misrepresentation of the article by Kitahata et al. The most obvious explanation is that the authors in these situations were willing to manipulate the results of the study to bolster their argument. Most of the articles that referred to this study as evidence of the benefits of specialized care (8 of 10) were review articles on the topic of specialized care⁸¹⁵ and half of those were focused on hospitalism^9,11-13; these are settings in which the authors may have had an incentive to present the study in this manner. It is possible, however, that the authors felt free to extrapolate from the results of this study and argue that since a relatively modest increase in experience improves outcomes, an even greater increase (ie, specialization) would improve outcomes even more. However, this argument was never explicitly made in these examples. A few of the articles reviewed did make this type of argument, but we classified these as “experience” articles. A final possible explanation is that the authors used a previous author’s reference without reviewing the study themselves; however, this seems less likely, since the title of the study clearly states that physician experience was the variable studied.

Limitations

Our study has a number of limitations. The source article was not chosen randomly, but was based on an observation of misrepresentation, so the frequency of misuse may be higher than with other articles. However, the misquotation rate found in this study is consistent with previous studies. One analysis of 6 journals reported a misquotation rate of 15%; 8% of these were felt to be major errors.³ A second study found 37 major quotation errors in 150 randomly selected references from 137 different articles.⁴ Another limitation of this study is that not all the articles that cited the study were analyzed; however, the number of unanalyzed studies is fairly small (8%) and unlikely to affect the overall conclusions.

Conclusions

Authors of scientific papers have a responsibility to convey accurately the information they have gathered to their readers. This study suggests that some authors are willing to bend or break this rule. It is unlikely that reminders about adherence to the responsibilities of authorship will alter this habit. Increased scrutiny of references in the peer-review process, although difficult, is most likely the only way to guard against these types of manipulations and misrepresentations.