Original Research

Prevalence and Impact of Self-Citation in Academic Orthopedic Surgery

Publish date: March 16, 2018

References

1. Atasoylu AA, Wright SM, Beasley BW, et al. Promotion criteria for clinician-educators. J Gen Intern Med. 2003;18(9):711-716.

2. Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-16572.

3. Stavrakis AI, Patel AD, Burke ZD, et al. The role of chairman and research director in influencing scholarly productivity and research funding in academic orthopaedic surgery. J Orthop Res. 2015;33(10)1407-1411. doi:10.1002/jor.22919.

4. Lopez J, Susarla SM, Swanson EW, Calotta N, Lifchez SD. The association of the H-index and academic rank among full-time academic hand surgeons affiliated with fellowship programs. J Hand Surg Am. 2015;40(7):1434-1441. doi:10.1016/j.jhsa.2015.03.026.

5. Schoenfeld AJ, Bhalla A, George J, Harris MB, Bono CM. Academic productivity and contributions to the literature among spine surgery fellowship faculty. Spine J. 2015;15(10)2126-2131. doi:10.1016/j.spinee.2015.03.026.

6. Jackson JB. Promotion at the Johns Hopkins School of Medicine. Johns Hopkins Medicine. https://www.hopkinsmedicine.org/gim/useful_links/PPC%20Promotion.pdf. Accessed February 1, 2015.

7. Appointments, promotion, and tenure. The Ohio State University School of Medicine. https://oaa.osu.edu/sites/default/files/uploads/governance-documents/college-of-medicine/surgery/Surgery_APT_5-20-14.pdf. Accessed February 1, 2015.

8. Colaco M, Svider PF, Mauro KM, Eloy JA, Jackson-Rosario I. Is there a relationship between National Institutes of Health funding and research impact on academic urology? J Urol .2013;190(3):999-1003. doi:10.1016/j.juro.2013.02.3186.

9. Svider PF, Husain Q, Folbe AJ, Couldwell WT, Liu JK, Eloy JA. Assessing National Institutes of Health funding and scholarly impact in neurological surgery. J Neurosurg. 2014;120(1):191-196. doi:10.3171/2013.8.JNS13938.

10. Svider PF, Mauro KM, Sanghvi S, Setzen M, Baredes S, Eloy JA. Is NIH funding predictive of greater research productivity and impact among academic otolaryngologists? Laryngoscope. 2013;123(1):118-122. doi:10.1002/lary.23659.

11. American Medical Association. FREIDA Online. http://www.ama-assn.org/ama/pub/education-careers/graduate-medical-education/freida-online.page? Accessed February 1, 2015.

12. NIH. Research Portfolio Online Reporting Tools. https://projectreporter.nih.gov/reporter.cfm. Accessed February 1, 2015.

13. Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB Journal. 2008;22(2):338-342. doi:10.1096/fj.07-9492LSF.

14. Gast KM, Kuzon WM Jr, Waljee JF. Bibliometric indices and academic promotion within plastic surgery. Plast Reconstr Surg. 2014;134(5):838e-844e. doi:10.1097/PRS.0000000000000594.

15. Svider PF, Pashkova AA, Choudhry Z, et al. Comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope. 2013;123(4):884-889. doi:10.1002/lary.23951.

16. Rad AE, Shahgholi L, Kallmes D. Impact of self-citation on the H index in the field of academic radiology. Acad Radiol. 2012;19(4):455-457. doi:10.1016/j.acra.2011.11.013.

17. Hakkalamani S, Rawal A, Hennessy MS, Parkinson RW. The impact factor of seven orthopaedic journals: factors influencing it. J Bone Joint Surg Br. 2006;88(2):159-162. doi:10.1302/0301-620X.88B2.16983.

18. Foley JA, Della Sala S. The impact of self-citation. Cortex. 2010;46(6):802-810. doi:10.1016/j.cortex.2010.01.004.

19. Opthof T. Inflation of impact factors by journal self-citation in cardiovascular science. Neth Heart J. 2013;21(4):163-165. doi:10.1007/s12471-013-0384-0.

20. Gami AS, Montori VM, Wilczynski NL, Haynes RB. Author self-citation in the diabetes literature. CMAJ. 2004;170(13):1925-1927.

21. Fassoulaki A, Paraskeva A, Papilas K, Karabinis G. Self-citations in six anaesthesia journals and their significance in determining the impact factor. Br J Anaesth. 2000;84(2):266-269.

22. Kulkarni AV, Aziz B, Shams I, Busse JW. Author self-citation in the general medicine literature. PloS One. 2011;6(6): e20885. doi:10.1371/journal.pone.0020885.

23. Bartneck C, Kokkelmans S. Detecting h-index manipulation through self-citation analysis. Scientometrics. 2011;87(1):85-98. doi:10.1007/s11192-010-0306-5.

The competitive nature of academic research requires objective metrics to define career end points, such as promotion and funding procurement. While various criteria are used to assess performance in academia, publications and research funding are particularly regarded.¹ Quantifying research dollars is relatively straightforward, but measuring research productivity is more complex. Not all articles are created equal, and disparities exist regarding effort and the ultimate impact of articles. In 2005, a physicist created the h-index to measure both research impact and productivity.² As a bibliometric, the h-index equals the number of publications h that have been cited at least h times. Given its simplicity, the h-index has gained wide popularity in diverse medical specialties, including orthopedic surgery.³ Other recent studies have applied the h-index to hand surgery and spine surgery.^4,5

Importantly, some authors have raised concerns regarding potential limitations of the h-index. One potentially significant limitation is the ability of authors to artificially inflate their h-index via self-citation. The impact of this practice is of particular interest as the h-index becomes widely adopted as a metric for promotion at many academic institutions.^6-7 Furthermore, scholarly productivity has remained a critical component of successful grant funding procurement, and future grant funding applications may evaluate the h-index.^8-10

The purpose of this study is to determine the prevalence and impact of self-citation on the h-index in a large cohort of orthopedic investigators. Given their high level of investment in academic orthopedic surgery, we focused on program directors, chairpersons, and National Institutes of Health (NIH)-funded research faculty at orthopedic surgery residency programs.

METHODS

INCLUSION CRITERIA

This study qualified as non-human and non-animal research and received exemption per the standing policy of the Institutional Review Board. The Fellowship and Residency Electronic Interactive Database (FREIDA) was accessed to generate a list of orthopedic residency program directors.¹¹ This database was also used to generate a list of allopathic orthopedic surgery residency programs. Official program websites were accessed to generate a list of orthopedic chairpersons. Lastly, the NIH RePORTER was used to generate a list of basic science orthopedic investigators who received funding anytime during 2011 to 2014.¹² This methodology was used due to the lack of reporting of basic science investigators on program websites. A list of NIH-funded orthopedic investigators was cross-referenced via an online search to isolate a cohort of PhD investigators.

Orthopedic faculty were defined as chairpersons, program directors, or NIH-funded investigators. In cases of overlap, preference was given in that order. Orthopedic investigators who had not published an article after 1995 were excluded (6 chairpersons, 1 program director).

BIBLIOMETRIC ANALYSIS

While several resources exist to calculate the h-index, the Scopus database (Elsevier) is one of the easiest programs to use.¹³ Author entries are created via institutional affiliations, thereby alleviating the need for manual reconciliations. Investigators were identified on Scopus via “author last name” and “first name, middle initial.” For each author, publications were screened for relevance to the field of orthopedics. Affiliated institutions were cross-referenced with information obtained from individual program websites. The “view h-graph” feature was used to calculate the number of publications, h-index, and number of citations. Then, the “Exclude self-citations” feature was used to calculate the number of corrected citations and the h-index excluding self-citations. Metrics were calculated over a 2-day period.

Continue to: STATISTICAL ANALYSIS