Original Research

Prevalence and Impact of Self-Citation in Academic Orthopedic Surgery

Publish date: March 16, 2018

References

1. Atasoylu AA, Wright SM, Beasley BW, et al. Promotion criteria for clinician-educators. J Gen Intern Med. 2003;18(9):711-716.

2. Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-16572.

3. Stavrakis AI, Patel AD, Burke ZD, et al. The role of chairman and research director in influencing scholarly productivity and research funding in academic orthopaedic surgery. J Orthop Res. 2015;33(10)1407-1411. doi:10.1002/jor.22919.

4. Lopez J, Susarla SM, Swanson EW, Calotta N, Lifchez SD. The association of the H-index and academic rank among full-time academic hand surgeons affiliated with fellowship programs. J Hand Surg Am. 2015;40(7):1434-1441. doi:10.1016/j.jhsa.2015.03.026.

5. Schoenfeld AJ, Bhalla A, George J, Harris MB, Bono CM. Academic productivity and contributions to the literature among spine surgery fellowship faculty. Spine J. 2015;15(10)2126-2131. doi:10.1016/j.spinee.2015.03.026.

6. Jackson JB. Promotion at the Johns Hopkins School of Medicine. Johns Hopkins Medicine. https://www.hopkinsmedicine.org/gim/useful_links/PPC%20Promotion.pdf. Accessed February 1, 2015.

7. Appointments, promotion, and tenure. The Ohio State University School of Medicine. https://oaa.osu.edu/sites/default/files/uploads/governance-documents/college-of-medicine/surgery/Surgery_APT_5-20-14.pdf. Accessed February 1, 2015.

8. Colaco M, Svider PF, Mauro KM, Eloy JA, Jackson-Rosario I. Is there a relationship between National Institutes of Health funding and research impact on academic urology? J Urol .2013;190(3):999-1003. doi:10.1016/j.juro.2013.02.3186.

9. Svider PF, Husain Q, Folbe AJ, Couldwell WT, Liu JK, Eloy JA. Assessing National Institutes of Health funding and scholarly impact in neurological surgery. J Neurosurg. 2014;120(1):191-196. doi:10.3171/2013.8.JNS13938.

10. Svider PF, Mauro KM, Sanghvi S, Setzen M, Baredes S, Eloy JA. Is NIH funding predictive of greater research productivity and impact among academic otolaryngologists? Laryngoscope. 2013;123(1):118-122. doi:10.1002/lary.23659.

11. American Medical Association. FREIDA Online. http://www.ama-assn.org/ama/pub/education-careers/graduate-medical-education/freida-online.page? Accessed February 1, 2015.

12. NIH. Research Portfolio Online Reporting Tools. https://projectreporter.nih.gov/reporter.cfm. Accessed February 1, 2015.

13. Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB Journal. 2008;22(2):338-342. doi:10.1096/fj.07-9492LSF.

14. Gast KM, Kuzon WM Jr, Waljee JF. Bibliometric indices and academic promotion within plastic surgery. Plast Reconstr Surg. 2014;134(5):838e-844e. doi:10.1097/PRS.0000000000000594.

15. Svider PF, Pashkova AA, Choudhry Z, et al. Comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope. 2013;123(4):884-889. doi:10.1002/lary.23951.

16. Rad AE, Shahgholi L, Kallmes D. Impact of self-citation on the H index in the field of academic radiology. Acad Radiol. 2012;19(4):455-457. doi:10.1016/j.acra.2011.11.013.

17. Hakkalamani S, Rawal A, Hennessy MS, Parkinson RW. The impact factor of seven orthopaedic journals: factors influencing it. J Bone Joint Surg Br. 2006;88(2):159-162. doi:10.1302/0301-620X.88B2.16983.

18. Foley JA, Della Sala S. The impact of self-citation. Cortex. 2010;46(6):802-810. doi:10.1016/j.cortex.2010.01.004.

19. Opthof T. Inflation of impact factors by journal self-citation in cardiovascular science. Neth Heart J. 2013;21(4):163-165. doi:10.1007/s12471-013-0384-0.

20. Gami AS, Montori VM, Wilczynski NL, Haynes RB. Author self-citation in the diabetes literature. CMAJ. 2004;170(13):1925-1927.

21. Fassoulaki A, Paraskeva A, Papilas K, Karabinis G. Self-citations in six anaesthesia journals and their significance in determining the impact factor. Br J Anaesth. 2000;84(2):266-269.

22. Kulkarni AV, Aziz B, Shams I, Busse JW. Author self-citation in the general medicine literature. PloS One. 2011;6(6): e20885. doi:10.1371/journal.pone.0020885.

23. Bartneck C, Kokkelmans S. Detecting h-index manipulation through self-citation analysis. Scientometrics. 2011;87(1):85-98. doi:10.1007/s11192-010-0306-5.

STATISTICAL ANALYSIS

Bibliometric analyses were presented descriptively with means ± standard deviation. After testing for normality, differences in the h-index between groups were assessed via analysis of variance tests. The proportional increase in the number of citations and the h-index were calculated by dividing the difference between the before and after exclusion of self-citations by the total number of citations before exclusion. The relationship between the number of publications and the proportional change in the h-index was assessed via calculation of the Spearman correlation coefficient. The independent variable was the number of publications, and the proportional increase in the h-index via self-citation was the dependent variable. Statistical tests were performed on STATA 13 (StataCorp) and the results considered significant if P < .05. Figures were created using GraphPad Prism 6.02 Software.

RESULTS

A total of 463 orthopedic investigators were included (147 chairpersons, 118 program directors, and 198 NIH-funded faculty) (Table 1). On average, these researchers produced 72.3 ± 83.0 articles and referenced 2139 ± 3222 articles (mean, 29.6 references per article). The cumulative h-index was 19.2 ± 15.6, and was the highest among NIH-funded researchers (24.3 ± 17.0) (P < .001). In all, 83.8% of orthopedic investigators self-cited their previous work at least once, and the total number of self-citations was highest among NIH-funded investigators (221 ± 355) (P < .001). After these self-citations were excluded, the h-index changed by 0.6 ± 1.1 for all investigators, and this change was greatest among NIH-funded researchers (1.1 ± 1.3) (P < .001).

Table 1. Effect of Self-Citation on NIH-funded Investigators, Chairpersons, and Program Directors in Orthopedics

Investigator	N (%)	Articles, n (mean ± SD)	Total Citations (mean ± SD)	h-index	Self-Citations (mean ± SD)	Corrected h-index	∆ h-index
NIH-funded	198 (42.8)	87.6 ± 84.9	3086 ± 3799	24.3 ± 17.0	221 ± 355	23.2 ± 16.3	1.1 ± 1.3
Chairperson	147 (31.7)	85.3 ± 95.5	2151 ± 3098	19.9 ± 15.0	85.2 ± 221	19.5 ± 14.5	0.4 ± 0.8
Program Director	118 (25.5)	30.5 ± 35.9	536.8 ± 785	9.6 ± 7.2	8.8 ± 19.9	9.5 ± 7.1	0.1 ± 0.3
Total	463 (100)	72.3 ± 83.0	2139 ± 3222	19.2 ± 15.6	123.9 ± 277.6	18.5 ± 14.9	0.6 ± 1.1

Abbreviation: NIH, National Institutes of Health.

Most orthopedic investigators did not increase their h-index via self-citation (63.7%, P < .001). Table 2 categorizes investigators by changes in their h-index after excluding self-citations (range, 0-11). The maximal change in the h-index was seen in the most prolific group of investigators, who produced 261.0 ± 149.3 articles. In this group, the h-index increased by 11.1% ± 5.2%. The Figure investigates the relationship between the number of articles and the proportional increase in the h-index. The number of publications was positively correlated with the change in h-index after self-citations were excluded (r = 0.448, P < .001).

Table 2. Stratification of Orthopedic Researcher Investigators by Change in h-index After Self-Citation

∆ h-index	N (%)	Articles (mean ± SD)	Self-Citations (mean ± SD)	h-index (mean ± SD)	% Increase in h-index
0	295 (63.7)	43.8 ± 51.3	27.6 ± 58.4	13.1 ± 10.7	0
1	101 (21.8)	87.9 ± 68.3	126.0 ± 130.6	24.0 ± 13.3	5.9 ± 4.1
2	42 (9.1)	141.9 ± 111.1	331.6 ± 318.0	32.4 ± 16.6	8.4 ± 5.5
3	14 (3.0)	203.1 ± 92.6	611.6 ± 332.9	45.4 ± 14.9	7.6 ± 3.6
4+	11 (2.4)	261.0 ± 149.3	1277.1 ± 692.4	53.1 ± 18.9	11.1 ± 5.2

DISCUSSION

The practice of self-citation is widely prevalent among experienced orthopedic investigators. However, this practice seems to have minimal impact on the h-index for most investigators. Self-citation had a measurable impact on the h-index only after an investigator had many publications. At a mean of 87.9 ± 68.3 articles, investigators had a ∆h-index of 1. This represented a mean 5.9% increase. Overall, these findings underscore the utility of the h-index in assessing scholarly impact and ameliorate concerns over bibliometric manipulation.

Among a large group of experienced orthopedic investigators, self-citation has minimal effect on the h-index. Importantly, most investigators (63.7%) did not experience a full integer increase in their h-index. At a threshold of ∆ h-index increase of 1, investigators had impressive h-indices (24.0 ± 13.3), which eclipsed those of recent studies of hand surgeons (10.2 ± 9.9) and spine surgeons (13.6 ± 8.7).^4,5 This finding suggests that committees for academic promotion in orthopedic surgery may disregard the impact of self-citation on the h-index. While the thresholds for promotion have not been defined in the orthopedic literature, a study in plastic surgery established an h-index threshold of 14.5 for promotion from associate to full professor.¹⁴ It may be, however, that h-indices are higher among orthopedic surgeons, as a previous study reported an h-index of 20 among full professors.¹⁵ Future research is needed to determine thresholds for promotion within orthopedic surgery, as the h-index varies by specialty according to unique citation patterns.

Continue to: It is worth highlighting...