The Standardized Inclusive Numeric Academic Index: An Index to Measure Academic Productivity in Plastic Surgery

Abstract

Purpose: Academic productivity is a poorly defined metric that is commonly used to determine faculty career advancement. While various indices incorporate scholarly activity, no specific index assimilates the perceived importance of a variety of academic accomplishments a physician may make. Herein, the development and validation of an algorithm to generate an academic productivity score based on surveying physicians nationwide are described.

Methods: From 2016 to 2018, an online cross-sectional survey was distributed to faculty members at an academic institution and plastic surgeons from different academic levels nationwide. Respondents were presented with randomized, binary comparisons of 42 different achievements of an academic physician and asked to choose the more important achievement. Descriptive statistics of demographics and "win rates" of each achievement were reported and an algorithm for academic productivity scoring was designed. To validate the proposed index, 30 curricula vitae of academic surgeons were anonymized and ranked in order of increasing academic achievement by 6 volunteers. Interrater reliability was assessed by Krippendorff α (α ≥ 0.800).

Results: Survey respondents completed an average of 116 (SD, 97.6) comparisons each, generating a total of 14,736 ranked comparisons. Of the 42 variables, the highest win rates were attained by being the dean of a medical school (0.90) and editor of a medical journal (0.88). The lowest win rates were attained by industry spokesperson (0.1) and members of the local medical society (0.1). Initial validity evidence found the interrater reliability for the 6 rankers to have a Krippendorff α value of 0.843. The interrater reliability between the average rater ranking and the algorithm-generated ranking had a Krippendorff α value of 0.925.

Conclusions: The present study demonstrates that the standardized inclusive numeric academic index may be used as a valid, comprehensive measure of academic productivity. Future studies should assess its application across different medical specialties.