Academic ranking score: A publication-based reproducible metric of thought leadership in urology

Background: Hospital rankings have become integral to the marketing strategies of many health care systems. Methodology used in compiling these lists appears highly flawed. Objective: To improve on current hospital ranking systems and to develop a more meaningful measure of a urology department's contribution to the field, we developed an academic ranking score (ARS) based on publicly available data. Design, setting, and participants: An active faculty list was assembled for each department. A list of all publications from each department from 2005 to 2010 was then compiled. Only publications with faculty members as first or last author were considered. The ARS was then derived by identifying the number of publications within an institution, normalized by the impact factor of the peer-reviewed journal in which the publication appeared. Measurements: The 2010 U.S. News & World Report (USNWR) urology list was reranked based on ARS and compared with the USNWR rank list. ARS was also calculated for several leading European urologic centers. Results and limitations: A total of 6437 urologic publications were indexed to calculate the ARS. Two of the top three programs in the USNWR rankings dropped out of the top 10. The top 10 academically ranked programs increased or decreased an average of >5 positions (range: 0-17). No correlation was seen between programs ranked in the top 10 by USNWR and our objective ARS method (Spearman ρ: -0.1; p = 0.75). Because ARS only includes first- or last-author publications for faculty with clinical duties, ARS likely excludes basic science contributions and contributions from nonclinical faculty. Conclusions: Ranking of urology departments through quantification of each program's recent academic contribution, as captured by the ARS, differs substantially from rankings developed by USNWR. Integration of such objective measures into an overall urology program ranking system would replace current subjective opinions marred by historical biases with up-to-date merit-based assessments.