1. Background
Is the h-index an appropriate scientometric criterion? Does it indicate all the research realities and potentials of a researcher? Is it an appropriate indicator for comparing researchers? As an indicator, the h-index is an author-level metric that attempts to measure both the productivity and citation impact of the publications of a scientist or scholar. The index is based on the set of the scientist’s most cited papers and the number of citations that they have received in other publications (Figure 1) (1). The h-index cannot be a comprehensive and general indicator for ranking faculty members and researchers since it is a confounded criterion far from reality. It has several weak points, some of which are discussed below to clarify the point further.
Figure 1.
The index is based on the set of the scientist’s most cited papers and the number of citations that they have received in other publications
1.1. The First Weak Point
Consider a researcher or a faculty member with 30 years and another with five years of research experience. Naturally, the first researcher has had the time and opportunity for conducting research activities six times more than the second researcher. At the first glance, it seems that the first researcher with 30 years of experience must have a higher h-index. In most cases, however, this is not the case, since there are countless researchers who, despite having a shorter time for research, have conducted numerous studies and, therefore, have a much higher h-index compared to researchers with over 30 years of research and scientific experience. For instance, a researcher with five years of research experience may have an h-index of 60 while another with 30 years of experience may have an h-index of 20. To compare the research activities of these two researchers, it does not suffice to simply compare the h-index values. Rather, one must take into account the difference in the duration of activity. In other words, before any comparison can be made, one must allocate appropriate weights to each researcher based on the duration of research activity. In this hypothetical example, first a five-time weight must be assigned to the h-index of the first researcher to compute the actual value of the h-index, i.e., 300 (instead of 60), and then compare it to that of the second researcher, i.e., 20. Consequently, a comparison without adjustment for the duration of research activities is, in fact, a comparison far from reality and, thus, confounded. In the same hypothetical example, the first researcher has conducted more effective research activities than the second based on the three-time-larger h-index. However, after adjustment for the duration of research activity, it can be concluded that the first researcher has conducted research 10 times more effective than the second (1).
1.2. The Second Weak Point
Naturally, not all researchers are heads of research centers, research institutes, departments, and so on, since these are one-person positions. Management and the rights delegated to managers can have outcomes which directly or indirectly overestimate the h-index of managers.
One only has to be the manager of a research institute; due to managerial influence, all researchers working in these research institutes must cite the name of managers as a co-author when publishing the results of their studies, while managers have actually not contributed to the study at all. This sad reality has interesting results; it is observed that, in the duration of a year, hundreds of articles with the name of managers are published in credible scientific journals, which are indexed in credible databases such as PubMed, Scopus, and Web of Science. A higher number of published articles increases the citation by other researchers, thereby increasing the h-index of managers. The fact becomes even harsher when we learn that holding such managerial positions in research centers and universities around the world is not based on one’s scientific capability but lobbying, non-scientific connections, and political influence, which is a serious warning sign for science and the scientific community.
1.3. The Third Weak Point
Unfortunately, selling and purchasing articles and dissertations has become customary in some countries. Many students have received their M.Sc./MA or Ph.D. by purchasing articles and dissertations, and many faculty members have been promoted by directly or indirectly purchasing articles. Prices are proportional to the impact factor of the journal; the more credible the journal, the higher the price.
Although there are few such graduate students and faculty members, the existing number is a disgrace for the scientific community. As no specified program exists for faculty members, they significantly differ in terms of income; clinical and therapeutic faculty members visit patients at the workplace, offices, and state and private hospitals and, therefore, have little time for conducting research and publishing the results, while having sufficient money to purchase ready-for-sale articles. Thus, the more they pay, the higher their h-index will be.
1.4. The Fourth Weak Point
Researchers conduct research in various specialized and scientific fields. Based on their specialty, the ability to conduct research and publish the results may significantly differ. For instance, biostatistics or epidemiology experts have sufficient knowledge to conduct studies or, at least, cooperate in data analysis. Consequently, more studies will be published with their names. Thus, the h-index is flawed in this regard and cannot be an appropriate scientometric indicator. It suffices to examine the h-index of Professor Maryam Mirzakhani, a mathematics genius and the winner of the most credible prize in mathematics, the Fields Medal. As of July 2017, her h-index is equal to 8 (2). Table 1 shows the h-index of the Fields Medal winner in mathematics and Nobel Prize winners in physics, chemistry, physiology or medicine in 2017 ranging from 8 to 96. Do these values truly represent their scientific influence in their scientific domains?
Table 1.The H-Index of the Fields Medal Winner in Mathematics and Nobel Prize Winners in Physics, Chemistry, Physiology or Medicine in 2017a
| Author | Documents | Subject Area | Affiliation | City | Country | H. Index | |
|---|---|---|---|---|---|---|---|
| 1 | Mirzakhani, Maryam | 20 | Mathematics | Stanford University | Palo Alto | United States | 8 |
| 2a | Weiss, Rainer | 239 | Physics | Massachusetts Institute of Technology | Cambridge | United States | 71 |
| 2b | Barish, Barry C. | 465 | Physics | California Institute of Technology | Pasadena | United States | 73 |
| 3a | Dubochet, Jacques | 144 | Chemistry | Universitat Lausanne Schweiz | Lausanne | Switzerland | 54 |
| 3b | Frank, Joachim | 368 | Chemistry | Columbia University in the City of New York | New York | United States | 86 |
| 3c | Henderson, Richard | 145 | Chemistry | Newcastle University | Newcastle | United Kingdom | 29 |
| 4a | Young, Michael | 118 | Physiology or medicine | Rockefeller University | New York | United States | 56 |
| 4b | Michael Rosbash | 318 | Physiology or medicine | Brandeis University | Waltham | United States | 96 |
| 4c | Hall, Jeffrey C. | 210 | Physiology or medicine | Brandeis University | Waltham | United States | 80 |
a1, mathematics; 2, physics; 3, chemistry; 4, medicine.
1.5. The Fifth Weak Point
Some researchers cite their own previously published articles. Self-citation is one way of increasing a researcher’s h-index.
1.6. The Sixth Weak Point
Articles published in credible and high impact scientific journals are often based on the work of a large team with international cooperation. If you examine the list of authors for these articles, you find over 100 - 150 names since each researcher has performed his/her specialized task in the study. These articles usually have very high citations, increasing the h-index for all contributors. However, participation in international research projects is not possible for all researchers, showing another weak point of the use of h-index for comparing researchers.
1.7. The Seventh Weak Point
First author and corresponding author: naturally, the majority of activities leading up to the publication of an article must be performed by the first author, and the role of the corresponding author is supervising the conduction of study and commenting on the work to promote its quality. Nevertheless, in universities, research centers, and research institutes, corresponding authors contribute very little to the publication of the study and, in some cases, does not even read the articles published with their name.
Varying degrees of this issue is observed in different countries, mostly in developing countries. Therefore, in these countries, we encounter researchers with a high h-index who have not read most articles published with their name, showing another weak point of the h-index.
It is evident that by accepting payment of the article processing charge (APC), a manuscript will be published in open access journals; therefore, such papers are visible to every researcher around the world, and the number of citations will be high.
Therefore, the visibility and accessibility of the paper is another factor that can easily increase the h-index of the authors. For sure, the scientific impact of research work has nothing to do with the visibility of that scientific work.
2. Results and Discussion
The h-index cannot be an appropriate criterion for judging research activities. It is especially misleading when used to compare researchers due to the fact that it is easily influenced by confounding variables such as years of experience, political and administrative influence, specialized field, payments and other privileges, and the type of university in which the researcher works. In short, the value of h-index may be misleading. Instead, it is recommended to consider another indicator called the adjusted h-index, which is even better to focus on the outcomes of research in terms of resolving the problems in the respective research field. In short, to rank the research activities of researchers, it is better to move from quantitative indicators to qualitative ones.
Developed countries have been changing the policies for evaluating research activities from quantitative indicators such as the h-index to another indicator named complementary scale for the h-index or the stratified h-index, which adjusted for few of the above-mentioned confounders (2-5). For sure, qualitative indicators will be a much better index to assess the impact of the researchers. Developing countries must also consider this change so that the real value of researchers would be known. The never-ending competition among researchers in developing countries for increasing their h-index must be over since it only wastes time and money.
