Abstract
Background:
Citation is being increasingly used as a key performance indicator in research policies and evaluation systems.Objectives:
This study aimed to investigate the factors affecting the citation status of clinical articles published in the Iranian Journal of Radiology (IJR).Materials and Methods:
This cross-sectional study, conducted during May-June 2023, employed bibliometric techniques and investigated the IJR website and the Scopus database. The analysis encompassed various characteristics of the published papers, such as citation status, number of authors, sample size, page count, number of references, year of publication, type of paper, scope of the paper's title, study design, number of articles by the first and corresponding authors in Scopus, h-indices of the first and corresponding authors in Scopus, and nationality of the first and corresponding authors. The citation status was considered as the dependent outcome in the analysis, with a significance level set at P < 0.05.Results:
From 2018 to 2022, the IJR published 357 papers, with an average citation rate of 33.6% and an impact factor of 0.9. The majority of journals citing the articles (36.03%) were ranked in the second quartile according to the Scopus Cite Score Index. The citation status was found to be significantly influenced by several factors, including the study design, scope of the paper's title, type of article, number of years since publication, number of articles by the first and corresponding authors in Scopus, as well as the h-index of both the first and corresponding authors in Scopus (P < 0.05). The results of multiple logistic regression analysis revealed that only the number of years since publication, type of paper, and scope of the paper's title were significantly associated with the citation status.Conclusion:
The present research indicated that the number of years since publication, type of paper, and scope of the paper's title were the most significant factors influencing the citation status of papers in the IJR. These findings provide valuable insights into the citation patterns of articles in the IJR and can help authors and policymakers develop strategies to enhance the visibility and impact of their research.Keywords
Radiology Paper Citation H-index Iranian Journal of Radiology Bibliometric
1. Background
Evidence-based medicine is a methodology that guides clinical practitioners in accessing, evaluating, and applying scientific evidence to make the most appropriate treatment decisions (1). Citation status is an important measure of a journal's impact within its field, as it reflects the extent to which the research published in the journal has been recognized and cited by other researchers (2). Additionally, the citation status is often used as a criterion for evaluating the quality and significance of a journal when making decisions about publishing, funding, and career advancement (3).
The Iranian Journal of Radiology (IJR) is one of the leading scientific journals in Iran, which publishes original research articles, reviews, and case reports related to radiology and imaging sciences. It is well-established that the citation status of a journal is an essential indicator of its impact and influence in the scientific community (4, 5). In recent years, there has been an increasing interest in identifying the factors that influence the citation status of scientific journals (6-8). Several studies have been conducted to investigate the determinants of citation status, such as the number and quality of published articles, the reputation of the journal, the editorial policies, and the scope of the journal (9, 10). In this regard, a study by López et al. on plastic surgery articles found that the number of citations was influenced by the subject area, conflict of interest, number of authors, and journal, but not by the level of evidence or the method of study. This finding suggests that other factors beyond the strength of evidence can also impact the citation rate of articles (11).
2. Objectives
In Iran, there is limited research on the citation status of scientific journals, particularly in the field of radiology. This study aimed to investigate the citation status of articles published in the IJR in Iran's scientific community, specifically in the field of radiology. It also aimed to identify the influential factors in citation status and provide strategies to enhance the journal's citation status and impact.
3. Materials and Methods
3.1. Study Design
This cross-sectional study employed bibliometric techniques to analyze articles published in the IJR.
3.2. Variables
Data was collected from the IJR website, as well as the Scopus database. A significant portion of the data was sourced from the IJR website, whereas the citation data was retrieved from the Scopus database. Data extraction involved two researchers who gathered information by reviewing journal papers and conducting searches in the Scopus database for indexed data. The following paper characteristics were retrieved from the articles: number of citations, number of authors, sample size, page count, number of references, number of years since paper publication, type of article (case report, research, review, etc.), study design (descriptive observational, analytical observational, interventional, etc.), number of articles by the first author in Scopus, number of articles by the corresponding author in Scopus, h-index of the first author in Scopus, h-index of the corresponding author in Scopus, nationality of the first author (Iranian, Chinese, etc.), and nationality of the corresponding author (Iranian, Chinese, etc.). Moreover, the h-index and the Scopus Cite Score Index of journals citing the IJR articles were gathered. The variables were categorized by the researchers. Table 1 presents the various strata for all the variables.
Descriptive Data Pertaining to the Qualitative and Quantitative Characteristics of Papers Published in the Iranian Journal of Radiology (IJR) During 2018 - 2022
| Variables | Mean ± SD/No. (%) |
|---|---|
| Number of papers per year | |
| 2018 | 89 (24.9) |
| 2019 | 75 (21.0) |
| 2020 | 64 (17.9) |
| 2021 | 71 (19.9) |
| 2022 | 58 (16.2) |
| Total | 357 (100) |
| Number of citations per year | |
| 2018 | 101 (31.46) |
| 2019 | 78 (24.29) |
| 2020 | 117 (36.44) |
| 2021 | 22 (6.85) |
| 2022 | 3 (0.93) |
| Total | 321 (100) |
| Mean number of citations per year | |
| 2018 | 1.13 ± 1.65 |
| 2019 | 1.04 ± 1.58 |
| 2020 | 1.83 ± 7.35 |
| 2021 | 0.31 ± 0.60 |
| 2022 | 0.05 ± 0.22 |
| Total | 0.9 ± 3.34 |
| Citation status of papers | |
| Yes | 119 (33.6) |
| No | 238 (66.4) |
| Categories of citation frequencies | |
| 0 | 238 (66.4) |
| 1 | 61 (17.1) |
| 2 - 5 | 48 (13.4) |
| 6 - 10 | 8 (2.5) |
| >10 | 2 (0.6) |
| Scope of the paper's title | |
| Vascular & interventional | 57 (16) |
| Chest | 43 (12) |
| Breast | 38 (10.6) |
| Neuroradiology | 32 (9) |
| Abdomen | 30 (8.4) |
| Musculoskeletal | 29 (8.1) |
| Physics | 20 (5.6) |
| Others | 108 (30.3) |
| Study design | |
| Descriptive observational | 144 (40.3) |
| Analytical observational | 183 (51.3) |
| Interventional | 30 (8.4) |
| Type of paper | |
| Case report | 77 (21.6) |
| Research | 275 (77) |
| Review | 5 (1.4) |
| Nationality of the first author | |
| Iranian | 109 (30.5) |
| Chinese | 95 (26.6) |
| Others | 153 (42.9) |
| Nationality of the corresponding author | |
| Iranian | 109 (30.5) |
| Chinese | 94 (26.3) |
| Others | 154 (43.1) |
| Scopus Cite Score Index quartile of citing journals | |
| First | 21 (17.60) |
| Second | 43 (36.10) |
| Third | 33 (27.70) |
| Fourth | 22 (18.50) |
| H-index of citing journals | 56.14 ± 44.62 |
| Sample size | 146.13 ± 667.46 |
| Number of references | 21.75 ± 9.69 |
| Number of authors | 5.33 ± 2.30 |
| Page count | 7.20 ± 2.20 |
| Number of papers by the first author in Scopus | 42.94 ± 64.82 |
| Number of papers by the corresponding author in Scopus | 45.83 ± 59.91 |
| H-index of the first author in Scopus | 8.75 ± 9.24 |
| H-index of the corresponding author in Scopus | 9.50 ± 9.32 |
3.3. Outcome
In this study, the outcome of interest was the citation status of articles over a five-year period after publication. The citation status was categorized as a binary variable, represented by ‘yes’ or ‘no’, depending on the citation status of each individual paper.
3.4. Statistical Analysis
Descriptive statistics were used to summarize the distributions of the variables. The relationships between independent factors and citation status were evaluated using univariate tests, including chi-square test, Fisher’s exact test, t-test, or Mann-Whitney U test. Moreover, a multiple binary logistic regression analysis was performed to identify adjusted factors influencing the citation status. The citation status served as the dependent variable, while relevant characteristics of the studies were considered as independent variables. The significance level for the inclusion of variables in the model was set at P < 0.05. Data analysis was carried out using SPSS Version 25.0 (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp.).
3.5. Ethical Considerations
To ensure adherence to ethical standards, the authors collected data from the articles and recorded it in a Microsoft Excel file, ensuring that individual authors were not identified. The data was presented in an aggregated manner to maintain anonymity. This study obtained ethical approval under the code, IR.TUMS.IKHC.REC.1402.215.
4. Results
From 2018 to 2022, the IJR published 357 articles, with 120 receiving 321 citations from 233 distinct journals. Most articles were research papers (76.5%) with an observational analytical design (51.3%). The top three journals citing the IJR studies were the IJR itself, the Egyptian Journal of Radiology and Nuclear Medicine, and the Academic Radiology. Among the citing journals, the Cochrane Database of Systematic Reviews held the highest h-index. The majority of citing journals fell into the second quartile of the Scopus Cite Score Index. The citation rates varied across years, with physics topics being the most cited. Based on the results, the IJR had an impact factor of 0.9. The average h-index for the journals that cited the articles was 56.14. As for the first and corresponding authors, their average h-indices were 8.75 and 9.50, respectively. For detailed information, readers can refer to Table 1.
The univariate analysis demonstrated significant relationships between the citation status and several factors, including the study design (P = 0.03), scope of the paper's title (P < 0.04), type of paper (P = 0.04), number of years since publication (P < 0.001), number of papers by the first author in Scopus (P = 0.04), h-index of the first author in Scopus (P = 0.01), and h-index of the corresponding author in Scopus (P = 0.04). However, no significant associations were found between the citation status and paper characteristics, such as the nationality of the first and corresponding authors, sample size, number of references, number of authors, page count, and number of papers by the corresponding author in Scopus (P ≥ 0.05). Readers can refer to Table 2 for more detailed information.
Associations Between Citation Status and Paper Characteristics Based on the Univariate logistic Regression Analysis a
| Variables | Cited papers | Uncited papers | P-value |
|---|---|---|---|
| Study design | 0.03 | ||
| Descriptive observational | 38 (26.4) | 106 (73.6) | |
| Analytical observational | 68 (37.2) | 115 (62.8) | |
| Interventional | 14 (46.7) | 16 (53.3) | |
| Area covered by the paper’s title | 0.04 | ||
| Vascular & interventional | 19 (33.3) | 38 (66.7) | |
| Chest | 16 (37.2) | 27 (62.8) | |
| Breast | 15 (39.5) | 23 (60.5) | |
| Neuroradiology | 9 (28.1) | 23 (71.9) | |
| Abdomen | 8 (26.7) | 22 (73.3) | |
| Musculoskeletal | 12 (41.4) | 17 (58.6) | |
| Physics | 13 (65.0) | 7 (35.0) | |
| Others | 28 (25.9) | 80 (74.1) | |
| Type of paper | 0.02 b | ||
| Case report | 19 (24.7) | 58 (75.3) | |
| Research | 96 (34.9) | 179 (65.1) | |
| Review | 4 (80.0) | 1 (20.0) | |
| Number of years since publication | 3.84 ± 1.16 | 2.85 ± 1.42 | <0.001 c |
| Number of papers by the first author in Scopus | 52.75 ± 74.28 | 37.97 ± 59.00 | 0.04 |
| H-index of the first author in Scopus | 10.47 ± 9.49 | 7.88 ± 8.75 | 0.01 |
| H-index of the corresponding author in Scopus | 10.89 ± 8.87 | 8.80 ± 9.48 | 0.04 |
The multiple binary logistic regression analysis (Table 3) indicated that the model was statistically significant and accounted for 16.2% of the variance in the citation status. The number of years since publication had the strongest predictive power, followed by the scope of the paper's title and the type of article. Each year of publication increased the odds of citation by 0.54. The odds of citation for review papers was more than 33.29 times higher than that of case reports. Furthermore, papers with a title in the field of physics had over 1.39 times higher odds of being cited compared to papers on other topics.
The Results of Multiple Binary Logistic Regression Analysis for Identifying Adjusted Factors Influencing the Citation Status of Papers Published in the Iranian Journal of Radiology (IJR) a, b
| Variables | Regression coefficient (β) | Standard error (SE) | Exp β (95% CI) | P-value |
|---|---|---|---|---|
| Number of years since publication | 0.54 | 0.09 | 1.72 (1.42 - 2.09) | <0.001 |
| Type of paper (review vs. case report) | 3.50 | 1.24 | 33.29 (2.89 - 382.58) | <0.001 |
| Area covered by the paper’s title (physics vs. others) c | 1.39 | 0.55 | 4.02 (1.42 - 2.09) | 0.01 |
5. Discussion
This study focused on papers published in the IJR from 2018 to 2022. The results of the univariate analysis revealed that study design, scope of the paper's title, type of paper, number of years since publication, number of the first author’s papers in Scopus, and h-indices of the first and corresponding authors in Scopus were significantly associated with the citation status. However, after adjusting for confounding variables in the multiple logistic regression analysis, the number of years since publication emerged as the most influential predictor of citation status, followed by the scope of the paper's title and the type of paper.
The present study found that the citation rate of articles published in the IJR is comparable to that of other academic journals, with 33.3% of papers receiving citation on average (5, 12). Likewise, a study published in the Journal of Informetrics (JOI) found that the average citation rate for articles published in 70 social sciences journals was 33.5% (13), which is highly similar to the IJR's citation rate. Another study published in the Scientometrics found that the average citation rate for articles published in 28 psychology journals was 35%, which is again comparable to the citation rate of articles published in the IJR (14). These studies indicate that the citation rate of articles published in the IJR is neither unusual, nor particularly low when compared to other academic journals across various fields.
The present study also found that the study design, number of years since publication, type of paper, and author’s h-index are important factors in determining the citation status of studies. For instance, a study published in the Journal of PLOS Medicine found that the study design, type of study, and sample size were significant predictors of citation impact in the field of medical informatics (9). Additionally, a study published in PLOS One found that study design, type of paper, and sample size were significant predictors of citation impact in the field of ecology (15). These studies suggest that various factors, including the study design, number of years since publication, type of article, and authors’ characteristics, such as h-index, can significantly influence the citation impact in different academic fields (16, 17).
Interestingly, the present study found no significant association between the citation status and several paper characteristics, such as the nationality of the first and corresponding authors, sample size, number of references, number of authors, page count, and number of papers by the corresponding author in Scopus. In this regard, a study published in the Journal of the American Society for Information Science and Technology (JASIST) found that the nationality of the first author, number of references, and number of pages did not significantly affect the citation impact in the field of library and information sciences (18). Additionally, a study published in the JOI found that the number of authors, number of references, and page count did not significantly affect the citation impact in the field of social sciences (19). These studies suggest that certain paper characteristics, such as the authors’ nationality, sample size, number of references, number of authors, page count, and number of papers by the corresponding author, may not significantly influence the citation impact in different academic fields (20, 21). However, it is important to note that the impact of these factors may vary depending on the specific field of study and other contextual factors (22).
The finding suggesting that the number of years since publication is the strongest predictor of citation, followed by the scope of the paper's title and the type of paper, underscores the significance of considering the year of publication, title’s scope, and type of article when evaluating its citation impact. For instance, a study published in the JOI found that the publication year of a paper is the strongest predictor of its citation impact in the field of social sciences, followed by the journal’s impact factor (23). Another study published in the JASIST found that the publication year of a paper and the number of citations in the first year after publication were strong predictors of citation impact in the field of library and information sciences (24). Overall, these studies suggest that the publication year and the type of article are important factors in determining the citation impact of articles in various academic fields.
Our study has several limitations. Firstly, this study analyzed articles published within a relatively short timeframe, which may not accurately reflect the long-term citation status of the journal. Secondly, the study was limited by its use of citation as the sole measure of impact. Other metrics, such as altmetrics or social media metrics, could potentially offer additional insights into the journal’s impact and influence.
In conclusion, the present research scrutinized the influence of various factors on a paper’s citation status, including the study design, scope of the paper's title , type of paper, number of years since publication, number of papers by the first author in Scopus, and h-index of the authors. However, after controlling for confounding variables, the present findings identified only three parameters, namely the year of publication, type of paper, and scope of the paper's title , as significant predictors of citation status, according to the multiple logistic regression analysis. Such insights can guide authors and policymakers in improving the research visibility and impact in the IJR.
References
-
1.
Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-based medicine: how to practice and teach it. Elsevier Health Sciences; 2011.
-
2.
Garfield E. How ISI selects journals for coverage-quantitative and qualitative considerations. Curr Cont. 1990;22:5-13.
-
3.
Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):498-502. [PubMed ID: 9056804]. [PubMed Central ID: PMC2126010]. https://doi.org/10.1136/bmj.314.7079.497.
-
4.
About the Journal. 2023, [cited 2023 Jun 22]. Available from: https://brieflands.com/journals/ij-radiology.
-
5.
Ramezani A, Ramezani-Pakpour-Langeroudi F. [A comparative study of scientific production in the field of radiology in Iran and Middle Eastern countries within the years 1996-2012]. The 30th Iranian Radiology Congress 4-7 May 2014; Tehran; Iran. 2014. Persian.
-
6.
Bornmann L, Daniel H. What do citation counts measure? A review of studies on citing behavior. J Doc. 2008;64(1):45-80. https://doi.org/10.1108/00220410810844150.
-
7.
Liskiewicz T, Liskiewicz G, Paczesny J. Factors affecting the citations of papers in tribology journals. Scientometrics. 2021;126(4):3321-36. https://doi.org/10.1007/s11192-021-03870-w.
-
8.
Morisawa F, Nishizaki Y, Devos P, Yanagisawa N, Matsuyama K, Homma Y, et al. The association between research funding status and clinical research papers' citation impact in Japan: A cross-sectional bibliometric study. Front Med (Lausanne). 2022;9:978174. [PubMed ID: 36341255]. [PubMed Central ID: PMC9626813]. https://doi.org/10.3389/fmed.2022.978174.
-
9.
Ioannidis JP. How to make more published research true. PLoS Med. 2014;11(10). e1001747. [PubMed ID: 25334033]. [PubMed Central ID: PMC4204808]. https://doi.org/10.1371/journal.pmed.1001747.
-
10.
Moed HF. Citation analysis in research evaluation. 9. Springer Science & Business Media; 2006.
-
11.
Lopez J, Calotta N N, Doshi A, Soni A, Milton J, May Jr JW, et al. Citation Rate Predictors in the Plastic Surgery Literature. J Surg Educ. 2017;74(2):191-8. [PubMed ID: 27651051]. https://doi.org/10.1016/j.jsurg.2016.08.005.
-
12.
Yu H, Young M. The Impact of Web Search Engines on Subject Searching in OPAC. Inf Technol Libr. 2017;23(4):168-80. https://doi.org/10.6017/ital.v23i4.9658.
-
13.
Waltman L, van Eck NJ. A systematic empirical comparison of different approaches for normalizing citation impact indicators. J Informetr. 2013;7(4):833-49. https://doi.org/10.1016/j.joi.2013.08.002.
-
14.
Erfanmanesh A, Didegah F. A Comparison of Web of Science and Scopus for Iranian Publications and Citation Impact. Int J Inf Sci Manag. 2013;11(1):11-27.
-
15.
West JD, Jacquet J, King MM, Correll SJ, Bergstrom CT. The role of gender in scholarly authorship. PLoS One. 2013;8(7). e66212. [PubMed ID: 23894278]. [PubMed Central ID: PMC3718784]. https://doi.org/10.1371/journal.pone.0066212.
-
16.
Bornmann L, Leydesdorff L. The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000. J Informetr. 2013;7(2):286-91. https://doi.org/10.1016/j.joi.2012.12.003.
-
17.
Eysenbach G, Kohler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324(7337):573-7. [PubMed ID: 11884321]. [PubMed Central ID: PMC78994]. https://doi.org/10.1136/bmj.324.7337.573.
-
18.
Kousha K, Thelwall M. Google Scholar citations and Google Web/URL citations: A multi‐discipline exploratory analysis. J Am Soc Inf Sci Technol. 2007;58(7):1055-65. https://doi.org/10.1002/asi.20584.
-
19.
Bornmann L, Leydesdorff L. Does quality and content matter for citedness? A comparison with para-textual factors and over time. J Informetr. 2015;9(3):419-29. https://doi.org/10.1016/j.joi.2015.03.001.
-
20.
Torres-Salinas D, Cabezas-Clavijo Á, Jiménez-Contreras E. Altmetrics: New indicators for scientific communication in Web 2.0. Comunicar. 2013;21(41):53-60. https://doi.org/10.3916/c41-2013-05.
-
21.
Thelwall M, Wilson P. Regression for citation data: An evaluation of different methods. J Informetr. 2014;8(4):963-71. https://doi.org/10.1016/j.joi.2014.09.011.
-
22.
Waltman L, van Eck NJ, Noyons EC. A unified approach to mapping and clustering of bibliometric networks. J Informetr. 2010;4(4):629-35. https://doi.org/10.1016/j.joi.2010.07.002.
-
23.
Velastegui-Montoya A, Montalván-Burbano N, Peña-Villacreses G, de Lima A, Herrera-Franco G. Land Use and Land Cover in Tropical Forest: Global Research. Forests. 2022;13(10):1709. https://doi.org/10.3390/f13101709.
-
24.
Leydesdorff L, Bornmann L. Integrated impact indicators compared with impact factors: An alternative research design with policy implications. J Am Soc Inf Sci Technol. 2011;62(11):2133-46. https://doi.org/10.1002/asi.21609.