Dear Editor,
Due to the increase in artificial intelligence (AI) applications in various fields of healthcare (1), ChatGPT, as one of the branches of AI, is used for writing scientific texts. In addition to its capabilities and advantages, ChatGPT also faces certain challenges and limitations. The authors aimed to provide evidence in this letter.
ChatGPT, powered by the generative pre-trained transformer 3.5 architecture, was introduced by OpenAI on November 30, 2022 (2). The advent of OpenAI's large language models began with GPT-1 in 2017, followed by GPT-2 and GPT-3, and finally the well-known ChatGPT (3). ChatGPT's main purpose was to produce human-like text responding to user inputs. Using a vast dataset gathered from various sources, ChatGPT acquired extensive knowledge spanning a wide range of topics up until its last update in September 2021. Despite chatbots and conversational agents being utilized in the medical field, it was only after the introduction of ChatGPT that the potential of chatbots was fully revealed (4).
ChatGPT is a valuable tool for various tasks in the medical field, such as extracting information from medical records, aiding in literature searches by finding academic papers and extracting key findings from them, emphasizing areas of uncertainty, and providing suggestions for structure (5). Moreover, its capabilities are evident as it can achieve acceptable scores on the United States medical licensing exam (USMLE) (6). Additionally, ChatGPT can generate abstracts that, in certain cases, resemble those produced by humans, making it a valuable resource for medical professionals and researchers (7).
ChatGPT, similar to other chatbots, depends on machine learning, particularly deep learning and natural language processing, to produce texts that closely resemble human writing when generating responses to user inputs. While specific characteristics like a lack of creativity, style, and originality may suggest that a paper was written by a chatbot, it's essential to note that not all AI-generated content is easily distinguishable from human-authored articles (8, 9). As AI-generated content becomes more advanced, it might become progressively challenging for individuals to differentiate between text produced by AI and text authored by humans (10). It is noted that advancements in technology are currently enabling the conversion of AI-generated text into human writing. Therefore, in the near future, it may become increasingly difficult for humans to distinguish between AI-generated and human-written text.
Gao et al. conducted a study where they presented a set of abstracts, 50 original and 50 generated by ChatGPT based on original abstracts, to reviewers and AI output detectors. After evaluation, the AI output detector effectively distinguished the generated abstracts, with high scores indicating AI generation, from the original abstracts with low probabilities of being AI-generated. However, when human reviewers were blinded to the source, they correctly identified 68% of the generated abstracts but mistakenly identified 14% of the original abstracts as AI-generated (9). The results of another related study concluded that detecting AI-based articles is challenging, and it often requires a combination of techniques, including language analysis, plagiarism detection, and using AI-powered tools (11).
ChatGPT's reliability and independent capability in scientific writing (SW) are questionable. One of the most concerning issues is that it tends to generate references that may not be legitimate. In a study conducted by Eysenbach, an interview was conducted with ChatGPT, where a series of questions were asked in the field of medicine, medical practice, and medical research, and the primary concern was that ChatGPT tends to invent references (12). In another study, 178 references listed on ChatGPT were analyzed. The analysis revealed that 69 of these references did not have a digital object identifier (DOI), and 28 references were inaccessible via Google search and lacked a DOI (13).
One limitation of ChatGPT is its tendency to generate responses that appear credible but are incorrect, which is called "artificial hallucination". Artificial hallucination is the term used to describe the phenomenon where a machine, like a chatbot, generates sensory experiences that appear realistic despite lacking any connection to real-world input. Language models like ChatGPT can generate impressive and appropriate responses, but they may sometimes generate content that is entirely made up and inaccurate (14).
Another challenge is plagiarism. ChatGPT's training involves a vast amount of text data, and it generates responses based on statistical probabilities and examples it has seen during its training. If a user inputs a question that is similar to content available on the internet, ChatGPT may produce a response that closely resembles existing content. ChatGPT rephrases or paraphrases information, and it may reproduce content from different sources and unintentionally result in plagiarism. As a result, there is a possibility that ChatGPT might use similar phrases to the text it has learned (15). In a study conducted by Khalil and Er, 50 topics were given to ChatGPT to generate 500-word articles. Afterward, 2 plagiarism detection software were used to measure the percentage of plagiarism. The results showed that out of 50 articles, 40 of them had originality, with the generated text by ChatGPT having 20% or less similarity (16).
ChatGPT can be a valuable tool in SW. It can assist in various aspects, such as selecting appropriate study topics, providing key terms, introducing databases, and summarizing articles. These tasks can be time-consuming, but with the use of AI, high-quality articles can be produced in a shorter time frame. In addition, ChatGPT can be beneficial for non-native English speakers in terms of finding appropriate vocabulary and assisting with grammar and sentence structure. With its high translation capabilities, it can also translate written text into English (15). While ChatGPT can assist in improving the content and structure of articles, it cannot be replaced by our deep understanding of SW. Text generated by ChatGPT should be carefully reviewed and analyzed to ensure its accuracy and reliability (7). Considering that ChatGPT may provide a mixture of accurate and fabricated information, revising the policies and practices for evaluating scientific manuscripts submitted to journals and medical conferences to improve scientific standards can be helpful (14).
Regarding the issue of plagiarism resulting from the use of ChatGPT, several solutions can be implemented. We can complement the text generated by ChatGPT with information from other sources. Properly citing the mentioned content in the article using accurate citations is also essential. Using plagiarism detection tools can be helpful, but ultimately, nothing can replace the importance of thorough review and the author (15, 17).
