ChatGPT, a Friend or a Foe in Medical Education: A Review of Strengths, Challenges, and Opportunities

authors:

avatar Mahdi Zarei ORCID 1 , avatar Maryam Zarei ORCID 2 , avatar Sina Hamzehzadeh ORCID 3 , avatar Sepehr Shakeri Bavil Oliyaei ORCID 3 , avatar Mohammad-Salar Hosseini ORCID 4 , *

Research Center for Evidence-Based Medicine, Tabriz University of Medical Sciences, Tabriz, Iran
Faculty of Computer Engineering, Khaje Nasir Toosi University of Technology, Tehran, Iran
Student Research Committee, Tabriz University of Medical Sciences, Tabriz, Iran
Research Center for Integrative Medicine in Aging, Aging Research Institute, Tabriz University of Medical Sciences, Tabriz, Iran
Shiraz E-Medical Journal: Vol.25, issue 6; e145840
published online: June 1, 2024
article type: Review Article
received: February 14, 2024
revised: March 14, 2024
accepted: April 16, 2024
DOI: https://doi.org/10.5812/semj-145840

how to cite: Zarei M, Zarei M, Hamzehzadeh S, Shakeri Bavil Oliyaei S, Hosseini M. ChatGPT, a Friend or a Foe in Medical Education: A Review of Strengths, Challenges, and Opportunities. Shiraz E-Med J. 2024;25(6):e145840. https://doi.org/10.5812/semj-145840.

Abstract

Background:

ChatGPT is a large-scale language model that utilizes artificial intelligence (AI) to answer a broad range of scientific inquiries, create clinical scenarios, and evaluate educational programs. While its use offers numerous advantages, it also presents several challenges.

Objectives:

The aim of this study is to conduct a comprehensive review of ChatGPT's functionality and explore the benefits, challenges, existing solutions, and future prospects of using AI in medical education.

Method:

A comprehensive literature review was conducted using PubMed, Scopus, Web of Science, and Google Scholar. The search phrases used were ChatGPT, Artificial Intelligence, Chatbot, Medical Education, and large language models (LLMs).

Results:

The application of ChatGPT in medical education offers several advantages, such as enhanced quality of interaction between medical students and patients, improved education quality, enhanced research opportunities, personalized learning, virtual patient simulations, and cost-effectiveness. However, there are also critical challenges, such as ethical and transparency concerns, limited access to reliable databases, restricted information availability after 2021, limited development of students' critical thinking ability, and the risk of generating AI hallucinations.

Conclusions:

Artificial intelligence models have become a popular tool for researchers to access scientific resources, comprehend articles and textbooks, and create scientific texts. ChatGPT has been used extensively in medicine and medical education in a short period. It is essential to weigh the benefits and challenges, use expert supervision, conduct frequent assessments, and provide feedback reviews to guarantee its efficacy. Although this technology should not replace human labor, it is essential to prepare for the changes brought by AI and create appropriate guidelines and curricula by reviewing existing solutions and conducting extensive studies.

1. Introduction

Artificial intelligence (AI) is an interdisciplinary approach that combines computer science and linguistics. The objective of AI is to create machines that can replicate and perform tasks that humans do using their intelligence (1). The creation and advancement of computers are undeniably among the greatest accomplishments of the 20th century, rivaling or even surpassing the impact of the printing press. Computer experts credit the development of AI to three key principles: The first being a theoretical concept that led to the creation of a basic mathematical model known as the Turing machine, which serves as a universal computing machine. The second principle is practical, resulting in the construction of one of the earliest electronic programmable digital computers. Lastly, the third principle involves a philosophical and operational definition of the concept of thinking. According to experts, Alan Mathison Turing played a significant role in all three of these principles, and it is undeniable that his efforts have paved the way for the future invention and advancement of AI (2, 3).

The term "Artificial Intelligence" was coined by John McCarthy in 1955. The origins of AI can be traced back to the Dartmouth summer research project in the mid-20th century (4), which was later expanded upon by the development of machine learning algorithms (5). AI is capable of understanding words, visuals, and material in addition to learning, summarizing, analyzing, and adapting to various circumstances. It can even carry out intricate human functions like creativity, emotion, and attentiveness (6, 7).

With advancements in technology and AI models, researchers have started utilizing these models for easy access to scientific resources, comprehension of key points in articles or textbooks, and scientific writing (8, 9). In recent years, chatbots or conversational agents have found applications in the field of medicine (10, 11). Some notable examples include Jasper chat, DialoGPT, Replica, and ChatGPT. ChatGPT, developed by OpenAI, is a large-scale language model that was introduced in November 2022 (12).

In a short period of time, ChatGPT has proven to be useful in various industries such as technology, banking, marketing, entertainment, engineering, medicine, and medical education. It possesses the ability to answer questions of different complexities and dimensions by analyzing large amounts of data and thinking critically. Moreover, it can accurately respond in multiple languages, making it a popular choice for users. As a result, the number of ChatGPT users reached 57 million within a month and reached 100 million by January 2023 (10).

Medical education is one area where ChatGPT has been applied. Studies have shown its effectiveness in performing writing and conversational tasks (13, 14), small group learning, clinical scenario design, exam administration, personalized learning promotion, and curriculum development (15, 16). However, ethical concerns, transparency, privacy issues, and potential negative effects on problem-solving and critical thinking abilities have been raised as challenges (17-19).

Given the increasing use of AI models in medical education and the scientific interest in ChatGPT (20), this study aims to examine the performance of ChatGPT, its application in medical education, the benefits and challenges of its use, potential solutions to improve it, and the future prospects of using AI models in medical education.

2. Method

In conducting this review study, the authors searched PubMed, Scopus, Web of Science, and Google Scholar for studies related to the functioning of large language models (LLMs), the application of ChatGPT and other chatbots in medicine and medical education, the assessment of the challenges and advantages of using these models, and articles related to their enhancement in medical education. Relevant articles were reviewed. The keywords used for the search were ChatGPT, Chatbot, Artificial Intelligence, Medical Education, and LLMs. The articles were initially reviewed by two of the authors, and in case of disagreement, the articles were reviewed by a third person, and any discrepancies were resolved through discussion among the authors. After evaluating and eliminating irrelevant studies, the content of the articles related to the study topic was summarized, compared, and examined.

3. Results

3.1. How Do Large Language Models Work?

Large language models like the generative pre-trained transformer (GPT-3) have made significant advances in recent years (21). These models are first trained on a large volume of data and have the ability to answer questions from users in various languages, translate texts and generate texts similar to natural language. These models can also perform other tasks related to natural language processing with high accuracy (22).

The use of transformer architectures and underlying attention mechanisms in this field has led to significant progress (23, 24). The self-attention mechanism has improved the relationship between input data when preparing responses to users (24). This allows the models to better understand the relationship between words in sentences and provide more appropriate responses based on this understanding. Training these models with a large volume of available data before using them enables better search and better response generation (25).

ChatGPT is a language model capable of answering questions on a wide range of topics with varying degrees of complexity, thanks to its extensive dataset. It can mimic human language by analyzing vast amounts of internet data, which it can use to perform tasks such as translation, summarization, storytelling, and entertainment (13, 26). Furthermore, it has a strong grasp of sentence meaning and can answer questions using critical thinking (26). ChatGPT is essentially a chatbot that can engage in text-based communication by analyzing and understanding texts and words. It is based on the generative pre-trained transformer (GPT) (13, 27), which utilizes neural network algorithms to evaluate input data and provide logical answers (27). The most recent iteration of ChatGPT boasts 100 trillion parameters that have undergone training on a substantial corpus of internet or user-provided data to carry out the aforementioned tasks (28).

3.2. Application of ChatGPT in Medical Education

The field of medical education has transitioned from traditional methods to those that focus on critical thinking and problem-based learning (PBL). Hybrid educational approaches have been suggested, leading to changes in the academic curriculum of certain fields. The use of AI has played a significant role in driving this shift (29).

It is worth mentioning that ChatGPT can respond to 81.3% of the residency entrance exam queries in Iran. Moreover, when these queries are translated into English, French, and Spanish, this chatbot can correctly answer six, one, and five more questions respectively (28). Additionally, ChatGPT successfully answered a total of 2210 questions, encompassing basic sciences, pre-internship, and residency entrance exam queries, achieving an overall accuracy of 48.5% and an overall concordance of 91% (30).

In the era of big data, where AI is extensively used (31), having continuous and 24/7 access to educational information is crucial for students. However, many countries have not made sufficient improvements in their educational methods and approaches despite the growing number of students in health-related fields (32). To address these challenges, various AI models, such as ChatGPT, have been employed in medical education (33, 34).

The utilization of AI, specifically ChatGPT, in medical education comes with both benefits and obstacles. It is important to acknowledge and examine these advantages and challenges, explore potential solutions, and assess how AI can enhance current educational methods. While AI can serve as a valuable tool in various aspects such as curriculum development, evaluation methods design, text summarization, key point extraction, and clinical scenario design, it should not completely replace the effectiveness of human resources and in-person training, particularly in the field of medicine and medical education (35). Thus, it is crucial to consider the advantages and challenges associated with incorporating ChatGPT in medical education.

In the following paragraphs, the advantages and challenges surrounding the use of ChatGPT in medical education are briefly discussed.

3.3. Benefits

3.3.1. Improving Personalized Learning

ChatGPT has the ability to create a customized study plan based on the student's individual learning style and abilities. It can identify the necessary educational content and suggest appropriate textbooks. The AI model can also generate quizzes according to the student's learning level and provide feedback to help them overcome their weaknesses (10, 16, 31, 36). ChatGPT is particularly useful for evaluating student progress and providing quick feedback to facilitate the achievement of educational goals (37). Additionally, it can free up time for teachers to engage in face-to-face communication with their students and answer specific scientific questions. ChatGPT can also solve complex questions and present them in an easily understandable format, including conceptual maps in the form of text (32, 38, 39).

3.3.2. Enhancing the Condition of Scientific Research

Improving medical education is greatly aided by research, which enables the teaching of new subjects and enhances problem-solving abilities to tackle new challenges. This leads to a better comprehension of complex medical issues and creates an interactive learning environment for students, supervised by faculty members, thereby expediting the attainment of educational objectives. ChatGPT is a valuable tool that can summarize literature, extract key points from articles, and assist researchers in designing research methods. It can also aid professors and students in preparing references and the original format or outline of articles, serving as a foundation for promoting research and enhancing the quality of medical education (36).

3.3.3. Designing Clinical Scenarios

It is feasible to use ChatGPT for creating clinical scenarios and imparting clinical reasoning and problem-solving skills to students. This approach helps students better handle real-life patients in clinical settings and facilitates active, student-centered learning through interactive environments such as problem-based and small group learning (15, 16, 40). The clinical scenarios designed using ChatGPT may cover history, clinical symptoms, laboratory results, and imaging techniques (10, 15).

3.3.4. Enhancing the Quality of Interaction Between Medical Students and the Patients

With its high accuracy, ChatGPT offers the option of translating into various languages. This feature enhances the interaction between students and patients from diverse cultures in clinical settings while also lightening the educational burden on clinical departments (31).

3.3.5. Providing Virtual Patient Simulation

ChatGPT has the capability to offer virtual patient simulation through the creation of clinical scenarios and case presentations, which can enhance the quality of education. This chatbot can provide a safe simulation environment that helps medical students improve their problem-solving skills, decision-making abilities, and clinical reasoning without putting actual patients at risk. Additionally, it can assist in improving communication skills with patients, which is essential in history taking and disease diagnosis for medical students (16, 41).

3.3.6. Help with Designing Curriculum

With the assistance and oversight of medical education professionals, ChatGPT has the ability to aid in the creation of educational curricula for various fields. It serves as a helpful resource in crafting the course structure, identifying assessment criteria, establishing learning objectives and techniques, and determining the desired outcomes. Due to the sensitive nature of curriculum design, the use of ChatGPT can be beneficial only if it is constantly monitored and all suggestions are executed under the supervision of an expert (42).

3.3.7. Cost-Effectiveness of Using ChatGPT

The utilization of ChatGPT in medical education has the potential to enhance teaching methods and address academic and scientific issues faced by students in a more efficient and economical manner. This implies a more cost-effective approach (43). For instance, ChatGPT can be employed to simulate dental clinical environments and facilitate self-learning at a reduced cost. Similarly, it can aid in teaching statistical analysis techniques and offer the opportunity for extensive assignment reviews, thus providing a cost-effective learning experience for students in public health-related fields (44, 45).

3.4. Challenges and Risks

The utilization of AI models has been on the rise in the medical field, aiding in the diagnosis and treatment process, as well as in medical education. A quick literature search confirms the increase in related articles. However, it is crucial to address the challenges that come with expanding the use of AI in medical education. ChatGPT is one such AI model, and understanding its challenges can lead to its effective application in medicine and medical education. Failure to identify and manage these challenges could negate the benefits of ChatGPT in medical education and result in losses. Some experts have even rejected its use due to its provision of unnecessary and illogical information (12). The following section briefly discusses the challenges associated with using ChatGPT in medical education.

3.4.1. Limiting Students' Critical Thinking Ability

According to certain professionals in the field of education and research, the extensive use of ChatGPT in educational settings may prevent students from introducing fresh concepts (12, 13). Over time, chatbots can curtail critical thinking and analytical skills in students by rapidly delivering information without engaging their cognitive processes. This could hinder their ability to make sound decisions and take prompt action, particularly during urgent situations.

3.4.2. The Inability of ChatGPT to Reach Certain Legitimate Databases

The lack of access to databases like PubMed, Cochrane, and UpToDate is a significant drawback of ChatGPT that hinders its effectiveness in the field of medicine and medical education (12). These databases contain a vast amount of medical information, and the unavailability of this information reduces the accuracy of ChatGPT's responses and prevents it from staying current. Staying up-to-date with the latest guidelines and evidence is crucial in medicine due to the high volume of information flow. Therefore, the lack of access to these databases is a significant challenge that needs to be addressed. For example, in research carried out by Farajollahi and his team, it was found that ChatGPT was unable to attain a passing score in the national board exam for endodontists in Iran, despite answering 40 out of 100 questions. This could be due to restricted access to scientific resources for the endodontics board exam (46).

3.4.3. Restriction on Accessing Information Published After 2021

The limited access to information and scientific content published after 2021 makes ChatGPT unsuitable as a guide for clinical decisions and teaching new scientific materials. Its use in such cases can result in erroneous choices or incorrect dissemination of new scientific information, ultimately reducing its utility (47).

3.4.4. The Challenge Surrounding Fraud and the Possibility of Plagiarism

The use of ChatGPT has the potential to enhance the standard of education, but some experts are apprehensive about its extensive use in medical education, as it may lead to an increase in cheating and plagiarism (48-50). Research indicates that ChatGPT can respond to a significant number of questions in the USMLE and perform like a third-year medical student (16). In a study carried out by Kung et al., ChatGPT demonstrated the ability to provide accurate responses to 45.4%, 54.1%, and 61.5% of Step 1, Step 2, and Step 3 queries in the USMLE, resulting in a successful outcome (43). Similarly, in a study conducted on the Chinese Medical Licensing Exam, ChatGPT demonstrated an accuracy rate of 36.7% in answering a dataset of 12,723 questions (51). While there are methods to evaluate the precision of assessment outcomes, concerns over this issue may have a negative impact on the quality of education.

3.4.5. Hallucination

Hallucination, which occurs when AI produces responses that deviate from its training data, is a significant issue in LLM (52-54). For instance, the generation of non-existent or incorrectly titled DOI articles in ChatGPT exemplifies this problem (9, 10). Such hallucinations can introduce bias and inaccuracies, posing obstacles to achieving the educational objectives outlined in the curriculum. Therefore, it is crucial to have expert oversight in monitoring the performance of ChatGPT.

3.4.6. Ethical Issues and Privacy

Ethical concerns and privacy of individuals may be compromised when using ChatGPT, as it involves the use of personal data of patients and students, particularly in case presentations and PBL (55, 56). These concerns are further magnified by the risk of cyber attacks. Additionally, ethical issues are raised with the potential for bias in ChatGPT results or discrimination based on the training data utilized. These concerns have been addressed in various studies (57-60).

3.4.7. Transparency

Transparency is a major challenge faced by ChatGPT in the field of medicine and medical education, particularly with regard to the disclosure of results and information. The Blackbox algorithm has been a cause for concern as it is not possible for users to understand or observe the method of presenting results (61). This lack of transparency can undermine the accuracy of the data provided by ChatGPT, but experts in AI argue that the Blackbox is necessary to safeguard privacy (13).

3.4.8. Limited Chat Number in Different Versions of ChatGPT

There are two iterations of ChatGPT: A complimentary edition employing model version 3.5 and a premium edition currently utilizing model version 4.0 (62). A significant obstacle is the utilization of ChatGPT for enhancing medical education in low-income and developing countries, as it restricts the number of chats available in both the free and premium versions. Given the advancements in medicine and medical education in Iran and the growing demand for integrating AI tools in this field, the economic sanctions pose a substantial challenge to the expansion of medical education and healthcare systems in Iran (63).

3.4.9. Inability to Generate a Suitable Emotional Conversation

The connection between teachers and students and the development of emotional intelligence in medical students are crucial for improving medical education and meeting educational objectives. Enhancing students' emotional intelligence can aid in their future decision-making and improve their interactions with patients (64). A primary obstacle in using ChatGPT for medical education is its limited capability to comprehend and generate emotional dialogues. While there have been advancements in emotional dialogue technology since 2017, ChatGPT still has room for improvement in understanding and generating emotional conversations (65).

3.4.10. Psychological, Economic, and Social Challenges

The utilization of ChatGPT and other AI models can offer cost-effective facilities and services without requiring specialized personnel, leading to potential social and economic impacts (66). However, it is important to acknowledge that while ChatGPT can decrease the necessity for skilled employees in certain contexts, expert oversight is crucial for ensuring optimal performance, and human talent cannot be completely substituted (Figure 1) (35).

Strengths and challenges of application of ChatGPT in medical education
Strengths and challenges of application of ChatGPT in medical education
Figure 1.

Strengths and challenges of application of ChatGPT in medical education

3.5. How to Improve the ChatGPT Application

ChatGPT offers numerous benefits in the field of medical education, but it also presents certain difficulties. To enhance the performance and precision of the outcomes and information derived from these models, it is essential for medical students and professionals to acquaint themselves with AI-based models and recent advancements in computer science and AI. This will enable them to fully utilize the capabilities of these models.

(a) Extensive interdisciplinary research is necessary for the utilization of AI, including ChatGPT, in medical education. It is crucial to organize scientific gatherings that concentrate on the implementation of AI in medical education, with the participation of computer specialists, experts in AI, and medical educators. This approach has been adopted by renowned universities in the past (67, 68).

(b) Medical education experts should regularly monitor the various aspects of using ChatGPT and provide feedback to educational authorities to assess its effectiveness (32). These aspects include the students' experience with ChatGPT in the learning environment, the extent to which educational goals are achieved, and the efficiency of the educational strategy (43). Furthermore, after collecting and summarizing feedback from medical education experts and students, they can be provided to the AI companies to reduce the knowledge gap and enhance the development of these chatbots in the field of medical education more than ever.

(c) In order to address the forthcoming challenges, it is advisable to promptly establish a set of principles for utilizing ChatGPT in the field of medical education, while considering the factors of responsibility, transparency, and effectiveness (69).

(d) When utilizing ChatGPT, it is crucial to give careful consideration to ethical concerns associated with it. It is important to address matters like potential bias, plagiarism, data security, safeguarding individuals' privacy, and the risk of discrimination based on input data. Creating ethical guidelines specifically for the use of ChatGPT in medical education can be highly beneficial (70).

(e) Special emphasis should be given to teaching evidence-based medicine (EBM) principles, logical thinking, and clinical reasoning in order to encourage the use of AI in medical education. This will enable students to effectively deal with major crises and assess the validity of AI results (71, 72).

(f) Educating students on the functioning of different AI models, statistical techniques, and particularly ChatGPT, can familiarize them with the implementation of other AI models across diverse domains and enable them to engage with AI-based educational approaches like ChatGPT. Organizing interdepartmental discussions and practical classes for students can prove beneficial in achieving this goal (68, 73, 74).

(g) To incorporate the use of AI in medical education, it is important to develop and revise educational curricula, integrate AI models into teaching strategies, create new evaluation methods that emphasize reasoning and logical thinking, update educational practices, and establish educational environments that are compatible with AI technology. These efforts can help structure the use of AI in medical education and facilitate its implementation (74-76).

(h) There are several technical engineering methods to boost the performance of LLMs, which can improve their effectiveness and efficiency in various fields, including medicine and medical education. For instance, increasing the pre-training data can enhance the performance of LLMs in addressing various questions and topics, but this may present challenges such as longer pre-training times and memory constraints. The use of parameter-reduction techniques can decrease memory usage and enhance the speed of language-related tasks (77). Moreover, employing few-shot learning and pathways machine learning systems can result in improvements in language-related tasks like translation or code generation (78). Additionally, LLMs can enhance their performance by retaining internal representations of past inputs and rapidly acquiring new knowledge (79). Lastly, fine-tuning LLMs using self-generated solutions from unlabeled datasets can improve their reasoning capability and achieve advanced performance (Figure 2) (80).

Improving the application of ChatGPT in medical education
Improving the application of ChatGPT in medical education
Figure 2.

Improving the application of ChatGPT in medical education

3.6. The Horizon Ahead

The World Health Organization considers digital technologies crucial for achieving health objectives. By utilizing these resources, an additional one billion individuals can access health services, receive emergency care, and experience better overall health (81). Artificial intelligence is extensively employed in numerous medical fields (11), including radiology, ophthalmology, surgery, rheumatology, internal medicine, pathology, and orthopedics, as stated earlier.

The increased utilization of AI products in the medical field has resulted in a twofold rise in the necessity for incorporating these modalities into medical education. Numerous studies have been conducted on this subject, and many universities worldwide have implemented a combination of conventional teaching methods with AI and hybrid educational techniques (10, 67, 68, 73, 82). This is further supported by the significant number of articles published on this topic in recent years (83).

The integration of AI models in medical education is becoming more anticipated as technology and AI continue to advance. For example, in June 2023, Apple introduced the Vision Pro during the Worldwide Developers Conference. This mixed-reality headset integrates Virtual Reality and Video See-Through features to serve as an Augmented Reality device. The Vision Pro has great potential in effectively addressing medical challenges (84), especially in advancing medical education.

Finally, as digital technologies improve and become more affordable, students are becoming more familiar with these tools. Additionally, faculty members are providing guidance and support to students in utilizing these technologies effectively (85). Consequently, the use of AI in medical education is expected to grow steadily.

3.7. Study Limitation

The present review offers a comprehensive perspective on the application of ChatGPT in medical education, including its advantages and challenges. However, it is important to acknowledge certain limitations. The study is constrained by its nascent status and the scarcity of research in this area. Although existing literature has been compiled, the number of studies is relatively small, limiting its applicability. Furthermore, the long-term effects of using AI models, particularly ChatGPT, in medical education remain unclear. Despite these limitations, this review provides valuable information for educators to encourage the use and assessment of AI in medical education.

4. Conclusions

Artificial intelligence models have become popular tools for researchers to access scientific resources, comprehend articles and textbooks, and create scientific texts. These models possess various abilities, such as summarizing texts, critical thinking, logical reasoning, and compiling scientific documents. However, using ChatGPT in medical education comes with challenges that can reduce its efficiency. It is essential to weigh the benefits and challenges, use expert supervision, conduct frequent assessments, and provide feedback reviews to ensure its efficacy. Although this technology should not replace human labor, it is crucial to prepare for the changes brought by AI and create appropriate guidelines and curricula by reviewing existing solutions and conducting extensive studies.

Acknowledgements

References

  • 1.

    Sarker IH. AI-Based Modeling: Techniques, Applications and Research Issues Towards Automation, Intelligent and Smart Systems. SN Comput Sci. 2022;3(2):158. [PubMed ID: 35194580]. [PubMed Central ID: PMC8830986]. https://doi.org/10.1007/s42979-022-01043-x.

  • 2.

    Pinar Saygin A, Cicekli I, Akman V. The Turing Test: the first 50 years. Trends Cogn Sci. 2000;10(4):463-518. https://doi.org/10.1023/a:1011288000451.

  • 3.

    Pinar Saygin A, Cicekli I, Akman V. Turing test: 50 years later. Minds Machines. 2000;10(4):463-518. [PubMed ID: 10689346]. https://doi.org/10.1016/s1364-6613(00)01453-4.

  • 4.

    McCarthy J, Minsky ML, Rochester N, Shannon CE. A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955. AI Magazine. 2006;27(4):12.

  • 5.

    Jordan MI, Mitchell TM. Machine learning: Trends, perspectives, and prospects. Sci. 2015;349(6245):255-60. [PubMed ID: 26185243]. https://doi.org/10.1126/science.aaa8415.

  • 6.

    Korteling JEH, van de Boer-Visschedijk GC, Blankendaal RAM, Boonekamp RC, Eikelboom AR. Human- versus Artificial Intelligence. Front Artif Intell. 2021;4:622364. [PubMed ID: 33981990]. [PubMed Central ID: PMC8108480]. https://doi.org/10.3389/frai.2021.622364.

  • 7.

    Lee J, Wu AS, Li D, Kulasegaram KM. Artificial Intelligence in Undergraduate Medical Education: A Scoping Review. Acad Med. 2021;96(11S):S62-70. [PubMed ID: 34348374]. https://doi.org/10.1097/ACM.0000000000004291.

  • 8.

    Zarei M, Eftekhari Mamaghani H, Abbasi A, Hosseini M. Application of artificial intelligence in medical education: A review of benefits, challenges, and solutions. Med Clín Práct. 2024;7(2):100422. https://doi.org/10.1016/j.mcpsp.2023.100422.

  • 9.

    SeyedAlinaghi S, Abbaspour F, Mehraeen E. The Challenges of ChatGPT in Healthcare Scientific Writing. Shiraz E-Med J. 2023;25(2). https://doi.org/10.5812/semj-141861.

  • 10.

    Eysenbach G. The Role of ChatGPT, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation With ChatGPT and a Call for Papers. JMIR Med Educ. 2023;9. e46885. [PubMed ID: 36863937]. [PubMed Central ID: PMC10028514]. https://doi.org/10.2196/46885.

  • 11.

    Zarei M, Asheghi M, Zarei M. Artificial Intelligence in Dementia Diagnosis: Past, Present, and Future. Int J Aging. 2023;1. e14. https://doi.org/10.34172/ija.2023.e14.

  • 12.

    Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: Is ChatGPT a blessing or blight in disguise? Med Educ Online. 2023;28(1):2181052. [PubMed ID: 36809073]. [PubMed Central ID: PMC9946299]. https://doi.org/10.1080/10872981.2023.2181052.

  • 13.

    Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare (Basel). 2023;11(6):887. [PubMed ID: 36981544]. [PubMed Central ID: PMC10048148]. https://doi.org/10.3390/healthcare11060887.

  • 14.

    Deng J, Lin Y. The Benefits and Challenges of ChatGPT: An Overview. Frontiers Comput Int Syst. 2023;2(2):81-3. https://doi.org/10.54097/fcis.v2i2.4465.

  • 15.

    Jin Q, Dhingra B, Liu Z, Cohen W, Lu X. PubMedQA: A Dataset for Biomedical Research Question Answering. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China. 2019:2567-77. https://doi.org/10.18653/v1/D19-1259.

  • 16.

    Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med Educ. 2023;9. e45312. [PubMed ID: 36753318]. [PubMed Central ID: PMC9947764]. https://doi.org/10.2196/45312.

  • 17.

    Stokel-Walker C, Van Noorden R. What ChatGPT and generative AI mean for science. Nature. 2023;614(7947):214-6. [PubMed ID: 36747115]. https://doi.org/10.1038/d41586-023-00340-6.

  • 18.

    Biswas S. ChatGPT and the Future of Medical Writing. Radiol. 2023;307(2). e223312. [PubMed ID: 36728748]. https://doi.org/10.1148/radiol.223312.

  • 19.

    D'Amico RS, White TG, Shah HA, Langer DJ. I Asked a ChatGPT to Write an Editorial About How We Can Incorporate Chatbots Into Neurosurgical Research and Patient Care…. Neurosurgery. 2023;92(4):663-4. [PubMed ID: 36757199]. https://doi.org/10.1227/neu.0000000000002414.

  • 20.

    Wogu IAP, Olu-Owolabi FE, Assibong PA, Agoha BC, Sholarin M, Elegbeleye A, et al. Artificial intelligence, alienation and ontological problems of other minds: A critical investigation into the future of man and machines.In: Wogu IAP, Olu-Owolabi FE, Assibong PA, Agoha BC, Sholarin M, Elegbeleye A, et al., editors. International Conference on Computing Networking and Informatics (ICCNI). Cairo, Egypt. International Conference on Computing Networking and Informatics (ICCNI); 2017. p. 1-10.

  • 21.

    Floridi L, Chiriatti M. GPT-3: Its Nature, Scope, Limits, and Consequences. Minds Machines. 2020;30(4):681-94. https://doi.org/10.1007/s11023-020-09548-1.

  • 22.

    Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Ind Diff. 2023;103:102274. https://doi.org/10.1016/j.lindif.2023.102274.

  • 23.

    Devlin J, Chang M, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv. 2018;preprint.

  • 24.

    Becker BA, Denny P, Finnie-Ansley J, Luxton-Reilly A, Prather J, Santos EA. Programming Is Hard - Or at Least It Used to Be.In: Becker BA, Denny P, Finnie-Ansley J, Luxton-Reilly A, Prather J, Santos EA, editors. Proceedings of the 54th ACM Technical Symposium on Computer Science Education. Toronto, Ontario. Proceedings of the 54th ACM Technical Symposium on Computer Science Education; 2023. p. 500-6.

  • 25.

    Min B, Ross H, Sulem E, Veyseh APB, Nguyen TH, Sainz O, et al. Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys. 2023;56(2):1-40. https://doi.org/10.1145/3605943.

  • 26.

    Zhai X. ChatGPT User Experience: Implications for Education. SSRN Electronic J. 2022;Preprint. https://doi.org/10.2139/ssrn.4312418.

  • 27.

    Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al. Language models are few-shot learners. Adv Neural Inf Proces Syst. 2020;33:1877-901.

  • 28.

    Khorshidi H, Mohammadi A, Yousem DM, Abolghasemi J, Ansari G, Mirza-Aghazadeh-Attari M, et al. Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examination. Inf Med Unlocked. 2023;41:101314. https://doi.org/10.1016/j.imu.2023.101314.

  • 29.

    Sallam M, Salim NA, Al-Tammemi AB, Barakat M, Fayyad D, Hallit S, et al. ChatGPT Output Regarding Compulsory Vaccination and COVID-19 Vaccine Conspiracy: A Descriptive Study at the Outset of a Paradigm Shift in Online Search for Information. Cureus. 2023. https://doi.org/10.7759/cureus.35029.

  • 30.

    Keshtkar A, Hayat A, Atighi F, Ayare N, Keshtkar M, Yazdanpanahi P, et al. ChatGPT's Performance on Iran's Medical Licensing Exams. Research Square. 2023;Priprint. https://doi.org/10.21203/rs.3.rs-3253417/v1.

  • 31.

    Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT - Reshaping medical education and clinical management. Pak J Med Sci. 2023;39(2):605-7. [PubMed ID: 36950398]. [PubMed Central ID: PMC10025693]. https://doi.org/10.12669/pjms.39.2.7653.

  • 32.

    Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: A descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1). e103. [PubMed ID: 38450035]. [PubMed Central ID: PMC10914078]. https://doi.org/10.52225/narra.v3i1.103.

  • 33.

    Kyaw BM, Posadzki P, Paddock S, Car J, Campbell J, Tudor Car L. Effectiveness of Digital Education on Communication Skills Among Medical Students: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(8). e12967. [PubMed ID: 31456579]. [PubMed Central ID: PMC6764329]. https://doi.org/10.2196/12967.

  • 34.

    Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP, et al. Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(1). e12959. [PubMed ID: 30668519]. [PubMed Central ID: PMC6362387]. https://doi.org/10.2196/12959.

  • 35.

    Kitamura FC. ChatGPT Is Shaping the Future of Medical Writing But Still Requires Human Judgment. Radiol. 2023;307(2). e230171. [PubMed ID: 36728749]. https://doi.org/10.1148/radiol.230171.

  • 36.

    Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, et al. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digit Med. 2023;6:75. https://doi.org/10.1038/s41746-023-00819-6.

  • 37.

    Anders BA. Why ChatGPT is such a big deal for education. C2C Digital Magazine. 2023;1(18):4.

  • 38.

    Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med Educ. 2023;9:e45312. [PubMed ID: 36753318]. [PubMed Central ID: PMC9947764]. https://doi.org/10.2196/45312.

  • 39.

    Wang X, Gong Z, Wang G, Jia J, Xu Y, Zhao J, et al. ChatGPT Performs on the Chinese National Medical Licensing Examination. J Med Syst. 2023;47(1):86. [PubMed ID: 37581690]. https://doi.org/10.1007/s10916-023-01961-0.

  • 40.

    Jungwirth D, Haluza D. Artificial Intelligence and Public Health: An Exploratory Study. Int J Environ Res Public Health. 2023;20(5). [PubMed ID: 36901550]. [PubMed Central ID: PMC10002031]. https://doi.org/10.3390/ijerph20054541.

  • 41.

    Topsakal O, Akinci TC, Celikoyar M. valuating Patient and Otolaryngologist Dialogues Generated by ChatGPT, Are They Adequate? SSRN. 2023;Preprint. https://doi.org/10.21203/rs.3.rs-2719379/v1.

  • 42.

    Gupta PK, Raturi S, Venkateswarlu P. Chatgpt for designing course outlines: A boon or bane to modern technology. SSRN Electronic J. 2023;preprint. https://doi.org/10.2139/ssrn.4386113.

  • 43.

    Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepano C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2). e0000198. [PubMed ID: 36812645]. [PubMed Central ID: PMC9931230]. https://doi.org/10.1371/journal.pdig.0000198.

  • 44.

    Giansanti D. Artificial Intelligence in Public Health: Current Trends and Future Possibilities. Int J Environ Res Public Health. 2022;19(19). [PubMed ID: 36231208]. [PubMed Central ID: PMC9565579]. https://doi.org/10.3390/ijerph191911907.

  • 45.

    Lomis K, Jeffries P, Palatta A, Sage M, Sheikh J, Sheperis C, et al. Artificial Intelligence for Health Professions Educators. NAM Perspect. 2021;2021. [PubMed ID: 34901780]. [PubMed Central ID: PMC8654471]. https://doi.org/10.31478/202109a.

  • 46.

    Farajollahi M, Modaberi A. Can ChatGPT pass the "Iranian Endodontics Specialist Board" exam? Iran Endod J. 2023;18(3):192. [PubMed ID: 37431530]. [PubMed Central ID: PMC10329760].

  • 47.

    Haque MU, Dharmadasa I, Sworna ZT, Rajapakse RN, Ahmad H. " I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data. arXiv. 2022;preprint.

  • 48.

    Thorp HH. ChatGPT is fun, but not an author. Sci. 2023;379(6630):313. [PubMed ID: 36701446]. https://doi.org/10.1126/science.adg7879.

  • 49.

    Mbakwe AB, Lourentzou I, Celi LA, Mechanic OJ, Dagan A. ChatGPT passing USMLE shines a spotlight on the flaws of medical education. PLOS Digit Health. 2023;2(2). e0000205. [PubMed ID: 36812618]. [PubMed Central ID: PMC9931307]. https://doi.org/10.1371/journal.pdig.0000205.

  • 50.

    Cotton D, Cotton P, Shipway J. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. Taylor & Francis Group. 2024;61(2):228-39. https://doi.org/10.35542/osf.io/mrz8h.

  • 51.

    Chen Y, Zhao C, Yu Z, McKeown K, He H. On the Relation between Sensitivity and Accuracy in In-Context Learning. arXiv. 2023;Preprint:155-67. https://doi.org/10.18653/v1/2023.findings-emnlp.12.

  • 52.

    Duong D, Solomon BD. Analysis of large-language model versus human performance for genetics questions. Eur J Hum Genet. 2024;32:466-8. https://doi.org/10.1038/s41431-023-01396-8.

  • 53.

    Huh S. Are ChatGPT's knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study. J Educ Eval Health Prof. 2023;20:1. [PubMed ID: 36627845]. [PubMed Central ID: PMC9905868]. https://doi.org/10.3352/jeehp.2023.20.1.

  • 54.

    Lin Z. Why and how to embrace AI such as ChatGPT in your academic life. Royal Soc Open Sci. 2023;10(8):230658.

  • 55.

    Patel SB, Lam K. ChatGPT: the future of discharge summaries? Lancet Digit Health. 2023;5(3):e107-8. [PubMed ID: 36754724]. https://doi.org/10.1016/S2589-7500(23)00021-3.

  • 56.

    Mijwil M, Aljanabi M, Ali AH. ChatGPT: Exploring the Role of Cybersecurity in the Protection of Medical Information. Mesopotamian J Cyber Security. 2023:18-21. https://doi.org/10.58496/mjcs/2023/004.

  • 57.

    Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A. Generating scholarly content with ChatGPT: ethical challenges for medical publishing. Lancet Digit Health. 2023;5(3):e105-6. [PubMed ID: 36754725]. https://doi.org/10.1016/S2589-7500(23)00019-5.

  • 58.

    Aczel B, Wagenmakers E. Transparency guidance for ChatGPT usage in scientific writing. PsyArXiv. 2023;Preprint. https://doi.org/10.31234/osf.io/b58ex.

  • 59.

    Hallsworth JE, Udaondo Z, Pedros-Alio C, Hofer J, Benison KC, Lloyd KG, et al. Scientific novelty beyond the experiment. Microb Biotechnol. 2023;16(6):1131-73. [PubMed ID: 36786388]. [PubMed Central ID: PMC10221578]. https://doi.org/10.1111/1751-7915.14222.

  • 60.

    Marchandot B, Matsushita K, Carmona A, Trimaille A, Morel O. ChatGPT: the next frontier in academic writing for cardiologists or a pandora's box of ethical dilemmas. Eur Heart J Open. 2023;3(2):oead007. [PubMed ID: 36915398]. [PubMed Central ID: PMC10006694]. https://doi.org/10.1093/ehjopen/oead007.

  • 61.

    Wadden JJ. Defining the undefinable: the black box problem in healthcare artificial intelligence. J Med Ethics. 2022;48:764-8. [PubMed ID: 34290113]. https://doi.org/10.1136/medethics-2021-107529.

  • 62.

    Meyer JG, Urbanowicz RJ, Martin PCN, O'Connor K, Li R, Peng PC, et al. ChatGPT and large language models in academia: opportunities and challenges. BioData Min. 2023;16(1):20. [PubMed ID: 37443040]. [PubMed Central ID: PMC10339472]. https://doi.org/10.1186/s13040-023-00339-9.

  • 63.

    Sajadi HS, Yahyaei F, Ehsani-Chimeh E, Majdzadeh R. The human cost of economic sanctions and strategies for building health system resilience: A scoping review of studies in Iran. Int J Health Plann Manage. 2023;38(5):1142-60. [PubMed ID: 37194133]. https://doi.org/10.1002/hpm.3651.

  • 64.

    Johnson DR. Emotional intelligence as a crucial component to medical education. Int J Med Educ. 2015;6:179-83. [PubMed ID: 26638080]. [PubMed Central ID: PMC4691185]. https://doi.org/10.5116/ijme.5654.3044.

  • 65.

    Zhao W, Zhao Y, Lu X, Wang S, Tong Y, Qin B. Is chatgpt equipped with emotional dialogue capabilities? arXiv. 2023;preprint.

  • 66.

    Sanmarchi F, Bucci A, Nuzzolese AG, Carullo G, Toscano F, Nante N, et al. A step-by-step researcher's guide to the use of an AI-based transformer in epidemiology: an exploratory analysis of ChatGPT using the STROBE checklist for observational studies. Z Gesundh Wiss. 2023:1-36. [PubMed ID: 37361298]. [PubMed Central ID: PMC10215032]. https://doi.org/10.1007/s10389-023-01936-y.

  • 67.

    Paranjape K, Schinkel M, Nannan Panday R, Car J, Nanayakkara P. Introducing Artificial Intelligence Training in Medical Education. JMIR Med Educ. 2019;5(2). e16048. [PubMed ID: 31793895]. [PubMed Central ID: PMC6918207]. https://doi.org/10.2196/16048.

  • 68.

    McCoy LG, Nagaraj S, Morgado F, Harish V, Das S, Celi LA. What do medical students actually need to know about artificial intelligence? NPJ Digit Med. 2020;3:86. [PubMed ID: 32577533]. [PubMed Central ID: PMC7305136]. https://doi.org/10.1038/s41746-020-0294-7.

  • 69.

    van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. 2023;614(7947):224-6. [PubMed ID: 36737653]. https://doi.org/10.1038/d41586-023-00288-7.

  • 70.

    Lund BD, Wang T. Chatting about ChatGPT: how may AI and GPT impact academia and libraries? Library Hi Tech News. 2023;40(3):26-9. https://doi.org/10.1108/lhtn-01-2023-0009.

  • 71.

    Kolachalama VB, Garg PS. Machine learning and medical education. NPJ Digit Med. 2018;1:54. [PubMed ID: 31304333]. [PubMed Central ID: PMC6550167]. https://doi.org/10.1038/s41746-018-0061-1.

  • 72.

    Zarei M, Zarei M. Error Assessment of Artificial Intelligence Results: What We Need to Do in Medical Education. Strides Dev Med Educ. 2023;20(1):38-9. https://doi.org/10.22062/sdme.2023.198333.1173.

  • 73.

    Park SH, Do KH, Kim S, Park JH, Lim YS. What should medical students know about artificial intelligence in medicine? J Educ Eval Health Prof. 2019;16:18. [PubMed ID: 31319450]. [PubMed Central ID: PMC6639123]. https://doi.org/10.3352/jeehp.2019.16.18.

  • 74.

    Srivastava TK, Waghmare L. Implications of Artificial Intelligence (AI) on Dynamics of Medical Education and Care: A Perspective. J Clin Diagnostic Res. 2020;14(3):1-2. https://doi.org/10.7860/jcdr/2020/43293.13565.

  • 75.

    van der Niet AG, Bleakley A. Where medical education meets artificial intelligence: 'Does technology care?'. Med Educ. 2021;55(1):30-6. [PubMed ID: 32078175]. https://doi.org/10.1111/medu.14131.

  • 76.

    Han ER, Yeo S, Kim MJ, Lee YH, Park KH, Roh H. Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review. BMC Med Educ. 2019;19(1):460. [PubMed ID: 31829208]. [PubMed Central ID: PMC6907217]. https://doi.org/10.1186/s12909-019-1891-5.

  • 77.

    Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. A lite bert for self-supervised learning of language representations. arXiv. 2019;preprint.

  • 78.

    Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, et al. Palm: Scaling language modeling with pathways. J Machine Learn Res. 2023;24(240):1-113.

  • 79.

    Wu Y, Rabe MN, Hutchins D, Szegedy C. Memorizing transformers. arXiv. 2022;preprint.

  • 80.

    Huang J, Gu SS, Hou L, Wu Y, Wang X, Yu H, et al. Large language models can self-improve. arXiv. 2022;Preprint.

  • 81.

    Kolachalama VB. Machine learning and pre-medical education. Artif Intell Med. 2022;129:102313. [PubMed ID: 35659392]. [PubMed Central ID: PMC10375468]. https://doi.org/10.1016/j.artmed.2022.102313.

  • 82.

    Nagarhalli T, Vaze V, Rana N. A Review of Current Trends in the Development of Chatbot Systems.In: Nagarhalli T, Vaze V, Rana N, editors. International Conference on Advanced Computing and Communication Systems (ICACCS). TamilNadu, INDIA. International Conference on Advanced Computing and Communication Systems; 2020. p. 706-10.

  • 83.

    Chan KS, Zary N. Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review. JMIR Med Educ. 2019;5(1). e13930. [PubMed ID: 31199295]. [PubMed Central ID: PMC6598417]. https://doi.org/10.2196/13930.

  • 84.

    Egger J, Gsaxner C, Chen X, Bian J, Kleesiek J, Puladi B. Apple Vision Pro for Healthcare:" The Ultimate Display"?--Entering the Wonderland of Precision. arXiv. 2023:;Perprint.

  • 85.

    Zarei M, Bavil F, Noordoost M. The role of student mentoring in revealing hidden curriculum. Res Dev Med Educ. 2022;11:16. https://doi.org/10.34172/rdme.2022.016.