The Design, Implementation, and Evaluation of a Virtual Objective Structured Clinical Examination as a Tool for Assessing Clinical Competence in Nursing Students

The objective structured clinical examination (OSCE) constitutes a crucial assessment modality within medical education. Developed by Harden et al. in the 1970s, the OSCE was designed to measure clinical competence in medical students, enabling the discrete assessment of individual clinical skills through a series of predetermined, timed stations (1). This examination format minimizes the influence of chance, ensures standardized conditions for all examinees, and facilitates precise scoring (2). McMaster University's School of Nursing pioneered the application of the OSCE for assessing primary care nursing skills in junior students in 1984 (3). Miller's pyramid provides a framework for clinical assessment, delineating suitable assessment methodologies for various learning stages. This model comprises four hierarchical levels, progressing from the foundational level of knowledge assessment to higher levels evaluating competence, performance, and actual practice. Within this framework, the OSCE is identified as a relevant tool for evaluating the demonstration of individual skills (4). The OSCE serves as a cornerstone of a comprehensive, multi-faceted approach to clinical assessment and has undergone significant evolution to accommodate diverse contexts (5).

The increasing prevalence of virtual OSCEs (VOSCEs) has been observed in recent years. Initially conceived as a novel approach to facilitate remote learner participation in these assessments (6, 7), VOSCEs emerged in the early 2000s under various nomenclatures, including electronic OSCE (e-OSCE), tele-OSCE, and web-OSCE, specifically designed to measure competences relevant to telehealth practices (8). Despite generally positive reception from both professors and students, and demonstrable comparability with traditional OSCE outcomes, VOSCEs have not achieved widespread adoption as a standard assessment modality over the past two decades (7). This limited uptake may be attributed to concerns regarding students' perceived capacity to effectively demonstrate their abilities in a virtual environment, coupled with potential challenges faced by examiners in accurately evaluating student skills remotely (9).

The coronavirus disease 2019 (COVID-19) pandemic significantly accelerated the exploration of virtual platforms for designing and implementing VOSCEs (10, 11). At the pandemic's peak, numerous medical schools transitioned their final-year OSCEs to a virtual format (12). Evidence suggests that VOSCEs effectively facilitated both the learning and assessment of student clinical competence (13, 14). Virtual assessment modalities not only provide students with insights into their strengths and weaknesses in knowledge and clinical skills but also encourage them to capitalize on their strengths and address their deficiencies. Administering clinical skills assessments virtually can promote students' self-awareness of their educational needs prior to entering professional practice. An additional aim of virtual clinical competence examinations is to evaluate students' capabilities across skill-based, educational, and communication domains to ensure their preparedness for clinical practice roles and responsibilities (15, 16).

Research comparing VOSCEs and traditional OSCEs, however, remains limited (8). The field of nursing is fundamentally grounded in practical and clinical skills. Within this context, even a single error can have irreversible consequences, potentially resulting in irreparable harm (17). Consequently, it is of paramount importance to rigorously assess the competence and capabilities of nursing students prior to their transition into professional practice (18).

Considering the increasing prevalence of technology in higher education teaching and assessment, and recognizing the importance of evaluating the readiness of senior nursing students for clinical practice, this study aimed to develop, implement, and assess the effectiveness of a VOSCE in nursing students.

This quasi-experimental study was conducted on a sample of 176 senior nursing students between February 2022 and February 2024. Participants were enrolled in the study through census and were simply randomly assigned to two groups: First the virtual OSCE test, then the traditional OSCE test, or vice versa. The research process, encompassing design, implementation, and evaluation, was structured into six distinct phases.

A cohort of 176 nursing students undertook concurrent OSCEs and VOSCEs across a five-semester period. The sample comprised 107 (61.14%) female and 68 (38.86%) male students. The mean score achieved on the VOSCE was 17.35, while the mean score on the in-person OSCE was 16.86. Statistical analysis revealed no significant difference between the two exam modalities (Table 1). A correlational analysis was performed to determine the relationship between individual station scores and the overall examination score. The results demonstrated a direct significant correlation between the total scores achieved on the virtual examination and those earned on the in-person examination (r = 0.899, P < 0.001) within the examined cohort. This correlation was observed across all individual stations in both the virtual and in-person examination formats (Table 2). A student satisfaction survey revealed overwhelmingly positive satisfaction with the VOSCE. Specifically, 94% of respondents felt that the VOSCE enhanced their self-efficacy and confidence. Furthermore, 86% reported that the examination improved their acquisition of practical, field-specific skills. A substantial majority (94%) believed that the virtual format could replace traditional, in-person examinations for many components in the future, and 93% expressed strong support for the VOSCE's continued implementation. These student satisfaction survey results are summarized in Table 3. Colleagues' discussions during EDO meetings further explored the VOSCE's advantages and disadvantages. Key benefits identified included reductions in time, cost, and resource utilization compared to traditional OSCEs. Survey data also suggested that the VOSCE environment resulted in lower student stress levels, positively impacting performance. Additional benefits were the familiarization of students with contemporary assessment methodologies and the expanded range of question types facilitated by the technological platform.

This study investigated the feasibility of virtual clinical competence examinations for nursing students, utilizing the Faradid electronic platform. The examination focused on the clinical and practical skills assessed at each station. Results demonstrated no statistically significant difference in mean scores between students who completed the exam virtually and those who took it in person. A significant proportion of the student cohort favored the continued use of virtual examinations, suggesting the potential for transitioning from traditional in-person testing to a virtual format. In a study conducted in Hong Kong, Chan et al. employed virtually designed clinical scenarios. Virtual simulators facilitated student interaction with virtual patients, replicating a real-world clinical environment. This method of virtual clinical simulation has been shown to improve students' decision-making capabilities, critical thinking skills, and overall performance and psychomotor proficiency (17). Furthermore, Arrogante et al. evaluated the clinical competence of nursing students in Paris during the COVID-19 pandemic using a standardized VOSCE. Their findings indicated that the VOSCE achieved comparable results to traditional in-person examinations while simultaneously reducing costs. Virtual OSCEs, utilizing standardized patients, present a viable alternative assessment method. Beyond their applicability during the COVID-19 pandemic, the use of VOSCEs warrants consideration for broader implementation, including in non-pandemic contexts (18). A study conducted in Iran by Tolabi and Yarahmadi exemplifies this potential. Their 2021 virtual clinical competence examination, administered to 42 senior nursing students at Khorramabad University of Medical Sciences via the Porsline platform across 9 stations, yielded results comparable to those of prior in-person examinations at the same institution. This finding suggests that VOSCEs can effectively substitute for traditional, in-person assessments (19). In another study conducted at the Nursing and Midwifery School of Lorestan University of Medical Sciences, Mojtahedzadeh et al. developed, implemented, and assessed a blended (combining in-person and virtual components) competence examination for graduating nursing students. As shown by their findings, this blended approach offers a viable alternative to traditional OSCEs, particularly during pandemics or periods of resource scarcity. The blended methodology allowed for the evaluation of fundamental and advanced clinical skills through in-person stations, while virtual stations, based on presented scenarios, facilitated the assessment of competences in communication, reporting, nursing diagnosis, professional ethics, mental health, and community health. Moreover, the blended format demonstrably reduced the requirements for human resources, station setup costs, and time spent compared to a fully in-person format (20). Prior research corroborates the findings of the present study, collectively demonstrating the efficacy of virtual assessment modalities. Specifically, these investigations indicate that the implementation of VOSCEs offers substantial advantages, extending beyond the exigencies of the COVID-19 pandemic to encompass routine post-pandemic practice. These benefits include, notably, reductions in both costs and time expenditures. Consequently, VOSCEs can serve as a viable alternative when traditional in-person examinations are infeasible. Furthermore, this methodology can be appropriately utilized as a replacement for, or a valuable adjunct to, conventional in-person assessments. Avraham et al. developed a VOSCE preparation program delivered via Zoom for second-year nursing undergraduates at Ben-Gurion University's Department of Nursing. The results indicated a high level of student satisfaction (88%) with the program in 2021, with students reporting feeling adequately prepared for the OSCE (26% agree and 62% strongly agree). Importantly, the study found no statistically significant difference in OSCE performance between students who participated in the 2021 virtual program and those who participated in traditional in-person OSCE preparation programs between 2017 and 2020 (21). Similarly, a traditional in-person OSCE was adapted and implemented using a virtual conferencing platform for nursing students at the University of Texas Medical Branch in Galveston. Survey data revealed that the majority of professors and students perceived the VOSCE as a highly effective method for assessing communication skills, history taking, differential diagnosis, and patient management (22). Hopwood et al. implemented a VOSCE at the Medical School of University College London (UCL) to evaluate the competences of senior medical students. The VOSCE comprised 18 stations designed to assess clinical, theoretical, practical, examination, and professional skills. Feedback solicited from all stakeholders demonstrated a high degree of satisfaction, with 94% recommending the virtual format for future examinations (10). Jenkins et al. employed a simulated virtual pilot examination for urology graduate certification, comprising a VOSCE and a multiple-choice written component. High satisfaction was reported, with 92.9% of examiners and 87.5% of candidates rating the experience positively (9). Similarly, Mojtahedzadeh et al. found equivalent and relatively high levels of faculty member satisfaction with in-person, virtual, and blended formats of a clinical competence examination (20). Recent research suggests that virtual simulations offer benefits beyond knowledge and skill acquisition. They may also foster increased self-confidence, influence attitudes and behaviors, and represent a novel, adaptable, and promising pedagogical tool for novice nurses and nursing students (22). The potential impact of virtual programs on nursing education is substantial. Their accessibility, flexible scheduling, and location flexibility (23) may significantly enhance the development of students' clinical competences (24). These studies corroborate the findings of the present research, indicating substantial satisfaction with virtual assessments among both professors and students. Furthermore, student survey responses in this study revealed that this assessment modality positively influenced their satisfaction, engagement, sense of empowerment, and self-efficacy.