1. Background
Cancer management has become increasingly challenging due to the emergence of a personalized approach that requires careful assessment, multidisciplinary efforts, and experienced physicians. As there is an increasing knowledge in cancer management and an increasing trend of sub-specialization among specialties related to cancer treatment, tumor boards (TB) have become successfully accepted for better management and evidence-based decision-making (1). The purpose of TB is to provide a thorough evaluation of each cancer patient regarding pre-treatment, treatment, and survivorship. This approach leads to proper staging, diagnosis, treatment modalities, management, follow-up, and recurrence monitoring (2).
Multidisciplinary tumor boards (MDT) consists of diverse specialties including medical oncologists, radiation oncologists, surgeons, radiologists, and pathologist cooperating to achieve high-quality treatment recommendations for cancer patients (3). Studies have reported that multidisciplinary approaches in tumor boards result in improved cancer patient treatment. However, there is a need to introduce novel approaches that enable community providers to access tumor board sessions regardless of their geographical locations. While some providers in urban hospitals have complete access to tumor board sessions and review all types of cases, there are some hospitals located in rural areas that lack access to tumor board sessions (4). Furthermore, the COVID-19 pandemic has altered holding meetings and management and treatment approaches of patients especially cancer patients, and multidisciplinary cancer management has become more challenging to manage. Therefore, there is a need to create and form virtual multidisciplinary boards to improve collaboration between healthcare providers across various geographical locations and in different situations such as the COVID-19 pandemic. Moreover, the transition to virtual tumor board meetings gives providers this opportunity to reduce time wasted on travel and allocate proper cases and specialists (3).
Shahid Beheshti University of Medical Sciences (SBMU) is one of the biggest medical educational universities in Iran with different hospitals serving urological care across Iran’s capital, Tehran. In this regard, we performed a prospective study to examine the efficacy and acceptability of virtual tumor boards for urologic cancers in educational hospitals of SBMU.
2. Objectives
In this study, we aimed at evaluating the efficacy of virtual tumor board meetings on the learning efficacy of urology residents and the acceptance of virtual tumor boards among urology professors and residents.
3. Methods
This was a prospective study, evaluating the designing, implementation, and learning efficacy of a virtual urology tumor board in the Urology Department of SBMU. This study was approved by the Ethics Committee of SBMU (IR.SBMU.SME.REC.1400.095).
3.1. Subjects and Participants
The Urology Department of SBMU is not large in terms of the number of faculty members and residents. Therefore, inclusion and exclusion criteria were not determined and all the urology faculty members (N = 25) and urology residents (N = 35) were included in this investigation. To make the sessions multidisciplinary, the panelists were invited to each session from other related departments including pathology, radiation oncology, medical oncology, radiology, and nuclear medicine.
3.2. Setting
Virtual tumor boards were implanted monthly in 20 sessions. SBMU is the second medical and educational university in Tehran, Iran, and one of the biggest national cancer referral centers in Iran. It provides all cancer-related services such as radiotherapy, chemotherapy, and cancer surgery. A strong point that was considered for designation this study was the presence of different disciplines and specialties including urology, pathology, radio-oncology, medical oncology, radiology, and nuclear medicine departments. Tumor boards consist of 3 parts: (1) the first part is the introduction of the cancer patient, (2) the second is the panel of specialists and experts who discuss the diagnostic and treatment modalities of the cases, (3) and the third part includes the learners, consisting of the urology residents in our study. In the virtual tumor board, the fourth part is the online platform, in which the meetings are held. After a discussion with experts in the field of information technology, we decided to use the Skyroom application.
3.3. Designing, Implementing, and Learning Framework
The aim of this study was to design, implement, and evaluate the efficacy of a virtual tumor board on the learning of urology residents. To identify and assess the requirements and needs of the study participants in 3 parts of the tumor board, a questionnaire was completed by the faculty members and urology residents before organizing the program.
To evaluate the results of virtual tumor board sessions on participants, we recorded questionnaires according to the Kirkpatrick model at the first and second levels (5). The results of this learning program and the level of satisfaction of participants according to the first level of the Kirkpatrick model were recorded in 3 areas, including (1) achieving one clinical decision for the treatment and management of cancer patient, (2) evidence-based topic, and (3) applicability of the content. Since the tumor board sessions were conducted virtually, the virtual presentation was included as the fourth part of the questionnaire. For designing the questions of this part, in addition to the opinions of faculty members and residents, the opinions of the experts in the field of information technology were also obtained. Evaluation of the results of the learning program on the learning of residents was done according to the second level of the Kirkpatrick model. This evaluation was conducted before and after the implementation of the tumor boards in 5 consecutive sessions. At the beginning of each TB, the pre-test and post-test of the previous session were taken virtually. For the management of the meeting, two coordinators were assigned for each session to handle the meeting, manage the timetable, interact with participants, and cooperate with information technologists to eliminate probable technical issues.
3.4. Data Collection and Statistical Analysis
This study took place over 20 months from December 2020 to July 2022. Data were recorded through short interviews before the implementation of the sessions to assess the needs of stakeholders and surveys after the sessions to evaluate the effect of the program. Statistical analysis included descriptive analysis and paired t test. All descriptive and statistical analyses were performed, using Statistical Analysis System version 9.1 software. P < 0.05 was considered statistically significant.
4. Results
The Urology Department of SBMU consists of 25 faculty members (including assistant professors, associate professors, and full professors) with a mean age of 52.2. The gender of faculty members included 4 females and 21 males. A short interview was conducted with all faculty members regarding their needs for holding a virtual tumor board. Their statements on a 5-point Likert scale were summarized into 3 groups of high, intermediate, and low importance. These items are presented in Table 1.
Table 1.Needs of Faculty Members Regarding Holding Tumor Board Sessions
| Needs of Faculty Members | Importance | |
|---|---|---|
| 1 | Discussing patients’ problems in an evidence-based manner | High |
| 2 | Discussing cases in a multidisciplinary manner | High |
| 3 | Reaching out to one or more management and treatment methods for cases | High |
| 4 | Definite and observed timetable for each patient and session | Intermediate |
| 5 | Implementation of sessions in cooperation with other universities | Intermediate |
| 6 | Complete para-clinical explanation of cases including radiology and pathology images | Intermediate |
| 7 | Involvement of urology residents in the discussions to have educational tumor board sessions | Low |
| 8 | Coordination with scientific programs of other faculties and universities | Low |
| 9 | Avoiding lectures and presentations in a patient-oriented manner | Low |
In the next step, we interviewed urology residents and recorded their needs regarding holding tumor board sessions. It is worth mentioning that residents in the first and second years are more involved in the emergency department and on the front line of visiting patients and implementing the medical orders of their senior residents. Therefore, most of their time is on duty, taking patients’ histories, and matters related to hospitalization and discharge, and have less opportunity to study and contribute to scientific meetings. On the other hand, the third and fourth-year urology residents are more involved in surgeries and patients’ decision-making; therefore, they have more time for studying and participating in scientific programs. Thus, the needs of residents would be different according to their entry year and should be divided (Table 2).
Table 2.Needs of Urology Residents Regarding Holding Tumor Board Sessions
| Third and Fourth-Year Urology Residents | First and Second-Year Urology Residents | Importance | |
|---|---|---|---|
| 1 | Avoiding highly specialized topics | Making it possible to participate in the program | High |
| 2 | Limiting the number of patients in each session | Definite and observed timetable for each patient and sessions | High |
| 3 | Educational explanation of each radiology image | - | High |
| 4 | More participation of residents to maintain sessions more educational | Having the opportunity to pre-study the topics | Intermediate |
| 5 | - | Participation of residents from a different discipline | Intermediate |
| 6 | Provide follow-up of patients in the next sessions | - | Low |
| 7 | Definite and observed timetable for each patient and sessions | - | Low |
The satisfaction questionnaire based on the first level of the Kirkpatrick model was recorded and is presented in Table 3.
Table 3.Satisfaction of Faculty Members and Urology Residents from Tumor Board Sessions
| Satisfaction Items | Faculty Members | Urology Residents |
|---|---|---|
| Achieving a specific clinical decision (number, percentage) | 23 (92) | 30 (85) |
| Evidence-based discussions (number, percentage) | 21 (84) | 28 (80) |
| Clinical applicability of topics (number, percentage) | 24 (96) | 31 (88) |
| Proper virtual presentation (number, percentage) | 19 (76) | 25 (71) |
aValues are expressed as No. (%).
The results of the intervention were evaluated according to the second level of the Kirkpatrick model and through the tests before and after the tumor board in 5 consecutive sessions. Table 4 shows the scores of the tests.
Table 4.Mean Score of 100-Point Pre-tests and Post-tests of Urology Residents of Shahid Beheshti University of Medical Sciences
| Tests | Third and Fourth-Year Residents | P -Value | First and Second-Year Residents | P-Value | ||
|---|---|---|---|---|---|---|
| Pre-tests | Post-tests | Pre-tests | Post-tests | |||
| 1 | 43.2 | 61.4 | < 0.05 | 34.6 | 45.7 | > 0.05 |
| 2 | 59.6 | 83.9 | < 0.05 | 41.9 | 63.4 | < 0.05 |
| 3 | 74.3 | 95.7 | < 0.05 | 55.4 | 83.3 | < 0.05 |
| 4 | 78.2 | 97.6 | > 0.05 | 64.7 | 89.9 | < 0.05 |
| 5 | 83.5 | 98.8 | > 0.05 | 71.9 | 90.8 | < 0.05 |
| Total | 67.76 | 87.48 | < 0.05 | 57.48 | 74.62 | < 0.05 |
In the next step for external evaluation as the last step in Glassick's Criteria (6) for evaluating educational Scholarship, we invited 5 faculty members of the Urology Department of Tehran and Iran Universities of Medical Sciences to participate and oversee the virtual tumor board sessions to explain their critics and suggestion about the implantation of the program. We also obtained their satisfaction rate regarding the implementation of virtual tumor board sessions, which is shown in Table 5. Their comments were collected through a short interview and are presented in Table 6 according to their importance.
Table 5.Comments of Faculty Members of Other Medical Universities
| Satisfaction Items | Percentage (%) |
|---|---|
| Achieving a specific clinical decision | 90 |
| Evidence-based discussions | 88 |
| Clinical applicability of topics | 96 |
| Proper virtual presentation and technical matters | 75 |
Table 6.Recommendation of External Evaluators Regarding the Virtual Tumor Board Sessions
| Suggestions | Importance | |
|---|---|---|
| 1 | Providing a balanced mix of rare and common diseases in sessions | High |
| 2 | Proper time management for each case | High |
| 3 | Improving the quality of radiology images | High |
| 4 | More participation of urology residents in sessions | Intermediate |
| 5 | Participation of urologic oncology fellowship residents in sessions | Intermediate |
| 6 | Using more different specialties in sessions | Low |
| 7 | Provide follow-up of patients in the next sessions | Low |
5. Discussion
The primary goal of this study was to evaluate whether the implementation of virtual tumor boards at SBMU is attainable. We demonstrated that the virtual tumor board is highly accepted and satisfactory by the faculty members and urology residents and is an effective learning method for the urology residents. Furthermore, we investigated the weak and strong points of our program, which could be used in upcoming scientific programs. As shown in the result section, virtual tumor board sessions, which is one of the requirements of the surgical departments that have cancer patients, were highly satisfactory for the faculty members and urology residents. We also tried to organize the program according to the needs and opinions of stakeholders. Unfortunately, one of the most important beneficiaries, the patients, was ignored in this research; we could not obtain their opinions regarding the amount of profit they got from holding these sessions.
The positive point of this investigation was the need assessment of faculty members and urology residents, which, as expected, was different from each other’s. The most important concern for urology residents was the opportunity to participate in the meetings. Due to the large number of patients referred to the urology department of SBMU, the most time of residents is spent on clinical and medical care rather than studying and participating in scientific meetings.
Regarding technical problems, most of the participants (76% of faculty members and 71% of urology residents) were satisfied and reported low technical issues. However, the most important technical issue was the result of bandwidth downspeed.
There are several studies supporting the utility of virtual tumor boards in the medical education of residents. A study conducted by Marshall et al. in 2014, tried to evaluate the satisfaction and acceptance rate of virtual tumor boards. They indicated that virtual tumor boards are highly feasible and acceptable and their effectiveness was equivalent to that observed in personal tumor board sessions. Therefore, they concluded that the implementation of a virtual tumor board is a validated method for medical education (7). Another investigation conducted by Look Hong et al. in 2003, assessed the efficacy of videoconferences for the oncology rounds and surgeons. The majority of participants (75%) were satisfied with the virtual session, demonstrating that oncologists and surgeons can engage in a multidisciplinary virtual round (8). A recent study conducted by Davis et al. revealed that holding tumor board sessions virtually increased the number of participants with different specialties (9). A study conducted by Hopkins et al. in 2022, showed that 72.5% of participants found that virtual tumor boards are more time efficient with the same productivity as personal tumor boards, 85.5% found that virtual tumor boards are easier to participate in, and 89.9% declared that decision-making process was not affected by this format (10).
In this study, the authors showed this training scholarship program could overcome the training problem, which was explained by Abedi et al. in the urology residency training program in Iran during the COVID-19 pandemic (11), and this virtual tumor board is an effective method for improving learning in urology residents, as demonstrated by level 2 Kirkpatrick model in our study. Senior urology residents had a greater mean score compared to junior residents, which could be due to more time opportunities that senior residents have to study. Pre-test and post-test scores improved gradually and over time, indicating that the virtual tumor board is an effective learning method. Furthermore, post-test scores were always higher than pre-test scores but did not always show a significant difference. These results could be due to the repetition of some common contents in sessions that helped residents to better remember these topics.
To improve the learning objectives and efficacy of these sessions, we invited different urology specialties and related specialties such as pathology, radiology, nuclear medicine, radio-oncology, and medical oncology. Holding the tumor board virtually made it possible to participate in the sessions from all parts of the country and even in some cases from abroad. As an example, we invited Iranian urologists living in other countries such as the USA, Canada, and England to share their opinions and management methods for each case and it helped to make sessions more interesting and improved the scientific richness of the sessions. With the slow progression of the COVID-19 pandemic, we decided to continue our sessions in a hybrid format, meaning that our sessions included both virtual and personal formats and it helped us to hold the sessions more collaborative and available for dispersed colleagues.
5.1. Limitations
This study had some limitations. Firstly, the quality of the radiology images was low and as one of the important priorities of the residents was to get familiar with reading stereotypes of radiology, we tried to reduce the number of images on each page and increase the quality of each image. However, we were not able to completely remove this issue. We tried to use picture achieving and communication system (PACS) to provide images; however, this system was only available for the patients referred from the center. Secondly, creating and maintaining audience participation was the duty of session managers, which was done appropriately in some circumstances. However, there were always some participants, who did not interact and were not involved in sharing their opinions and commenting. A solution should be found to better manage and handle the sessions to give these people more opportunities to interact. Thirdly, satisfaction of patients and their related needs was not considered in this study.
5.2. Conclusions
A virtual tumor board is an effective learning method for the medical education of urology residents. It could help the practitioner to coordinate and discuss with different specialties. This method increases convince for participants, reduces travel time to central location, and makes it possible for every provider to participate in sessions regardless of geographical distance.
