Evaluation of Foundation Course for First-Year Medical Undergraduates in a Medical College of Eastern India

1. Background

Undergraduate medical education in India has undergone a paradigm shift with the introduction of competency-based medical education (CBME) in 2019 (1). Competencies needed by an Indian Medical Graduate for passing Bachelor of Medicine and Bachelor of Surgery (MBBS) were enlisted by the National Medical Council (formerly Medical Council of India) as clinical competencies, lifelong learning skills, leadership skills, communication skills, and professionalism (2). The foundation course (FC) at the beginning of the curriculum was proposed as a part of CBME to train newly admitted medical undergraduates on soft skills, such as communication, professional ethics, and leadership abilities. The purposes of the FC have been defined as (1) orienting the students to all aspects of the medical college environment; (2) equipping the students with important basic skills required for patient care and enhancing their communication, language, computer, and learning skills; and (3) providing an opportunity for peer and faculty interactions and overall sensitization to various learning methodologies (3).

The attitude, ethics, and communication (AETCOM module) under the FC is an opportunity to develop competencies in communication skills (4). Previous studies have reported poor skill levels for isolated components of the FC; accordingly, 72% of recently admitted undergraduates considered communication skills training to be important, and 68.4% felt the need for formal training (5). A study conducted on Pakistani medical undergraduates indicated a poor disposition to critical thinking (6). There is evidence that the learning approach is related to both academic performance and anxiety among first-year medical undergraduates, thereby requiring facilitation (7). Communication, leadership, and learning skills are nontechnical skills and are expected to be present at a variable level at the baseline among freshers. The FC has been an innovation in medical education, and its present structure allows for interinstitutional variations. Therefore, institution-level evaluation was required.

As the FC heavily relies on affective components of learning, self-rating was preferred over a conventional recall-based written test. Furthermore, learning in the affective domain is built upon past experiences and is likely to be modified with new experiences in the FC. Consequently, the research questions which needed to be evaluated were as follows: (1) What are the self-rated baseline communication skills, interpersonal skills, intellectual skills, self-management skills, learning skills, and management-career skills of recently admitted medical undergraduates? (2) How far the relevant components in the FC can change their self-rated skills?

Accordingly, the objectives were to assess the self-rated baseline communication skills, interpersonal skills, intellectual skills, self-management skills, learning skills, and management-career skills of recently admitted medical undergraduates and the magnitude of change in self-rated knowledge and skills by completing the FC.

2. Methods

2.1. Study Design

The study design was a prospective, pre-post evaluation of a learning program by comparing participants’ self-rated scores. All first-year MBBS students admitted at Medical College Kolkata, India, in 2021 were eligible to participate in this study. Data collection was performed in March 2021 at the baseline and in June 2021 at the end of the FC. The total study duration was 6 months within January to June 2021.

2.2. Tools and Technique

The data were collected electronically by self-administered 39 items on skills in a learning questionnaire developed by the College of Physiotherapists of Ontario, Canada. This questionnaire is a validated tool with six subscales, namely communication skills, interpersonal skills, intellectual skills, self-management skills, learning skills, and management-career development skills (8). The pre-course questionnaire required participants to give separately 1 - 6 ratings on how important they think they should possess or acquire a specific skill and the extent to which they think they possess the mentioned skill. In both cases, 1 and 6 were the worst and best ratings, respectively. The students were required to rate only the extent to which they think that they possess a skill on the post-course questionnaire. The numbers of items under each subscale were 10, 4, 4, 8, 6, and 7 for communication skills, interpersonal skills, intellectual skills, self-management skills, learning skills, and management-career development skills, respectively. Maximum possible scores in each subscale were 60, 24, 24, 48, 36, and 42 for communication skills, interpersonal skills, intellectual skills, self-management skills, learning skills, and management-career development skills, respectively. The maximum possible score was 234. Data elements also included participants’ demographic details, such as age in the completed year, gender, living arrangement (living in the campus hostel or not), residence (rural or urban), and knowledge of local vernacular Bengali. A Google form was prepared based on the skills for learning questionnaire. The link to complete the pre-course questionnaire was mailed on the first day of the FC and returned on the same session; similarly, the post-course questionnaire was mailed after 3 months on the penultimate day of the FC.

2.3. Sample

In 2021, there were 256 admissions at the study institute. With the consideration of 256 participants as the total population, 50% minimum expected improvement in skill (as there is no similar study performed in India to ascertain a priori probability of improvement), the α error of 5%, the relative error of 10%, and the nonresponse rate of 25%, the sample size was approximately 89 subjects. It was rounded up to 100. The students were invited by email to participate in the study. The selection of students was based on the simple random sampling technique using Google random number generator, where students’ roll numbers were the sampling frame.

2.4. Curricula/Intervention

The FC included 36 hours of orientation module, 41 hours of training on professional development, ethics, and communication in the AETCOM module, 38 hours of skill module which also included practice sessions on communication skills, 8 hours of field visits to urban and rural training centers attached to the institution, 40 hours of English and Bengali vernacular communication and computer application, 23 hours of outdoor games, self-defence training, extra-curricular activities, yoga, and meditation. During the implementation of the modules, students are required to engage in 10 hours of self-directed learning (SDL) (3). Although it was recommended to complete the FC in 1 month, due to the present pandemic situation, the 2021 FC was of 3 months duration at the study institute. The teaching-learning method was participatory, and all modules required concurrent or end-of-session learner’s presentation.

2.5. Outcome Variables and Their Definitions

- Pre-course importance: Mean of pre-course self-rated importance of skill for each subscale and total.

- Pre-course skill: Mean of pre-course self-rating of present skill for each subscale and total.

- Importance-skill gap: Difference of mean between pre-course importance and pre-course skill of participants for each subscale.

- Post-course skill: Mean of post-course self-rating of acquired skill for each subscale and total

- Skill improvement: Difference of mean between post-course and pre-course skills for each subscale and total

- Percentage of skill improvement: The percentage of students who had positive skill improvement

Statistical Analysis

A spreadsheet was generated for recorded responses. The total and subscale mean scores, standard deviations, confidence intervals, and standard errors of the mean were calculated for pre-course and post-course responses. Inferential statistics were primarily based on the difference of means for pre-course and post-course skills for subscales and total. A paired t-test was used to test for the difference in means of pre-course and post-course skills. An unpaired t-test was performed to check for the change in the mean scores of skills from the baseline and whether this change was significantly different between the subgroups. For the statistical tests, the α error was set at 0.05. The percentage of students showing improvement in self-rating skills after completing the FC was also calculated. The SPSS software 2010 (version 19.0) was used for statistical analyses.

2.6. Ethical Issues

The study protocol was approved by the Independent Ethics Committee of Medical College Kolkata (reference no.: MC/KOL/IEC/NON-SPON/1065/03/2021). The study was compliant with the revised Helsinki Declaration on bioethics. All participants gave informed consent to participate in the study. Anonymity was strictly maintained during data recording and coding. Pre-course and post-course questionnaires were compared using unique alphanumeric identities.

3. Results

A total of 97 students completed both pre-course and post-course questionnaires. These 97 questionnaires were included for analysis. The mean age of the respondents was 19.8 years (± 4 months). Furthermore, 61 (62.9%) students were male. In this study, 65 (67%) students resided in an urban area, and 61 (62.9%) students knew the local language. Among the participants, 47 (48.5%) students stayed at the campus hostel. Pre-course importance was high for all subscales but remarkably high for communication skills (mean: 56.32, where the maximum possible rating was 60). Pre-course importance was rated higher than pre-course skill for all subscales making the importance-skill gap positive. Post-course skill was rated higher than pre-course skill. However, skill improvement was lower than the importance-skill gap for all subscales. Post-course improvement was relatively modest, compared to the importance-skill gap of scale total. Despite such an existing gap, skill improvement was significant for communication skills, interpersonal skills, management-career development skills, and total skills (Table 1).

Table 2 shows the change in mean rating from the baseline and the standard errors of the mean for demographic subgroups. On demographic subgroup analysis, it was observed that learning skills significantly improved for students living in the campus hostel, and communication skills significantly improved when the local language was known (Table 2).

Following the FC, the highest proportion of students showed improvement in interpersonal (67.01%) and communication (62.89%) skills; nevertheless, the least improvement was observed in learning skills (48.45%). Management-career development skills (59.79%), intellectual skills (52.58%), and self-management skills (54.64%) improved for around half of the students (Figure 1). The percentage of skill improvement was higher for soft skills than academic skills.

Radar chart showing the percentage of students with perceived improvement at post-course for each subscale (n = 97).

Figure 1.

Radar chart showing the percentage of students with perceived improvement at post-course for each subscale (n = 97).

4. Discussion

This study aimed to assess improvement in learning skills by comparing the students’ ratings of their skills before and after the FC. There was an improvement in overall skills and the subscales, which was significant for communication, interpersonal, and management-career development skills. The highest number of students reported improvement in communication skills.

A study carried out by the Faculty of Sri Manakula Vinayagar Medical College Puducherry in India evaluated the FC at Kirkpatrick level 1, where learners’ satisfaction was evaluated; however, the present study required learners to rate their skills before and after the course (Kirkpatrick level 2) (9). In the Puducherry study, the outcome variable was the consensus score of agreement on the usefulness of a session. A score of 75% and above was considered to be a reasonable consensus. The sessions on learning styles (75.42%), SDL (75.72%), development of leadership skills, conflict management and time management (75.88%), communication skills (75.86%), and medical ethics (77.96%) had good consensus; nevertheless, the session on interpersonal skills (73.70%) reflected poor consensus (10). High consensus for communication skills is in line with the results of the present study; however, high consensus for learning styles and SDL and low consensus for interpersonal skills are not similar to the results of the present study, where 67% of students rated their interpersonal skills as improved and only 48% of students rated their learning skills to be improved after the FC. Furthermore, in the current study, post-course improvement was significant for interpersonal skills. Such difference can be attributed to the learners’ rating of sessions rather than their learning levels. It is reported that students can accurately identify but not quantify well learning by pre-post self-assessment tools (11).

In a study on cross-sectional evaluation of communication skills among medical undergraduates in Maharashtra, India, the mean importance of communication skills was 8.98 ± 1.31, and the mean of self-rated skills was 6.06 ± 1.52 (12). The importance-skill gap in the communication subscale in the present study was 9.63. Postgraduate ophthalmology students reported poor communication skills and poor baseline awareness of communication skills (13). A study conducted in Israel showed poor baseline performance at breaking bad news before a simulation-based mastery learning curriculum (14). The improvement of communication and interpersonal skills might have been related to the precourse importance-skill gap. This gap might have been a motivation for learning. Among the five dimensions of the Course Valuing Inventory (i.e., learning experience worthiness, emotional awareness, personal development, cognitive enhancement, and task drive), higher scores correlated positively and significantly to stronger self-confidence and greater motivation (15).

The greatest change was observed in interpersonal and communication skills. Competency in communication and knowledge of local vernacular among medical students is an under-explored area. Such findings would be particularly valuable in multilingual countries. A study on bachelor of dental surgery (BDS) students reported 55.3 - 78.8% correct responses by first-year BDS students when an assessment was carried out in local vernacular Marathi. The mean value of the correct response rate was the lowest for Hindi, slightly higher for Marathi, and the highest for English (16). In the current study, the communication subscale included the item “ability to speak and understand local language”. Expectedly, knowing Bengali, the local language, significantly improved communication skills.

Improvement in the learning domain was the least in this study; however, it was higher for students staying in the campus hostel. One study reported that medical students living at the hostel had lower stress levels. Among the group staying at the hostel, the academic performance showed a high positive correlation with the General Health Questionnaire-12 score (17). A hostel environment provides the opportunity for peer and near-peer mentoring, which is beneficial for professional development (18). Peer mentoring could be the reason for improved learning in the present study.

Probably less importance given to the improvement of learning style is a lacuna of FC curriculum and needs to be revised. Teaching-learning methods, class size, duration, topic selection, and evaluation of the FC should be planned, keeping in sight that 33 - 52% of students did not show any improvement in different subscales after the FC.

In essence, the present study provides a critical appraisal of an intervention by collecting feedback from the target group and identifying factors associated with a higher probability of improvement. The strength of this study was the assessment of learning in multiple domains and the comparison of improvement with the baseline. However, the self-assessment of learning might not have accurately quantified learning gaps which can be considered a limitation of this study. Test-retest recall bias was another limitation of this study. The right blend of structure and flexibility is yet to be achieved by the fledgling FC. The current study can be helpful for qualitative improvement in the FC and medical education by further extension. The questions not answered by the present study are as follows: (1) Can the skills of communication and ethics learned in controlled conditions be applied to a real-life situation? (2) What will be the rate of deterioration or improvement in skill levels with time? (3) To what extent does the self-rating of learning reflect authentic learning?

These questions can provide avenues for future research.

4.1. Conclusions

The FC at the study institute was effective in the improvement of communication skills, interpersonal skills, and management-career development skills. Improvement in communication skills was associated with knowing the local language. The least improvement was reported in the subscale of learning skills; students living in the campus hostel had comparatively greater improvement in learning skills. Depending on the type of subscale, 33 - 52% of students reported no improvement after the course. Therefore, it can be concluded that overall competency in learning skills showed modest improvement after the FC. Medical colleges can tailor the FC based on a rapid baseline appraisal of students’ skills, and the curriculum should also focus on the improvement of learning skills.