Abstract
Background:
Assessment and evaluation should be a continuous activity associated with curriculum development. Continuous formative examinations are the stepping stone for good learning and better performance in summative examination. However, the correlation or predictability of formative examinations has not been quantified in the community medicine subject in the bachelor of surgery (MBBS) course in India.Objectives:
The objective of this study was to explore the correlation of performance in continuous assessments and final summative evaluation in third-year professional MBBS students in community medicine and the predictability of formative scores for the final examination marks.Methods:
An institution-based retrospective longitudinal study was conducted in the Medical College of Kolkata, India. A total of 237 undergraduate medical students were followed up for two years to track their results in three successive examinations. The scores obtained in formative and summative evaluations in third-year professional MBBS students in community medicine were analyzed to find out the correlation between them, and multiple linear regression was also performed in SPSS.Results:
The strong and significant correlation (0.64) between the sixth semester and internal examination results suggests the importance of the sixth semester examination for the preparation of finals. The third semester examination, although conducted two years prior to the finals, still had significant correlation with the finals, but the correlation was weak (0.17).Conclusions:
Formative examination in community medicine, which spans for overall three and half years, plays an essential role in the preparation of students for the finals, more so for achieving good scores. Performance in these two examinations can only explain more than one fourth of the variability of the performance in the finals.Keywords
Summative Assessment Formative Assessment Community Medicine
1. Background
Throughout the history of education, different examination formats have been applied for the measurement of academic performance (1). After the 1940s, a universal method of testing for admission into universities was introduced, (1) where it was decided that after acknowledging some predefined academic credentials, students’ admission to university will be granted. Thus, school educators were enabled to address curricular concerns and prepare students to meet college admission requirements (2). Though admission criteria were set, but the nature of pre-and post-admission evaluations was still in the nascent stage and needed a structural implementation. Educational systems underwent vast changes during the 1980s. The tests became known as “performance assessments”. Since then, performance was judged not only on the basis of written tests but also oral, practical, and hands-on performance catering to all the three domains of education, namely cognitive, assertive, and psychomotor.
Education in India has always emphasized on academic achievement in school. Successful students are often measured by the score received in state or national higher secondary (HS) examination. However, for technical courses like MBBS, it is not sufficient, and students have to pass a national-level competitive examination.
The Ministry of Human Resource Development (MHRD) has established the National Testing Agency (NTA) as an independent organization for conducting efficient, transparent, and international standard tests to assess the competency of candidates for admission to MBBS/BDS courses in Medical/Dental Colleges in India. Students with a minimum score of 50% in Biology, physics, and mathematics are allowed to take this exam. According to their score, a merit list is published by the testing agency, and students are admitted to various medical colleges.
After such a competitive entrance examination, it is expected that all examinees will excel in their MBBS evaluations, but in practice, that is not the scenario. One of the reasons is that the entrance examination is based on one terminal evaluation, whereas assessment in MBBS is a continuous activity designed simultaneously with curriculum development (3). A well-designed system of assessment containing both formative and summative examinations is a powerful educational device (4). In 1997, the Medical Council of India (MCI) announced the assessment pattern of medical graduates (5). The periodic updating of a curriculum is necessary with a proper assessment pattern of students.
A continuous formative assessment and evaluation system is very important for a healthy growth of learning among students (6). It allows the student to evaluate their progress by judging their periodic performance, clarify their doubts and previous mistakes of the formative assessment, and obtain guidance for improvement in the final summative examination (7). Though it is known that formative examinations help in summative examination, but how much these two types of examination performances are correlated varies in different subjects. It is also not quite quantified whether we can predict the marks in summative examination statistically with the help of formative examination marks.
2. Objectives
This study was conducted to explore whether performance in the continuous assessment method as determined by formative assessment correlates with the final summative evaluation in third-year professional MBBS students in community medicine. This study also aimed to propose a formula to predict the summative examination marks with the help of marks obtained in formative examinations in community medicine.
3. Methods
An institution-based retrospective longitudinal study was carried out in the Medical College of Kolkata, West Bengal, India, for the period of two years from February 2018 to January 2020.
Formative and summative evaluations of 252 undergraduate medical students were followed up for two years to track their results in three successive examinations. The first formative examination was held in their third semester, the second one in the sixth semester, and the third examination on in the seventh semester, which is also considered as a summative examination.
Results of the formative examinations were available in the Department of Community Medicine. Seven students who missed either of the two abovementioned formative examinations were considered as defaulter and excluded from the study. The remaining 245 students were contacted for their final university examination results after it was published. Overall, 237 of them informed us of their score, and they consisted the final sample size.
Data was analyzed to find out the predictors of both passing the examination and securing honours marks, which were 50% and 75%, respectively. Multiple linear and logistic regression were used for this purpose in SPSS. Paired t-test was run to find out improvements over the years, and correlations between marks obtained in these three examinations were also calculated to find out the effectiveness of the medical education program in the institute.
3.1. Ethical Consideration
The study was conducted after obtaining consent from the examinees who were the study subjects and being approved by the independent ethics committee of Medical College, Kolkata.
4. Results
It was found out that from the third semester to final examination, the mean of obtained marks increased, except for the sixth semester where it slightly decreased from the third semester. In case of standard deviation, it decreased steadily from 14.38 in the third semester to 8.25 in the final examination. Whereas the percentage of maximum marks obtained was quite similar in all the examinations, the minimum obtained marks increased from 10% in the third semester to 38% in the final examination.
From the third semester examination to the final examination, the percentages of passed students were 73.1%, 69.7%, and 88.4%, respectively, whereas students who secured honours marks were 5%, 2.1%, and 4.5% in these three examinations, respectively. Analysis of the marks in the three examinations showed the mean value and measures of dispersion as described in Table 1.
Measures of Central Tendency and Dispersion of Scores Obtained in the Third Semester, Sixth Semester, and Final Examinations (N = 237)
| Third Semester Formative | Sixth Semester Formative | Summative | |
|---|---|---|---|
| Mean | 57.38 | 55.59 | 60.43 |
| Median | 60 | 55 | 61 |
| Mode | 67 | 53 | 64 |
| Standard deviation | 14.38 | 10.23 | 8.25 |
| Minimum | 10.00 | 25.00 | 38.00 |
| Maximum | 85.00 | 82.00 | 83.00 |
When the marks are plotted in frequency distribution curve, it is evident that all the three examination scores followed normal distribution with the peak of the curve going left to right from the sixth, seventh, and third semester results, respectively, but looking at the widths of the curves, it is evident that dispersion is highest in the third and least in the final examination (Figure 1).
Multiple frequency curve showing the distribution of study subjects according to marks obtained in the third semester, sixth semester, and final examinations (n = 237)
Paired t-test between the scores of the third semester and sixth semester showed no significant change (P = 0.1), but it revealed a dip in mean score. Final summative examination score, however, displayed significant improvement both from the third semester score (P = 0.003) and sixth semester examination score (P < 0.001).
One-way repeated measure analysis of variance (ANOVA) was conducted to find out whether over time there was any significant change of score obtained in the third, sixth, and final evaluations. The results of ANOVA demonstrated a significant time effect, Wilk’s lambda = 0.72, F = 44.79, P < 0.001, n2 = 0.27.
Further comparison indicated a significant difference in score between final examination and third semester results (P = 0.004). Summative and sixth semester scores showed a significant difference (P < 0.001), but no significant difference could be elicited between the third and sixth semester results (P = 0.287).
Pearson’s correlation coefficient showed a weak correlation between the third semester results and the sixth semester results (r = 0.21) and between the third semester and final examination score (r = 0.17). Both of these correlations were found to be significant. A strong correlation was found between the sixth semester score and the final score (0.64), which was also highly significant (P < 0.001).
A multiple-grid scatter diagram (Figure 2) is drawn below to show the correlation pattern between the three examination scores mentioned above.
Multiple-grid scatter diagram showing correlation between examination scores of the third semester, sixth semester, and final examinations (n = 237)
When a model is formed by multiple linear regression using the third semester and sixth semester marks as predictors (x1, x2) and final examination marks as outcome variable (y), the model is found to be like this,
y = 30.97 + 0.055 × x1 + 0.625 × x2
This model could explain 40% of the variability in the final marks (adjusted R2 = 0.4). Regression coefficients found in this model showed that though the sixth semester marks were a significant predictor of final marks (P < 0.001), but the third semester marks were not (P = 0.3).
A multiple logistic regression model was devised with passing in the final examination as the outcome variable, and the third and sixth semester passing as predictor variables. Neglekar’s R2 was found to be 0.272, with both of these factors significantly predicted passing in finals. Whereas passing in the third semester increased the chance of passing in finals by almost four times (OR = 4.03 & 95% CI = 1.66 to 9.81), passing in the sixth semester increased it by 7.7 times (OR. = 7.79 & 95% CI = 3.04 to 19.36).
Another similar multiple logistic regression model was devised with honours marks in the final examination as the outcome variable and the third & sixth semester honours as predictor variables. Neglekar’s R2 was found to be 0.178. Here, it was noted that although honours in the sixth semester significantly predicted (OR. = 39.7 & 95% CI. = 5.8 to 272.1) honours in final examinations, but the third semester honours did not.
5. Discussion
MBBS course in India takes four and half years, which is divided into nine semesters. Community medicine is taught from the first semester to sixth semester though most of the topics are covered in the sixth and seventh semesters. After three semesters, the first formative examination is conducted. There are no classes and examinations of community medicine in the fourth and fifth semesters, which start again in the sixth semester.
The third semester examination is based on Module-I, which basically consists of ‘Man and Medicine’, ‘Concept of Health and Diseases’, ‘Principles of Epidemiology’ and ‘Epidemiology of Communicable Diseases’. This examination is held in mid-June.
The sixth semester examination is held in mid-September with Module-II, which consists of ‘Screening of Disease, ‘Epidemiology of non-Communicable Diseases, ‘Health Programmes in India’, ‘Essential Medicine’, ‘Preventive Medicine in Gynaecology & Obstetrics, Paediatrics & Geriatrics’, ‘Nutrition and Health’, ‘Medicine and Social Science’, ‘Environment and Health’, ‘Hospital Waste Management’, and ‘Health information and Basic Medical Statics’.
The final examination after the seventh semester is conducted with the syllabus of Module-I, Module-II, & Module-III. The Module-III consists of ‘Disaster Management’, ‘Occupational Health’, ‘Genetics’, ‘Mental Health’, ‘Health Communication & Health Education’, ‘Health Planning & Management’, and ‘Health care of the Community’.
In the final third professional examination of community medicine, there are two theory papers. Each paper contains questions of 60 marks, and the practical examination comprises of one viva table, one table for ‘Family Study’, one for ‘Project Study’, one for ‘Epidemiological & Statistical Problem’, and one for ‘Clinical Problem’. In the examinations, a student has to secure 50% in both theory and practical examinations to get passed. Someone who secures 75% marks is considered to have honours in the subject.
In this study, it was seen that the mean score obtained by students in the sixth semester was less than that of the third semester, which apparently seems spurious, but there are some factors that may contribute to this. The third semester examination is taken on smaller syllabus, and when this examination is conducted students are used to continuous teaching of one and half years of community medicine classes on relatively smaller number of chapters. Also, they only study three other subjects, namely ‘Anatomy’, ‘Physiology’, and ‘Biochemistry’, during most of this tenure. In the sixth semester, there has been a one-year gap of community medicine classes (4th & 5th semester), which breaks the flow; in the sixth semester apart from ‘Community Medicine’, they also have to study ‘Ophthalmology’, ‘Otorhinolaryngology’, ‘General Medicine’, ‘General surgery’, ‘Paediatrics’, and ‘Gynaecology’. All of these subjects are practical, and apart from regular lecture and demonstration classes, they have to visit wards to take case histories of patients regularly. Evidently, in this time period, their load of study is much higher than that of the third semester, and the sixth semester examination is taken on a bigger syllabus than that of the third semester.
Although the final examination is taken on complete syllabus, a significant improvement in scores is seen from the third semester. The standard deviation of scores decreased from the third to sixth semester, and it is the least in finals, which suggests that although during the initial stages of community medicine classes some students do exceptionally good and some are quite bad, but as time passes, others catch up, and by the time of final examinations, the level of learning of students is more condensed centrally.
The strong and significant correlation (0.64) between sixth semester and final examination results suggests the importance of sixth semester examination for the preparation of finals. The third semester examination, though conducted two years prior to the finals, still has significant correlation with the finals, but this correlation is weak (0.17).
Linear regression results point out a weak regression coefficient (0.055) for the third semester marks but a quite strong and significant regression coefficient (0.625) for the sixth semester marks. This model could explain 40% of the variability in the final marks. Logistic regression also suggests the significant role of third and sixth semester passing in predicting whether someone passes the finals. This clearly shows the importance of these formative examinations for the passing of students in the finals. The importance of sixth semester increases many folds when honours mark (> 75%) is taken as the dependent variable. There is almost 40 times more chance of getting honours marks in finals if someone gets > 75% in the sixth semester.
Previous studies conducted on a similar topic showed the predictive power of performance in final formative examination for marks obtained in summative university examination in pharmacology, (8, 9), but this study extends this scope further and concludes that even other formative examinations like those in the third and sixth semesters have a good predictive power for summative examination performance. This study also concluded that many other factors play a role in the final performance, our study also found that about 27% of variability of passing final examination was explained by the model constructed in this study (10-12).
A systematic review performed by Ahmady et al (13) found out that factors related to academic performance in medical institutions were many fold, like personal causes, learning styles, personality traits, motivational strategies and self-efficacy, quality of sleep, stress, coping strategies, etc., and they should be addressed early to find out problematic learners. As the present study showed a significant correlation between the initial formative examinations in earlier semesters and final formative examination at the end, these personal contributing factors for examination performance should be searched for after the earlier semester results so that they can be addressed to improve performance in the final examination.
Various studies have pointed out institution and course-related factors for academic performance, like instructional design, teaching strategies, course assessments, course structure, critical thinking, blended learning, predictive courses, admission tests, learning environment, curriculum planning, and pre-matriculation program (14-19). Thus, once formative examination results are out, institutional strategic review can also be planned if the overall performance of a batch is not satisfactory. It can help improve the academic performance of the batch in the final examination.
This very thing is one of the key limitations of the study, as it is a retrospective record-based study due to anonymity, there was no scope to elicit other factors like school performance, performance, ranks in MBBS entrance examination, in preclinical subjects, and demographic factors of the students as done in other studies conducted in the USA (20), Ethiopia (21), Pakistan (22), etc. Although these same factors would have acted upon the formative examination performances, this effect may be neutralized to some extent when we are comparing three examinations of the same individual. A continuous evaluation that takes place as a form of yearlong viva examination could not be taken into account, as it forms a part of total final marks in community medicine, so they will obviously be correlated. This study has also devised a formula to predict score in the finals on the basis of the other two examinations, which may differ for other colleges as this study is longitudinally focused on one batch of students in one college.
5.1. Conclusions
Formative examinations in community medicine, which spans for overall three and half years, play a critical role in the preparation of students for the finals, more so for achieving good scores like honors marks. Performance in these two examinations can only explain more than one-fourth of the variability of the performance in the finals.
References
-
1.
Stiggins RJ. Relevant Classroom Assessment Training for Teachers. Educ Meas. 1991;10(1):7-12. https://doi.org/10.1111/j.1745-3992.1991.tb00171.x.
-
2.
Giordano G. How testing came to dominate American schools: The history of educational assessment. Peter Lang; 2005. 314 p.
-
3.
Kumar M, Agarwal V, Tiwari R. Effect of Active Learning on Teaching Physiology to Undergraduate Physiotherapy Students. Indian J Physiother Occup Ther. 2008;2(4).
-
4.
Pai MRSM, Sanji N, Pai PG, Kotian S. Comparative assessment in Pharmacology multiple choice questions versus essay with focus on gender differences. J Clin Diagn Res Ser Online. 2010;4(3):2515-20.
-
5.
Zayapragassarazan Z, Kumar S, Kadambari D, Dinesh Kumar V. Effective Educational Practices in Health Profession Education (A Collection of Educational Projects completed by NTTC Alumni from 2006 to 2013). Online Submission. 2020.
-
6.
Nnodim JO. Multiple‐choice testing in anatomy. Med Educ. 1992;26(4):301-9.
-
7.
Day SC, Norcini JJ, Diserens D, Cebul RD, Schwartz JS, Beck LH, et al. The validity of an essay test of clinical judgment. Acad Med. 1990;65(9 Suppl):39. [PubMed ID: 2400498]. https://doi.org/10.1097/00001888-199009000-00034.
-
8.
Santra R, Pramanik S, Mandal A, Sengupta P, Das N, Raychaudhuri P. A Study on the Performance of Medical Students in Internal Assessment and its Correlates to Final Examinations of 2(nd) MBBS Pharmacology Curriculum in a Medical College of Eastern India. J Clin Diagn Res. 2014;8(12). [PubMed ID: 25653964]. [PubMed Central ID: PMC4316270]. https://doi.org/10.7860/JCDR/2014/8318.5227.
-
9.
Yaqoob N, Akram Bhatti S, Shahid Javed M. Send-up Exam Performance; A Predictor of Scores in Annual Exam for Preclinical Medical Students. Professional Med J. 2015;22(2):159-62.
-
10.
Oyebola DD, Adewoye OE, Iyaniwura JO, Alada AR, Fasanmade AA, Raji Y. A comparative study of students' performance in preclinical physiology assessed by multiple choice and short essay questions. Afr J Med Med Sci. 2000;29(3-4):201-5.
-
11.
Moqattash S, Harris PF, Gumaa KA, Abu-Hijleh MF. Assessment of basic medical sciences in an integrated systems-based curriculum. Clin Anat. 1995;8(2):139-47. [PubMed ID: 7712326]. https://doi.org/10.1002/ca.980080211.
-
12.
Mujeeb AM, Pardeshi ML, Ghongane BB. Comparative assessment of multiple choice questions versus short essay questions in pharmacology examinations. Indian J Med Sci. 2010;64(3):118-24. [PubMed ID: 22569324]. https://doi.org/10.4103/0019-5359.95934.
-
13.
Ahmady S, Khajeali N, Sharifi F, Mirmoghtadaei ZS. Factors related to academic failure in preclinical medical education: A systematic review. J Adv Med Educ Prof. 2019;7(2):74.
-
14.
Bastías S G, Villarroel del P L, Zuñiga P D, Marshall R G, Velasco F N, Mena C B. [Academic performance of medical students: a predictable result?]. Rev Med Chil. 2000;128(6):671–8. Spanish. https://doi.org/10.4067/s0034-98872000000600015.
-
15.
Wayne SJ, Fortner SA, Kitzes JA, Timm C, Kalishman S. Cause or effect? The relationship between student perception of the medical school learning environment and academic performance on USMLE Step 1. Med Teach. 2013;35(5):376-80. https://doi.org/10.3109/0142159x.2013.769678.
-
16.
Iputo JE, Kwizera E. Problem-based learning improves the academic performance of medical students in South Africa. Med Educ. 2005;39(4):388-93. [PubMed ID: 15813761]. https://doi.org/10.1111/j.1365-2929.2005.02106.x.
-
17.
Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. The impact of team-based learning on medical students' academic performance. Acad Med J Assoc Am Med Coll. 2010;85(11):1739-45. [PubMed ID: 20881827]. https://doi.org/10.1097/ACM.0b013e3181f52bed.
-
18.
Poljicanin A, Caric A, Vilovic K, Kosta V, Marinovic Guic M, Aljinovic J, et al. Daily mini quizzes as means for improving student performance in anatomy course. Croat Med J. 2009;50(1):55-60. [PubMed ID: 19260145]. [PubMed Central ID: PMC2657562]. https://doi.org/10.3325/cmj.2009.50.55.
-
19.
Schripsema NR, van Trigt AM, Borleffs JC, Cohen-Schotanus J. Selection and study performance: comparing three admission processes within one medical school. Med Educ. 2014;48(12):1201-10. https://doi.org/10.1111/medu.12537.
-
20.
De Ball S, Sullivan K, Horine J, Duncan WK, Replogle W. The relationship of performance on the dental admission test and performance on Part I of the National Board Dental Examinations. J Dent Educ. 2002;66(4):478-84. [PubMed ID: 12014563].
-
21.
Robi YS. University Entrance Exam Result and Preparatory Class Average Score as Predictors of College Performance. Int J Indian Psychol. 2014;1(4). https://doi.org/10.25215/0104.013.
-
22.
Huda N, Dosa TI, Alam E, Agha S. Selection procedure as predictor of performance in university examination. J Pak Med Assoc. 2001;51(11):381-4. [PubMed ID: 11840603].