Academic member’s knowledge improvement effect on the results of multiple choice questions in residency exams analysis (2009)

Background: Multiple choice questions are used for evaluation of learning in promotion residency exams. Due to holding these exams in a standard way, we conducted this study to evaluate the effect of board members, promotion scholarship who designed the questions on these exams results in 2009.

Methods: A semi experimental research was designed and 105 exam boards of academic members were studied. The results of promotion exams analysis in 2008 plus a tripartite pamphlet for developing standard exams were sent to all. Quantity data was collected by discrimination index, difficulty level, standard deviation and Reliability of test index (KR20), computer report of health ministry analyzing software, and the quality data by percent of Taxonomy II & III questions and report questions from the secretariat of medical educational salinity. Data was analyzed by Even and independent T test and variance analysis.

Results: Data from 11 different academic groups were compared. A statistically significant increase was observed among average of distinction index, exam reliability, percent of questions without structural defects and Taxonomy II & III in 2008 compared to 2009 results (p=0.05). The highest difficulty level was related to questions of urology promotion test 0.54 (p=0.005) and the lowest in psychiatry group 0.69 (p=0.013). The most percent rise in the discrimination index was in the psychiatry group.

Conclusions: This study suggested that medical academic members, scholarship promotion increase by using an educational pamphlet and providing feedback of results from previous exams.