Effect of training on formulation of multiple choice questions: a cross-sectional study amongst faculty in the department of biochemistry of a Medical Institution in India
DOI:
https://doi.org/10.18203/2320-6012.ijrms20162601Keywords:
MCQ, Assessment tool, Medical educationAbstract
Background: Multiple choice questions (MCQs) are an important tool for assessing whether learning objectives are being achieved. A formal training is necessary to have high standards in preparing MCQs. the effect of a training program on the quality of MCQs, amongst the faculty of Department of Biochemistry was assessed.
Methods: A training program was designed with the main focus on how to construct appropriate MCQs’ by participants based on sound scientific standards and guidelines. Topics from biochemistry syllabus for first year MBBS course were selected for the study. All the study participants were newly appointed lecturers and had not undergone any formal training program in medical education technology. Training was carried out by three experts who were formally trained in medical education and were fellows of foundation for advancement of International Medical Education and Research (FAIMER). Knowledge gained by the study participants was tested by pre-test post-test comprising objective questions. The quality of MCQs before and after intervention was assessed with a validated (checklist) containing 60 items.
Results: Eight faculty members participated in the study. A statistically significant improvement was observed in the knowledge of appropriate formulation of MCQs (P<0.05) and the overall quality of MCQs (P=0.004), following training of the study participants. Especially, marked improvements were observed in providing directions for MCQs, highlighting of negative stems in the MCQs and providing keys to MCQs after the training. No significant changes were observed either in the quality of subject content or in the types of formulation of MCQs.
Conclusions: A significant improvement in knowledge of how to construct appropriate MCQ items and the quality of MCQs following training program was noted. Such training program to be administered to the teaching staff early in their career was recommended.
References
Schuwirth L, Vleuten VDC. ABC of learning and teaching in medicine: Written assessment. British Med J. 2003;326:643-5.
Nnodim J. Multiple-choice testing in anatomy. Med Educ. 1992;26:301-9.
Bacon D. Assessing learning outcomes: a comparison of multiple-choice and short answer questions in a marketing context. J Marketing Education. 2003;25:31-6.
Chandratilake M, Davis M, Ponnamperuma G. Assessment of medical knowledge: the pros and cons of using true/false multiple choice questions. National Med J India. 2011;24:225-8.
Case S, Swanson D. Extended-matching items: a practical alternative to free response questions. teaching and learning in medicine. International Journal. 1993;5:107-15.
Athari Z, Sharif M, Nematbakhsh M, Babamohammadi H. Evaluation of critical thinking skills in Isfahan University of medical sciences' students and its relationship with their rank in university entrance exam rank. Iran J Med Edu. 2009;9:5-12.
Kuechler W, Simkin M. How well do multiple choice tests evaluate student understanding in computer programming classes? J Inf Syst Educ. 2003;14:389-400.
Abdulghani H, Ahmad F, Irshad M, Khalil M, Al-Shaikh G, Syed S, et al. Faculty development programs improve the quality of multiple choice questions items' writing. Sci Rep. 2015;5:955-6.
Haladyna T, Downing S, Rodriguez M. A review of multiple choice item-writing guidelines for classroom assessment. Applied Measurement Edu. 2002;15:309-34.
Mchugh M. Interrater reliability: the kappa statistic. Biochemia Medica. 2012;22:276-82.
Wass V, Vleuten VDC, Shatzer J, Jones R. Assessment of clinical competence. Lancet. 2001;357:945-9.
Schuwirth L, Vleuten VDC, Donkers H. A closer look at cueing effects in multiple-choice questions. Med Edu. 1996;30:44-9.
Schuwirth L, Vleuten VDC. Different written assessment methods: what can be said about their strengths and weaknesses? Med Edu. 2004;38:974-9.
Bland C, Schmitz C, Stritter F, Henry R, Aluise J. Successful faculty in academic medicine: essential skills and how to acquire them. 1st edition. New York, NY: Springer Publication; 1990.
Sayyah M, Vakili Z, Alavi NM, Bigdeli M, Soleymani A, Assarian M, et al. An item analysis of written multiple-choice questions: Kashan University of Medical Sciences. Nursing Midwifery Studies. 2012;1:83-7.
Battista D, Kurzawa L. Examination of the quality of multiple-choice items on classroom tests. Canadian J Scholarship Teaching Learning. 2011;2:4.
Sadaf S, Khan S, Ali S. Tips for developing a valid and reliable bank of multiple choice questions (MCQs). Educ Health. 2012;25:195-7.
Fincher R, Simpson D, Mennin S, Rosenfeld G, Rothman A, Mcgrew M, et al. Scholarship in teaching: an imperative for the 21st century. Acad Med. 2000;75:887-94.
Osarumwense H, Oyedeji S. Empirical comparison of methods of establishing item difficulty index of test items using classical test theory (CTT). J Educational Policy Entrepreneurial Res. 2015;2:98-109.
