Evaluation of validity and reliability of multiple-choice questions in second MBBS competency-based medical education-based pharmacology examination of medical institute of India

Rushikesh P. Patil; Satish E. Bahekar; Madhuri D. Kulkarni; Mirza S. Baig

doi:10.18203/2320-6012.ijrms20223091

Authors

Rushikesh P. Patil Department of Pharmacology, Government Medical College, Aurangabad, Maharashtra, India
Satish E. Bahekar Department of Pharmacology, Government Medical College, Aurangabad, Maharashtra, India
Madhuri D. Kulkarni Department of Pharmacology, Government Medical College, Aurangabad, Maharashtra, India
Mirza S. Baig Department of Pharmacology, Government Medical College, Aurangabad, Maharashtra, India

DOI:

https://doi.org/10.18203/2320-6012.ijrms20223091

Keywords:

Difficulty index, Discrimination index, Distractor efficiency, Cronbach’s alpha, Item-analysis

Abstract

Background: Multiple choice questions (MCQs) are most commonly used assessment tool in undergraduate medical examination. Assessment method must be reliable and valid. To improve quality of MCQs, item analysis was carried out by determining their validity and reliability using parameters like difficulty index, discrimination index, distractor efficiency and Cronbach’s alpha value.

Methods: Study was carried out among 193 second year medical students. Each student was given 40 MCQs of 1 mark each. After assessment of MCQs, validity of test was analyzed by using difficulty index, discrimination index and distractor efficiency while reliability was analyzed by using Cronbach’s alpha.

Results: Mean ± SD of difficulty index, discrimination index, functioning and non-functioning distractors were 59.80±23.38, 0.25±0.12, 1.98±0.92 and 13.25±13.05 respectively with reliability value of 0.7. About 47.5% items had moderate difficulty index, 22.5% items have excellent discrimination index with 35% items having 100% distractor efficiency. Reliability of test as measured by Cronbach’s alpha value was 0.7. There was weak correlation between difficulty index and discrimination index.

Conclusions: It is concluded from study that given MCQs test have reliability but not validity and needs to improve quality of MCQs. Validity of test is improved by improving difficulty index, discrimination index, distractor efficiency of items.

Metrics

Metrics Loading ...

References

Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR et al. Competency-based medical education: theory to practice. Med Teacher. 2010;32(8):638-45.

Shah N, Desai C, Jorwekar G, Badyal D, Singh T. Competency-based medical education: An overview and application in pharmacology. Ind J Pharmacol. 2016;48(1):S5.

Medical Council of India. Assessment Module for Undergraduate Medical Education Training Program. 2019;1-29.

Al-Rukban MO. Guidelines for the construction of multiple choice questions tests. J Family Community Med. 2006;13(3):125.

Amin Z, Khoo HE. Basics in medical education. World Scientific. 2003.

Brown GT, Abdulnabi HH. Evaluating the quality of higher education instructor-constructed multiple-choice tests: Impact on student grades. In Frontiers in Education. 2017;2:24.

Moses T. A review of developments and applications in item analysis. Advancing Human Assessment. 2017;19-46.

Bichi AA. Classical Test Theory: an introduction to linear modeling approach to test and item analysis. Int J Social Studies. 2016;2(9):27-33.

Mehta G, Mokhasi V. Item analysis of multiple choice questions-an assessment of the assessment tool. Int J Health Sci Res. 2014;4(7):197-202.

Rudolph MJ, Daugherty KK, Ray ME, Shuford VP, Lebovitz L, DiVall MV. Best practices related to examination item construction and post-hoc review. Am J Pharmaceutical Educ. 2019;1;83(7).

Sood R, Singh T. Assessment in medical education: Evolving perspectives and contemporary trends. Natl Med J India. 2012;25(6):357-64.

Brady AM. Assessment of learning with multiple-choice questions. Nurse Educ Pract. 2005;5(4):238-42.

Rao C, Prasad HK, Sajitha K, Permi H, Shetty J. Item analysis of multiple choice questions: Assessing an assessment tool in medical students. Int J Educational Psychological Res. 2016;2(4):201.

Namdeo SK, Sahoo B. Item analysis of multiple choice questions from an assessment of medical students in Bhubaneswar, India. Int J Res Med Sci. 2016;4(5):1716-9.

Hingorjo MR, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. J Pak Med Asso. 2012;62(2):142.

Burud I, Nagandla K, Agarwal P. Impact of distractors in item analysis of multiple choice questions. Int J Res Med Sci. 2019;7(4):1136-9.

Kolte V. Item analysis of multiple choice questions in physiology examination. Indian J Basic Applied Med Res. 2015;4(4):320-6.

D'Sa JL, Visbal-Dionaldo ML. Analysis of Multiple Choice Questions: Item Difficulty, Discrimination Index and Distractor Efficiency. Int J Nursing Educ. 2017;9(3).

Alsubait T, Parsia B, Sattler U. A similarity-based theory of controlling MCQ difficulty. In 2013 second international conference on e-learning and e-technologies in education (ICEEE). 2013;283-8.

Velou MS, Ahila E. Refine the multiple-choice questions tool with item analysis. Int Arch Integrated Med. 2020;7(8):80-5.

Tangianu F, Mazzone A, Berti F, Pinna G, Bortolotti I, Colombo F et al. Are multiple-choice questions a good tool for the assessment of clinical competence in Internal Medicine? Italian J Med. 2018;20;12(2):88-96.