Are we assessing them right? A study on the content validity of undergraduate pathology question papers and role of blueprinting in assessment
Keywords:Assessment, Blueprinting, Weightage
Background: Assessment forms a very essential and critical part of the learning process in medical education. Written examinations are widely used to test the knowledge component of learning. For a written examination to be valid, it should match the contents of the course and should provide proportional weightage to each of the content. A periodic evaluation of question papers to examine the weightage of each topic and the use of blue printing is necessary to ensure the validity of the written assessments.
Methods: The ten year question papers (2007-2016) of II MBBS summative written examinations in Pathology conducted by Rajasthan University of Health Sciences (RUHS) were analysed and the appropriateness of weightage given to content areas was examined in relation to the syllabus.
Results: The syllabus distribution in Paper 1 and Paper 2 was highly skewed with extreme concentration of topics in Paper 1 (due to clubbing of both general and systemic pathology) which led to under-representation of many topics despite their high Impact and Frequency. There was also evidence of paper setter’s bias leading to disproportionate (over/under) representation of many topics. An ideal blueprint for paper 1 and 2 was prepared with appropriate syllabus distribution and allotment of marks as per weightage of each topic according to its impact and frequency score.
Conclusions: Frequent analysis should be carried out to provide feedback and to ensure that the assessments are aligned with the learning objectives. A blueprint is a vital component and helps us to plan written assessments in a rational and balanced manner.
Abbatt FR. Teaching for better learning; a guide for teachers of primary health care staff, Geneva; World Health Organization. 1980:107-9.
Robin G, Dhiraj S, Sushila S, Neha D. Analytical study of written examination papers of undergraduate Anatomy; Focus on its content validity. IJBAMR. 2013;2(8):1110-6.
Ronald ME. Assessment in Medical Education. N Eng J Med. 2007;356:387-96.
Narvekar RS, Bhandare NN, Bhandare PN. Analysis of undergraduate pharmacology question papers at Goa medical college as regards to their content areas. IntJ Sci Rep. 2016;2(8):182-6.
Edward JP, Peter GD. Assessment of higher order cognitive skills in undergraduate education; modified essay or multiple choice questions? Research paper. BMC Medical Education. 2007;7:49.
Adkoli BV. Attributes of a good question paper. In: Sood.R, (Chief Editor ) Assessment in medical education trends & tools. K.L. Wig Centre for Medical Education and Technology, All India Institute of Medical Sciences, Ansari Nagar, New Delhi. 1995:67-84.
Downing SM. Validity: on the meaningful interpretation of assessment data Medical Education. 2003;37:830-37.
Shumway JH, Harden RM. Amce guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach. 2003;25(6):569-84.
Mc Aleer S. Choosing assessment instuments. In: Dent JA , Harden RM, (Editors ). A practical guide for medical teachers, Edinbergh: Churchill Livingstone. 2001;303-13.
Adkoli BV, Deepak KK. Blueprinting in assessment. In: Tejinder S, Anshu, editors.Principles of assessment in medical education , India : Jaypee Brothers Medical Publishers. 2012:205-13.
Patil SY, Gosavi M, Bannur HB, Ratnakar A. Blueprinting in assessment: A tool to increase the validity of undergraduate written examinations in Pathology. Int J App Basic Med Res. 2015;5:S76-9.
Syllabus: M.B.B.S, Scheme of examination and courses of study, University of Rajasthan, Jaipur. 2006-2007.
Tejinder S. Basics of assessment. What to assess? In: Tejinder S, Anshu, editors. Principles of assessment in medical education, India: Jaypee Brothers Medical Publishers. 2012:1-24.
Hamdy H. Blueprinting in medical education. N Engl J Med. 2007;356:387-95.
Downing SM, Haladyna TM. Validity and its threats. In: Downing SM, Yudkowsky R, editors. Assessment in Health Professions Education. New York: Routledge. 2009:21-56.