Email this article Interrater reliability of client categorization system as a psychiatric status rating scale in measuring psychiatric patients’ health status
Abstract
Background: Reliable instrument is needed by nurses to measure patient’s status. This study aimed to determine the interrater reliability of client categorization system instrument used in psychiatric setting.
Methods: Data was collected two periods in a year by twelve raters. They used the CCS to simultaneously gauge the condition of 30 patients for 98 measurements. Data was analyzed using the Cohen’s-Kappa formula and percent agreement.
Results: The analysis showed that the Cohen-Kappa values of interrater reliability were 0.59 which can be categorized as moderate (accepted, if value >0.41) and percent agreement value was 77.78% indicated acceptable.
Conclusions: The CSS has moderate and acceptable category of interrater reliability and can be recommended for use in clinical practice.
Keywords
Full Text:
PDFReferences
Schroder PJ, Washington PW, Deering DC, Ciyne L. Testing validitiy and reliability in a psychiatric patient classification system Psychiatric nurses thest their patient classification system. Nursing Management. 1986;49(1):49-54.
Fries BE, Nerenz DR, Falcon SP, Ashcraft MLF, Lee CZ. A classification system for long-staying psychiatric patients. Medical Care. 1990;28(4):311-23.
Overall JE. The brief psychiatric rating sclae in psychopharmacology research. Modern Problems of Pharmacopsychiat. 1974;7(67-68).
Lachar D, Bailley SE, Rhoades HM, Varner RV. Use of BPRS-A percent change scores to identify significant clinical improvement: accuracy of treatment response classification in acute psychiatric inpatients. Psychiatric research. 1999;89:259-68.
Eklof M, Qu W. Validating a Psychiatric patient Classification System. JONA. 1986;16(5):1986.
Croft A. A psychiatric patient classification system that works! Nursing Management. 1993;24(11):66.
Iglesia C, Villa AM. A system of patient classification in long-term psychiatric inpatient: Resources Utilization Groups T-18 (RUG T-18). 2005.
Morath J, Fleischmann R, Boggs G. A missing consideration: The psychiatric patient classification for scheduling staffing systems. Perspective in Psychiatric Care. 1989;XXV(3, 4):40 - 7.
Ehrman ML. Using a factored patient classification system in psychiatry. Nursing Management. 1987;18(5):48-53.
Ringerman ES, Luz S. A psychiatric patient classification system. Nursing Management. 1990;21(10):66-71.
O'Leary C. A psychiatric patient classification system. Nursing Management. 1991;22(9):66.
Nurjannah I. Treatment guidelines for mental illness patient: management, nursing process and nurse-patient therapeutic relationship. Yogyakarta: Mocomedia. 2004.
Robins E. Categories Versus Dimensiions in Psychiatric Classification. 1976;8:39-55.
van der Vleuten C. Validity of final examinations in undergraduate medical training. BMJ. 2000;321(7270):1217.
McHugh ML. Interrater Reliability: The Kappa Statistic. The Journal of Croatian Society of Medical Biochemistry and Laboratory Medicine. 2012:276-82.
Morris R, MacNella P, Scott A, Treacy P, Hyde A, O'Brien J, et al. Ambiguities and conflicting results: The limitations of the Kappa Statistics in establishing the interrater reliability of the Irish nursing minimum data set for mental health: A discussion paper. Inter J Nursing Studies. 2008;45:645-7.
Craddock J. Interrater reliability of psychomotor skill assessment in athletic training: ProQuest. 2009.
Cargo M, Stankov I, Thomas J, Saini M, Rogers P, Mayo-Wilson E, et al. Development, interrater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider based prevention and treatment progrmas targeting children and youth. Med Res Meth. 2015;15:73.
Morris R, MacNeela P, Scott A, Treacy P, Hyde A, O’Brien J, et al. Ambiguities and conflicting results: The limitations of the kappa statistic in establishing the interrater reliability of the Irish nursing minimum data set for mental health: A discussion paper. Inter J Nursing studies. 2008;45(4):645-7.
Gorsuch RL. Factor Analysis. Hillsdale. NJ: Erlbaum Associates. 1983.
Hatcher L. A step by step approach to using the SAS System for Factor Analysis and Structural Equation Modeling. Cary, N, C: SAS Institute, Inc; 1994.
Stuart GW, Sundeen SJ. Principles and Practice of Psychiatric Nursing. St. Louis: Mosby Year Book; 1995.
Rushforth HE. Objective Structured Clinical Examination (OSCE): Review of Literature and Implications for Nursing Education. Nurse Education Today. 2007:481-90.
McCray. Assessing Interrater Agreement For Nominal Judgement Variables. Language Testing Forum. 2013;15-7.
House A, House B, Campbell M. Measures of inter observer agreement: Calculation formulas and distribution effects. J Behavioral Assessment. 1981;3(1):37-57.
Landis J, Koch G. The measurement of observer agreement for categorical data. biometrics. 1977:159-74.
Osborne JW. Best practices in quantitative methods: Sage. 2008.
McHugh M. Interrater reliability: the kappa statistic. Biochemia Medica. 2012;22(3):276-82.
Parloca PK, Henry SB. The usefulness of Georgetown Home Health Care Classification System for Coding Patient Problems and Nursing Interventions in Psychiatric Home Care. Computers in Nursing. 1998;16(1):45-52.
Feinstein A, Cicchetti D. High agreement but low kappa: I. The problems of two paradoxes. J Clinical Epid. 1990;43(6):543-9.
Nurjannah I, Siwi S, M. Guidelines for analysis on measuring interrater reliability of nursing outcome classification. International Journal of Research in Medical Sciences. 2017;5(4):1169-75.
Neuendorf KA. The Content Analysis Guidebook. Thousand Oaks, CA: Sage; 2002.
Eye AV, Mun EY. Analyzing Rater Agreement: Manifest Variable Methods. New Jersey: Lawrence Erlbaum Associates; 2005.
