Inter-rater and intra-rater agreement in causality assessment of adverse drug reactions: a comparative study of WHO-UMC versus Naranjo scale

Sushil Sharma; Ajay Kumar Gupta; G. Jyothi Reddy

doi:10.18203/2320-6012.ijrms20174564

Authors

Sushil Sharma Department of Pharmacology, Armed Forces Medical College, (AFMC) Pune-40, Maharashtra, India
Ajay Kumar Gupta Department of Pharmacology, Armed Forces Medical College, (AFMC) Pune-40, Maharashtra, India
G. Jyothi Reddy Technical Associate (PvPI) ADR Monitoring Centre, AFMC Pune-40, Maharashtra, India

DOI:

https://doi.org/10.18203/2320-6012.ijrms20174564

Keywords:

Causality assessment, Naranjo scale, Rater agreement, WHO-UMC scale

Abstract

Background: The causality assessment system proposed by the WHO collaborating centre for international drug monitoring, the Uppsala monitoring centre (WHO-UMC) and the Naranjo probability scale are the generally accepted and most widely used methods for causality assessment. Both these scales are structured, transparent, consistent, and easy to apply with distinct advantages. The PvPI recommends use of WHO-UMC scale while many clinicians prefer Naranjo scale for its simplicity. As both these scales are used very widely in practise, it is important to study the level of agreement among them, to ensure proper interpretation of the causality assessment of the drugs. Objective was to study the inter-rater and intra-rater agreement in causality assessment of adverse drug reactions between WHO-UMC and Naranjo scale.

Methods: A total of 200 cases were analyzed by three raters. Each ADR reporting form was analyzed by the raters independently using both the Naranjo scale and the WHO-UMC scale. The inter-rater and Intra-rater agreement of all the three raters was analyzed using kappa statistics.

Results: The most common category of causality assessment was ‘possible’ which was around 73%, while the probable, definite and unlikely accounted for 23%, 3% and 1% respectively. The inter-rater agreement for the various categories of causality assessment when using Naranjo scale ranged from “very good to excellent” (Kappa value 0.95) while the same ranged from “good to very good” when the WHO-UMC scale was used (Kappa value 0.89) while the Intra-rater agreement for the three raters ranged from good to very good. The mean time (in minutes) taken for assessing the suspected adverse drug an ADR was 8. 26±2.53 for WHO-UMC scale while it was 14.18±3.44 when Naranjo scale was used.

Conclusions: Both the WHO-UMC scale and Naranjo scale are reliable and valid tools for causality assessment but the Inter-rater agreement was slightly better with Naranjo as compared to WHO-UMC scale.

Metrics

Metrics Loading ...

References

Patel TK, Patel PB. Incidence of adverse drug reactions in Indian hospitals: a systematic review of prospective studies. Curr Drug Saf United Arab Emirates. 2016;11(2):128-36.

Bouvy JC, De Bruin ML, Koopmanschap MA. Epidemiology of adverse drug reactions in Europe: a review of recent observational studies. Drug Saf New Zealand. 2015;38(5):437-53.

Pharmacovigilance programme of India (Internet). Performance Report 2015-16. 2016. Available at http://www.ipc.gov.in/PvPI/pub/Pharmacovigilance%20Programme%20of%20India%20Annual%20performance%20Report%202015-2016.pdf.

Agbabiaka TB, Savovic J, Ernst E. Methods for causality assessment of adverse drug reactions: a systematic review. Drug Saf. 2008;31(1):21-37.

Belhekar MN, Taur SR, Munshi RP. A study of agreement between the Naranjo algorithm and WHO-UMC criteria for causality assessment of adverse drug reactions. Indian J Pharmacol. 2014;46(1):117-20.

Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981;30(2):239-45.

Son MK, Lee YW, Jung HY, Yi SW, Lee KH, Kim SU, et al. Comparison of the Naranjo and WHO-Uppsala monitoring centre criteria for causality assessment of adverse drug reactions. Korean J Med. 2008;74:181-7.

Rehan HS, Chopra D, Kakkar AK. Causality assessment of spontaneously reported adverse drug events: comparison of WHO-UMC criteria and Naranjo probability scale. Int J Risk Saf Med. 2007;19:223-7.

Mittal N, Gupta MC. Comparison of agreement and rational uses of the WHO and Naranjo adverse event causality assessment tools. J Pharmacol Pharmacother. 2015;6:91-3.

García-Cortes M, Lucena MI, Pachkoria K, Borraz Y, Hidalgo R, Andrade RJ. Evaluation of Naranjo adverse drug reactions probability scale in causality assessment of drug-induced liver injury. Aliment Pharmacol Ther. 2008;27(9):780-9.

Byrt T. How good is that agreement? Epidemiol. 1996;7:561.

McHugh M. Interrater reliability: the kappa statistic. Biochem Medica. 2012;22(3):276-82.

Viera AJ, Garrett JM. Understanding interobserver agreement: The Kappa statistic. Fam Med. 2005;37(5):360-3.

Sharma M, Gupta SK, Gupta VB, Chatterjee A. A comparative study of causality assessment scales used in the analysis of spontaneously reported events: WHO-UMC criteria versus Naranjo probability scale. J Pharmacovig Drug Safety. 2009;6(1):5-9.