Pyry A.J. Välitalo(1), Vivi Rottschäfer(2), Monique van Dijk(3), Catherijne A.J. Knibbe(1,4), Elke H.J. Krekels(1)
(1) Division of Pharmacology, Leiden Academic Centre for Drug Research, Leiden University, Leiden, the Netherlands; (2) Mathematical Institute, Leiden University, Leiden, the Netherlands; (3) Department of Pediatric Surgery and Pediatrics, Erasmus MC-Sophia Children’s hospital, Rotterdam; (4) Department of Clinical Pharmacy, St. Antonius Hospital, Nieuwegein, the Netherlands
Objectives: A composite score of observational scales with multiple items is often used to quantify disease severity. Graded response models (GRM) are a subset of item response theory (IRT) models, where the grades of an item are considered ordered categorical data[1]. In an IRT context, adding items to a test always leads to increased test information about the latent variable. The same may not be true for the total score approach, and total scores may become less informative as the variance of discrimination parameters increases[2]. The objectives of this study were: 1) to derive an equation for total score information, based on GRM parameters and 2) to systematically compare the informativeness of total score approach in various scenarios with simulated and real clinical data on pain in children.
Methods: The information of a continuous variable can be calculated using the first derivative of the variable with respect to a given parameter, and the variance of the variable[3]. This was used as the starting point for the development of the equation for total score information. Subsequently, the test information from total scores approach was compared to test information from GRM modeling approach in a set of 504 simulation scenarios. For the real clinical data, GRM parameters were retrieved from a previous study quantifying the informativeness of items in the COMFORT scale[4]. This is a multi-item observational scale to assess pain in neonates. The contribution of each COMFORT item to the total COMFORT score was evaluated.
Results: The test total score was transformed to a continuous variable, upon which the test information of the total score could be calculated[3]. In simulated scenarios, the total score was always less informative (by 1-39%) than an IRT analysis of the same items. If the GRM parameters were identical for each item, the information of the total score was 96-99% of the IRT test information. If the GRM parameters differed between items, the total score information decreased relative to the IRT test information. The case study showed that COMFORT scale items “Heart rate”, and “Blood pressure” decrease the informativeness of the total COMFORT score. Moreover, the influence of COMFORT item “Respiratory response” on the total score informativeness was minimal.
Conclusions: The equation can be used to assess the impact of individual items on the overall informativeness of clinical observational scales, enabling the optimization of such scales by exclusion of uninformative items.
References:
[1] Samejima F. Psychometrika 1970;35:139–139.
[2] Schindler E. et al. PAGE 24 (2015) Abstr 3468.
[3] Lehmann E. Theory of Point Estimation. John Wiley & Sons, 1983.
[4] Välitalo P. et al. Pain 2016 Jan 5 [epub ahead of print]
Reference: PAGE 25 () Abstr 5712 [www.page-meeting.org/?abstract=5712]
Poster: Methodology - New Modelling Approaches