INNOVATIONS in pharmacy

Imaginary Worlds: The Status of Modeled Economic Evaluation Claims Published in Value in Health January 2015 to December 2015

Paul C Langley

University of Minnesota

Taeho Greg Rhee

University of Minnesota, Twin Cities

DOI: https://doi.org/10.24926/iip.v7i2.441

Keywords: Value in Health, pseudoscience, cost-effectiveness modeling, credibility, imaginary worlds, scientific method

---

Abstract

The purpose of this paper is to assess the extent to which modeled or simulated cost-effectiveness claims published in Value in Health in 2015 meet the standards of normal science. To meet these standards, modeled or simulated claims must be credible. They must be capable of empirical evaluation and replication. If these standards are not met then such claims run the risk of being labeled as pseudoscience. Following a systematic review of all publications in Value in Health, 16 papers were identified. Of these 14 presented a cost-per-QALY analysis, with 9 presenting their claims for comparative effectiveness in a lifetime cost-per-QALY framework. With the focus of the assessment on whether or not these studies generated testable claims, none of the studies met this standard. They were best seen as thought experiments or imaginary worlds. Recipients of such claims can of course, on the one hand, reject them outright as not meeting accepted standards in normal science. After all, QALYs are never collected by health care systems and are unlikely to be collected. This means that the outcome metrics are untestable and may never have been intended to be tested. On the other hand, if the recipient believes that the model or simulation provides a sufficient correspondence to reality then the claims made are necessarily entailed. The issue of testing is irrelevant to the belief in the credibility of the claims. The modeled or simulated claims are immune to failure. The review concluded that none of the claims presented were in a testable form and that while 7 (at most) of the studies had the potential to generate testable claims, the rest were immune to failure. In the absence of testable claims, the studies reviewed are most appropriately characterized as imaginary worlds or thought experiments.

 

Type: Commentary