16.11.2023 New Preprint on Uncovering Null Effects in Null Fields

Uncovering Null Effects in Null Fields: The Case of Homeopathy

Autor:innen:
Erdfelder, E., Nagel, J., Heck, D. W., & Petras, N.

Abstract:

Sigurdson, Sainani, and Ioannidis (Journal of Clinical Epidemiology, 2023) discussed homeopathy as a prototypical example of a “null field” where true effects are nonexistent and positive effect sizes reflect bias only. Based on a sample of published randomized placebo-controlled trials, they observed a remarkable effect in favor of homeopathy (Hedges’ g = 0.36). The authors concluded that this reflects “the average impact of the bias present in the field”. We argue that the estimated amount of bias largely depends on the meta-analytic measure used to quantify treatment effects. By applying a bias-corrected measure instead, we show that the maximum-likelihood estimate of the true effect reduces to virtually zero when selective publishing of significant results is appropriately taken into account. We conclude that inclusion of bias-corrected measures should become routine practice in meta-analyses. In line with Sigurdson, Sainani, and Ioannidis’ reasoning, we recommend “null fields” as domains to validate such measures.

Erdfelder, E., Nagel, J., Heck, D. W., & Petras, N. (in press). Uncovering null effects in null fields: The case of homeopathy. Journal of Clinical Epidemiology. https://doi.org/10.31234/osf.io/x6buj