11.01.2024 New publication about null effects

Uncovering null effects in null fields: the case of homeopathy

Authors 
Erdfelder, E., Nagel, J., Heck, D. W., & Petras, N.

What is new?

• Sigurdson, Sainani, and Ioannidis (2023) observed a significant overall bias in randomized controlled trials addressing effects of homeopathy. We discuss selective publication of significant results as a parsimonious explanation of this bias.
• A meta-analytic mixture model is proposed that accounts for effects of selective publication with two parameters only (i.e., the true homeopathy effect and the proportion of results published only when statistically significant).

Key findings

• For the data of Sigurdson et al. (2023), the mixture-model estimate of the true homeopathy effect reduces to virtually zero (estimated d = 0.05, 95% confidence interval: [−0.05 to 0.16]) when taking selective publication into account.

What this adds to what was known?

• To uncover the true effect of treatments from published effect sizes, it is important to make use of statistical models that properly adjust for selective publication of significant results. Given the data of Sigurdson et al. (2023), the meta-analytic mixture model supports the claim that homeopathy essentially is a “null field”.

What is the implication and what should change now?

• Inclusion of effect size measures adjusting for selective publication practices – such as the mixture-model estimate proposed here – should become routine practice in meta-analyses of published research.

Erdfelder, E., Nagel, J., Heck, D. W., & Petras, N. (2023). Uncovering null effects in null fields: The case of homeopathy. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2023.11.006