07.06.2021 Neue Publikation in Psychological Review

Assessing the 'Paradox' of Converging Evidence by Modeling the Joint Distribution of Individual Differences: Comment on Davis-Stober and Regenwetter (2019)

Autor: Daniel W. Heck 

Abstract:

Davis-Stober and Regenwetter (2019; D&R) showed that even when all predictions of a theory hold in separate studies, not even a single individual may be described by all predictions jointly. To illustrate this 'paradox' of converging evidence, D&R derived upper and lower bounds on the proportion of individuals for whom all predictions of a theory hold. These bounds reflect extreme positive and negative stochastic dependence of individual differences across predictions. However, psychological theories often make more specific assumptions such as true individual differences being independent or positively correlated (e.g., due to a common underlying trait). Based on this psychometric perspective, I extend D&R's conceptual framework by developing a multivariate normal model of individual effects. Assuming perfect consistency (i.e., a correlation of one) of individual effects across predictions, the proportion of individuals described by all predictions of a theory is identical to D&R's upper bound. The proportion drops substantially when assuming independence of individual effects. However, irrespective of the assumed correlation structure, the multivariate normal model implies a lower bound that is strictly above D&R's lower bound if a theory makes at least three predictions. Hence, the scope of a theory can be improved by specifying whether individual effects are assumed to show a certain level of consistency across predictions (similar to a trait) or whether they are statistically independent (similar to a state).

Den Preprint finden Sie unter: https://psyarxiv.com/ca8z4