Null Regions: A Unified Conceptual Framework for Statistical Inference
Traditional null hypothesis significance testing (NHST) can be counterproductive to scientific progress; increasing the precision of a study (e.g., increasing sample size) makes it easier to reach statistical significance, providing support to even poorly performing theories that do not account for much of the phenomena of interest. It also falls short of answering practical questions such as “is the effect strong enough to matter?” or “is the effect small enough to be considered equivalent to 0?” When using the “null regions framework” (https://doi.org/10.1098/rsos.221328), however, if the observed value is inside the “null region” (i.e., population values that one seeks to rule out), statistical tests will never be significant no matter how large the sample size (e.g., “big data” studies). It makes it possible to more meaningfully define successful replication and empirical confirmation of the predictions made in registered reports. Confidence intervals’ role in these tests naturally encourages researchers to focus on effects estimation, rather than the binary “significant” vs. “not significant” dichotomy. An especially appealing feature of these tests is that they can be conducted without any computing devices as long as confidence intervals are available for the statistic of interest.