"It can really happen because it is at heart a linguistic rule. Even if rigorously enforced, it just means that editors would force people in papers to say "statistically suggestive for a p of a little less than .05, and only allow the phrase "statistically significant" in a paper if the p value is .005 or less. As a well-defined policy, it is nothing more than that. Everything else is general equilibrium effects...
"For exploratory research with very low prior odds (well outside the range in Figure 2), even lower significance thresholds than 0.005 are needed. Recognition of this issue led the genetics research community to move to a “genome-wide significance threshold” of 5×10^{-8} over a decade ago. And in high-energy physics, the tradition has long been to define significance by a “5-sigma” rule (roughly a P-value threshold of 3×10^{-7} ). We are essentially suggesting a move from a 2-sigma rule to a 3-sigma rule."...
The authors of "Redefine Statistical Significance" are careful to say that people should be able to publish papers that have no statistically significant results:
"We emphasize that this proposal is about standards of evidence, not standards for policy action nor standards for publication. Results that do not reach the threshold for statistical significance (whatever it is) can still be important and merit publication in leading journals if they address important research questions with rigorous methods. ""
FB: "In other words, as things are now, something declared "statistically significant" at the 5% level is much more likely to be false than to be true.
By contrast, the authors argue, results declared significant at the 1/2 % level are at least as likely to be true as false, in the sense of being replicable about 50% of the time in psychology and about 85% of the time in experimental economics"
No comments:
Post a Comment