This limitation leads us into the next section to cover a very common misinterpretation of P values. P values address only one question: how likely are your data, assuming a true null hypothesis? It does not measure support for the alternative hypothesis. This P value indicates that if the vaccine had no effect, you’d obtain the observed difference or more in 4% of studies due to random sampling error. In technical terms, a P value is the probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis.įor example, suppose that a vaccine study produced a P value of 0.04.

Low P values: your data are unlikely with a true null.Ī low P value suggests that your sample provides enough evidence that you can reject the null hypothesis for the entire population.

High P values: your data are likely with a true null.

How likely is the effect observed in your sample data if the null hypothesis is true? It measures how compatible your data are with the null hypothesis. P values evaluate how well the sample data support the devil’s advocate argument that the null hypothesis is true. Consequently, the devil’s advocate position is that the observed difference in the sample does not reflect a true difference between populations. In fact, it is extremely unlikely that the sample groups will ever exactly equal the null hypothesis value. The null hypothesis is true: there is no difference between the experimental groups at the population level.ĭespite the null being true, it’s entirely possible that there will be an effect in the sample data due to random sampling error. To see why, let’s imagine an experiment for a drug that we know is totally ineffective. This lack of a difference is called the null hypothesis, which is essentially the position a devil’s advocate would take when evaluating the results of an experiment.

Unfortunately for the researchers, there is always the possibility that there is no effect, that is, that there is no difference between the groups. It could be the effectiveness of a new drug, building material, or other intervention that has benefits. In every experiment, there is an effect or difference between groups that the researchers are testing. In order to understand P values, you must first understand the null hypothesis. What Is the Null Hypothesis in Hypothesis Testing? In this post, I'll help you to understand P values in a more intuitive way and to avoid a very common misinterpretation that can cost you money and credibility.

In fact, P values often determine what studies get published and what projects get funding.ĭespite being so important, the P value is a slippery concept that people often interpret incorrectly. Everyone knows that you use P values to determine statistical significance in a hypothesis test. The P value is used all over statistics, from t-tests to regression analysis.

P value minitab