#### Re: Learned any statistics lately?

You are quite correct, the paper you quote does, indeed, say that "P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone".

One of the points that this statement is making is that factors other than random chance can affect a result. This is also perfectly true. And if we had complete knowledge of the situation we would also know those other factors and (if we were smart enough) take them into consideration. But if we don’t know any of those, then the statement given in the article “There is a 25 per cent chance that the difference in these figures is simply due to random chance.” is the closest estimate that we have.

It is also worth noting that the paper you quote also says “Informally, a p-value is the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.” This, informally, agrees with the statement given in the Register article.

However this very discussion we are having simply highlights that the field of statistics is complex and open to interpretation. Indeed, to again quote from the same you cite:

“On behalf of the Board, he reached out to over two dozen such people, all of whom said they would be happy to be involved [in creating an ASA statement on p-values and statistical significance]. Several expressed doubt about whether agreement could be reached, but those who did said, in effect, that if there was going to be a discussion, they wanted to be involved.

Over the course of many months, group members discussed what format the statement should take, tried to more concretely visualize the audience for the statement, and began to find points of agreement. That turned out to be relatively easy to do, but it was just as easy to find points of intense disagreement.”

So the experts assembled to create the paper from which you quote clearly disagreed. This is perfectly reasonable, as I said above, the field is complex and even eminent statisticians disagree about interpretation.

I believe that the author of this Register article was making the point that most people:

do not have the required training to interpret statistics accurately.

are not interested in the finer detail.

If that is the case, our options are to:

continue not to inform the business user of the availability of statistical analysis.

use statistics to give the business user SOME guidance.

So, by telling the user that “There is a 25 per cent chance that the difference in these figures is simply due to random chance.” we are giving them the best guidance that we can, given the information we currently have and certainly more than by giving them a bar chart.

We can argue until we are blue in the face about the wording that should be used but we need to bear in mind that, if it is made too complex it will convey no information and we will be back at square one.