Calculating The Power of a Test in Hypothesis Testing with R

We provide effective and economically affordable training courses for R and Python, Click here for more details and course registration !

In hypothesis testing, the analyst has chance to commit both Type I and Type II errors. The Type I error (α) refers to the probability of wrongly rejecting a true Null hypothesis – H0, while the Type II error (ß) represents the probability that failing to reject a false H0. The value of 1- ß is called the Power of Test in hypothesis testing. Its value says the ability of correctly rejecting a false H0, under the specified Null hypothesis – H0 and Alternative hypothesis – H1.

In this post, we provide an example illustrating the calculation of Power of Test using R programming. The function used in the example is with binomial distribution, but the grounds of the method can be appropriately applied when using other statistical distributions as well.

1. Description of the problem

In a statistical testing whether the new instrument is more effective than the current one in use, there are 20 patients randomly selected from the population to test with this machine. If there are more than 8 patients getting positive results with this new instrument, the analyst will reject the Null hypothesis H0 about he success rate: p = 0.25, in favor of the Alternative hypothesis about the success rate : p = 0.5. Otherwise, the H0 will be held and analyst concludes that the new instrument acts not significant better than the current one in use.

2. Calculate the Power of Test in this hypothesis testing.

The problem can be modeled as a Bernoulli process in which total trial number is 20, and the positive number follows a Binomial distribution.

Because the critical value with the decision rule is located at 8 for positive number, the Type II error (ß) here is the probability of getting less than equal to 8 positives when the Alternative hypothesis is true. Then using 1 to minus the Type II error (ß) will reach to our solution for the Power of Test.

Following code shows the implementation of computation in R.

#pbinom() Function is used to find the cumulative probability #of a data following binomial distribution till a given value
#i.e. it finds P(X <= k)
# Syntax:
# pbinom(k, n, p)



#probability of x occur less than or equal to 8 times among #total 20 trials, with probability 0.5 for each trial
> type_2_error <- pbinom(8, size = 20, prob = 0.5)
 
> power_test <- 1 - type_2_error 
 
> power_test
[1] 0.7482777
> 

So, the Power of Test now is around 0.75, which means among large number of sampling tests, about 3/4 of the time the Null hypothesis is correctly rejected, and 1/4 of the time the Null hypothesis is not rejected and actually the Alternative hypothesis is true.

You can also watch video on your YouTube channel for more understanding of the topic.