When you perform a hypothesis test, there are four possible outcomes depending on the actual truth, or falseness, of the null hypothesis H0 and the decision to reject or not. The outcomes are summarized in the following table:
| ACTION | H0 IS ACTUALLY | ... |
|---|---|---|
| True | False | |
| Do not reject H0 | Correct outcome | Type II error |
| Reject H0 | Type I error | Correct outcome |
The four possible outcomes in the table are as follows:
- The decision is not to reject H0 when H0 is true (correct decision).
- The decision is to reject H0 when, in fact, H0 is true (incorrect decision known as a Type I error).
- The decision is not to reject H0 when, in fact, H0 is false (incorrect decision known as a Type II error).
- The decision is to reject H0 when H0 is false (correct decision whose probability is called the Power of the Test).
Each of the errors occurs with a particular probability. The Greek letters α and β represent the probabilities.
α = probability of a Type I error = P(Type I error) = probability of rejecting the null hypothesis when the null hypothesis is true.
β = probability of a Type II error = P(Type II error) = probability of not rejecting the null hypothesis when the null hypothesis is false.
α and β should be as small as possible because they are probabilities of errors. They are rarely zero.
The Power of the Test is 1 – β. Ideally, we want a high power that is as close to one as possible. Increasing the sample size can increase the Power of the Test.
The following are examples of Type I and Type II errors.
Example 9.5
Suppose the null hypothesis, H0, is: Frank's rock climbing equipment is safe.
Type I error: Frank does not go rock climbing because he considers that the equipment is not safe, when in fact, the equipment is really safe. Frank is making the mistake of rejecting the null hypothesis, when the equipment is actually safe!
Type II error: Frank goes climbing, thinking that his equipment is safe, but this is a mistake, and he painfully realizes that his equipment is not as safe as it should have been. Frank assumed that the null hypothesis was true, when it was not.
α = probability that Frank thinks his rock climbing equipment may not be safe when, in fact, it really is safe. β = probability that Frank thinks his rock climbing equipment may be safe when, in fact, it is not safe.
Notice that, in this case, the error with the greater consequence is the Type II error. (If Frank thinks his rock climbing equipment is safe, he will go ahead and use it.)
Try It 9.5
Suppose the null hypothesis, H0, is: the blood cultures contain no traces of pathogen X. State the Type I and Type II errors.
Example 9.6
Suppose the null hypothesis, H0, is: a tomato plant is alive when a class visits the school garden.
Type I error: The null hypothesis claims that the tomato plant is alive, and it is true, but the students make the mistake of thinking that the plant is already dead.
Type II error: The tomato plant is already dead (the null hypothesis is false), but the students do not notice it, and believe that the tomato plant is alive.
α = probability that the class thinks the tomato plant is dead when, in fact, it is alive = P(Type I error). β = probability that the class thinks the tomato plant is alive when, in fact, it is dead = P(Type II error).
The error with the greater consequence is the Type I error. (If the class thinks the plant is dead, they will not water it.)
Try It 9.6
Suppose the null hypothesis, H0, is: a patient is not sick. Which type of error has the greater consequence, Type I or Type II?
Example 9.7
It’s a Boy Genetic Labs, a genetics company, claims to be able to increase the likelihood that a pregnancy will result in a boy being born. Statisticians want to test the claim. Suppose that the null hypothesis, H0, is: It’s a Boy Genetic Labs has no effect on gender outcome.
Type I error: This error results when a true null hypothesis is rejected. In the context of this scenario, we would state that we believe that It’s a Boy Genetic Labs influences the gender outcome, when in fact it has no effect. The probability of this error occurring is denoted by the Greek letter alpha, α.
Type II error: This error results when we fail to reject a false null hypothesis. In context, we would state that It’s a Boy Genetic Labs does not influence the gender outcome of a pregnancy when, in fact, it does. The probability of this error occurring is denoted by the Greek letter beta, β.
The error with the greater consequence would be the Type I error since couples would use the It’s a Boy Genetic Labs product in hopes of increasing the chances of having a boy.
Try It 9.7
Red tide is a bloom of poison-producing algae—a few different species of a class of plankton called dinoflagellates. When the weather and water conditions cause these blooms, shellfish such as clams living in the area develop dangerous levels of a paralysis-inducing toxin. In Massachusetts, the Division of Marine Fisheries montors levels of the toxin in shellfish by regular sampling of shellfish along the coastline. If the mean level of toxin in clams exceeds 800 μg (micrograms) of toxin per kilogram of clam meat in any area, clam harvesting is banned there until the bloom is over and levels of toxin in clams subside. Describe both a Type I and a Type II error in this context, and state which error has the greater consequence.
Example 9.8
A certain experimental drug claims a cure rate of at least 75 percent for males with a disease. Describe both the Type I and Type II errors in context. Which error is the more serious?
Type I: A patient believes the cure rate for the drug is less than 75 percent when it actually is at least 75 percent.
Type II: A patient believes the experimental drug has at least a 75 percent cure rate when it has a cure rate that is less than 75 percent.
In this scenario, the Type II error contains the more severe consequence. If a patient believes the drug works at least 75 percent of the time, this most likely will influence the patient’s (and doctor’s) choice about whether to use the drug as a treatment option.
Try It 9.8
Determine both Type I and Type II errors for the following scenario:
Assume a null hypothesis, H0, that states the percentage of adults with jobs is at least 88 percent.
Identify the Type I and Type II errors from these four possible choices.
- Not to reject the null hypothesis that the percentage of adults who have jobs is at least 88 percent when that percentage is actually less than 88 percent
- Not to reject the null hypothesis that the percentage of adults who have jobs is at least 88 percent when the percentage is actually at least 88 percent
- Reject the null hypothesis that the percentage of adults who have jobs is at least 88 percent when the percentage is actually at least 88 percent
- Reject the null hypothesis that the percentage of adults who have jobs is at least 88 percent when that percentage is actually less than 88 percent