4.4 Analysis of Variance (ANOVA)

One-Way ANOVA

To compare three or more means, the sample sizes for each group can be different sample sizes (samples sizes do not need to be identical). For one-way ANOVA hypothesis testing, we will follow the same general outline of steps for hypothesis testing that was discussed in Hypothesis Testing.

The first step will be to write the null and alternative hypotheses. For one-way ANOVA, the null and alternative hypotheses are always stated as the following:

$H_0$: All population means are equal: ${\mu }_1={\mu }_2={\mu }_3=\dots$

$H_a$: At least one population mean is different from the others.

Here are the requirements to use this procedure:

1. The samples are random and selected from approximately normal distributions.
2. The samples are independent of one another.
3. The population variances are approximately equal.

In this discussion, we assume that the population variances are not equal.

When these requirements are met, the F-distribution is used as the basis for conducting the hypothesis test.

The F-distribution is a skewed distribution (skewed to the right), and the shape of the distribution depends on two different degrees of freedom, referred to as degrees of freedom for the numerator and degrees of freedom for the denominator.

Figure 4.10 shows the typical shape of the F-distribution:

Figure 4.10 Shape of the F-Distribution

The test statistic for this hypothesis test will be the ratio of two variances, namely the ratio of the variance between samples to the ratio of the variances within the samples.

The numerator of the test statistic (variance between samples) is sometimes referred to as variation due to treatment or explained variation.

The denominator of the test statistic (variance within samples) is sometimes referred to as variation due to error, or unexplained variation.

The details for a manual calculation of the test statistic are provided next; however, it is very common to use software for ANOVA analysis.

For the purposes of this discussion, we will assume we are comparing three means, but keep in mind the ANOVA method can be applied to three or more means.

1. Find the mean for each of the three samples. Label these means as ${\bar{x}}_1,{\bar{x}}_2,{\bar{x}}_3$. Find the variance for each of the three samples. Label these variances as $s_1^2,s_2^2,s_3^2$.
2. Find the grand mean (label this as $\stackrel{\text{=}}{x}$). This is the sum of all the data values from the three samples, divided by the overall sample size.
3. Calculate the quantity called “sum of squares between (SSB)” according to the following formula:
   
   $$SSB=n_1{({\bar{x}}_1-\stackrel{\text{=}}{x})}^2+n_2{({\bar{x}}_2-\stackrel{\text{=}}{x})}^2+n_3{({\bar{x}}_3-\stackrel{\text{=}}{x})}^2$$
4. Calculate the quantity called “sum of squares within (SSW)” according to the following formula:
   
   $$SSW=(n_1-1)s_1^2+(n_2-1)s_2^2+(n_3-1)s_3^2$$
5. Calculate the degrees of freedom for the numerator (${df}_n$) and degrees of freedom for the denominator (${df}_d$).
   For degrees of freedom for the numerator, this is calculated as the number of groups minus 1. For example, if a medical researcher is comparing three different pain relief medications to compare the average time to provide relief from a migraine headache, there are three groups, and thus the degrees of freedom for the numerator would be $3-1=2$.
   For degrees of freedom for the denominator, this is calculated as the total of the sample sizes minus the number of groups.
6. Calculate the quantity called “mean square between (MSB)” according to the following formula:
   
   $$MSB=\frac{SSB}{{df}_n}$$
7. Calculate the quantity called “mean square within (MSW)” according to the following formula:
   
   $$MSW=\frac{SSW}{{df}_d}$$
8. Calculate the test statistic ($F$), according to the following formula:
   
   $$F=\frac{MSB}{MSW}$$

Once the test statistic is obtained, the p-value is calculated as the area to the right of the test statistic under the $F$-distribution curve (note that the ANOVA hypothesis test is always considered a “right-tail” test).

Usually, the results of these computations are organized in an ANOVA summary table like Table 4.17.

Using Python for One-Way ANOVA

As mentioned earlier, due to the complexity of these calculations, technology is typically used to calculate the test statistic and p-value.

Using Python, the f_oneway() function is provided as part of the scipy library, and this function provides both the test statistic and p-value for a one-way ANOVA hypothesis test.

The syntax is to call the function with the data arrays as arguments, as in:

f_oneway(Array1, Array2, Array3, …)

The function returns both the F test statistic and the p-value for the one-way ANOVA hypothesis test.

The details of this Python function are shown in Example 4.25.

Here is a step-by-step example to illustrate this ANOVA hypothesis testing process.

    
    # import libraries
    import numpy as np
    from scipy.stats  import f_oneway
     
    # enter airline arrival data
    
    AirlineA = np.array([8, 12, 7, 9, 11, 4])
    AirlineB = np.array([13, 0, 2, 7, 6, 5])
    AirlineC = np.array([11, 15, 9, 11, 12])
    
    #f_oneway function returns both the test statistic and p-value
    f_oneway(AirlineA, AirlineB, AirlineC)
    

F_onewayResult(statistic=4.388264306955828, pvalue=0.03314999857295516)