Advanced ANOVA/Factorial ANOVA

Resource type: this resource contains a tutorial or tutorial notes. Completion status: this resource is ~75% complete. The purpose of this tutorial is to teach use of factorial ANOVA Practical exercises are based on using SPSS.

Design

Factorial ANOVA involves testing of differences between group means based on two or more categorical independent variables (IVs), with a single, continuous dependent variable (DV). In other words, a factorial ANOVA could involve:

• Two or more between-subjects categorical/ordinal IVs
• One interval or ratio DV

The results of interest are:

• Main effect for IV1
• Main effect for IV2
• Interaction between IV1 and IV2

If significant effects are found and more than two levels of an IV are involved, then provide follow-up tests, which could be either:

• A priori (planned) contrasts
• Post-hoc contrasts

(Note: Use appropriate control of the family-wise Type I error rate)

Effect sizes should also be reported. Eta-squared provides an estimate of the percentage of variance in the DV explained by each main effect and interaction effect. Cohen's p provides an estimate of the size of differences between two groups in standard deviation groups.

Example

"What is the effect of Gender (2) and Degree Type (3) on Overall Student Satisfaction?" This could be described as a 2 (Gender) by 3 (Degree Type) factorial ANOVA.

General steps

1. Establish hypothesis/hypotheses
   • Make sure you have separate hypotheses for:
     • Main effect for each IV
     • Interactions between IVs
     • Planned contrasts (if warranted)
2. Examine assumptions:
   • IVs (categorical; between-subjects) and DV (at least interval)
   • The data in each cell is normally distributed
   • Homogeneity of variance (the variance in each cell is similar)
   • Cells are independent
3. Examine descriptive statistics, particularly the four moments (M, SD, Skewness, Kurtosis) overall, and also for each cell
4. Examine graphs
5. Conduct inferential test (ANOVA) and interpret significance of F scores
6. Conduct follow-up tests (planned contrasts or post-hoc tests) if F is significant
7. Interpret interactions
8. Calculate and interpret effect sizes
   • Eta-square (omnibus - equivalent to R²)
   • Standardised mean effect size (difference b/w two means) e.g., Cohen's d

Example SPSS outputs

• Factorial ANOVA (example) - Are there differences in University Student Satisfaction levels between Gender and Age?
• Factorial ANOVA (example) - Are there differences in University Student Satisfaction levels between Gender and Age? - Are there differences in Locus of Control between Gender and Age?

Descriptives

1. A table of descriptive statistics (M, SD, Skewness, and Kurtosis) for each cell and for each marginal total, and grand total should be presented when reporting results.
2. For a 2-way ANOVA, the descriptives table it is recommended to provide a breakdown of one IV in the columns and the other IV in the rows, such as illustrated in the following tables.

	Age
Gender	Younger	Older	Total
Males
Females
Total

Note that each of the three columns on the right should be further split into five columns to allow reporting of M, SD, Skewness, and Kurtosis, n. So, the expanded descriptive tables layout could look like this: (note that rows and columns have been swapped around from the table above - this is somewhat arbitrary):

Gender

Age

Males

Females

Overall

M

SD

Sk

Kurt

M

SD

Sk

Kurt

M

SD

Sk

Kurt

Young

Middle

Older

Overall

Here's an example of such an APA style table:

Understanding interactions

• One of the keys to understanding Factorial ANOVA is being able to intepret interactions.
• A recommended experiential exercise for learning about interactions is to fabricate a dataset which can be used to demonstrate factorial ANOVAs in which there are:
  1. No effects
  2. Main effect A, no main effect B, no interaction
  3. Main effect A, no main effect B, interaction
  4. No main effect A, main effect B, no interaction
  5. No main effect A, main effect B, interaction
  6. Main effect A, main effect B, no interaction
  7. Main effect A, main effect B, interaction
  8. Interaction, no main effects

Francis exercises

• 3.3 (Analysing Differences)
  • 3.3.8 Factorial ANOVA
    • AQUES.sav
    • Motiv.sav

Effect sizes

• Cohen's d
• Eta-squared

Write-up

A write-up checklist for a factorial ANOVA might include:

1. What is the goal/purpose of the analysis?
2. What is the design - i.e., what are the IVs and the DV?
3. What are the relevant descriptive statistics for the cells and the marginal descriptives? (M, D, skewness, kurtosis, n)
4. To what extent does the data meet the assumptions for ANOVA (e.g., independent observations (often assumed), normality, and homogeneity of variance)?
5. What are the main and interaction effect statistics, including the direction, statistical significance, and size of effects (consider the merits of reporting eta-squared and/or a standardised mean difference statistic such as Cohen's d; see effect sizes)?
6. Provide a figure - often a line graph is used.
7. Are follow-up tests necessary and/or warranted? e.g., if there are planned contrasts and/or a significant effect with three or more levels, then appropriate follow-up tests should be conducted using a method for controlling the family-wise Type I error rate as appropriate.
8. What are the conclusions?

See also

• Factorial ANOVA

External links

• Factorial ANOVA (ucspace)
  • Factorial ANOVA Notes (Handout; 2007)