Survey research and design in psychology/Assessment/Lab reports/5

Lab report 5: ANOVA

Task: Conduct and report on a Mixed ANOVA (at least 2 x 3 or 3 x 2) or ANCOVA (at least 2 x 2 plus a covariate).

Marking criteria

In addition to the generic guidelines, this report should include:

1. 5%. Title/Abstract: Provide a succint overview of the study.
   1. Sumarise the study's purpose, method, findings, and implications in less than 150 words.
2. 10%. Introduction:
   1. 5%. Background: Briefly introduce the topic and background literature.
   2. 5%. Research questions: Propose logically-derived research question(s) (which are addressed in the Results) - in their simplest form, the research questions are likely to be e.g., "Is there a main affect for A for Y? Is there a main effect for B for Y? Is there an interaction between A and B for Y?" (for Mixed ANOVA). ANCOVA questions would include "... whilst controlling for C" and/or hypotheses for each tested effect.
3. 10%. Method:
   1. Participants: Not applicable
   2. 10%. Materials: Explain the operationalisation of measures used in the current study, including summarising how any composite scores were derived and their internal consistency.
   3. Procedure: Not applicable
4. 50%. Results: Describe and present the results of a Mixed ANOVA (at least a 2 x 3 or 3 x 2 design) or ANCOVA (at least a 2 x 2 design (so that the analysis deals with an interaction) plus a covariate). Reporting should summarise any data screening and/or recoding (if not already covered in Materials), type of analysis, assumption testing, descriptive statistics for the cells and marginal totals, ANOVA results (including F, df, p, eta-squared and/or partial eta-squared, and standardised mean effect sizes (d) with an accompanying figure (e.g., error-bar chart), and follow-up planned contrasts or post-hoc comparisons if necessary. The results should demonstrate a clear understanding of the differences in the DV by the IVs and possible interaction between the IVs, which includes understanding of the significance, direction, and strength of tested effects.
5. 25%. Discussion: Explain what the analysis found out about the research question(s) and/or hypotheses e.g.,
   1. What was the difference in Y means for IV1? Why?
   2. What was the differences in Y means for IV? Why?
   3. What was the interaction between IV1 and IV2 for Y means? Why?
   4. What were the study's strengths and weaknesses? What could be improved?
   5. What conclusions can be drawn and what are the implications?

FAQ

Is a composite overall score needed for a mixed ANOVA?

1. Creating a composite total score of the within-subjects IV is not necessary for a mixed ANOVA analysis, but it is needed for calculating the marginal total descriptive statistics in SPSS.

Which effect sizes should be reported?

1. For the overall ANOVA, eta-squared can be reported (optional). It needs to be calculated by hand from SPSS ANOVA output. It is the equivalent of R2 in MLR, i.e., the % of variance in the DV explained by the IVs
2. For each ANOVA effect (main and interaction), report the partial eta-squared. This is the percentage of variance explained by each effect
3. For pairwise contrasts (i.e., describing the size of mean difference between two groups or two variables), use a standardised mean difference effect size. This can be reported for all contrasts or only those involving post-hoc testing or planned contrasts.

How can adjusted descriptive statistics be obtained for ANCOVA?

1. In SPSS Via Options (in the General linear model/ANOVA dialogue boxes) - for more detail see [1] - this will provide means and standard errors (SEs). Standard deviations (SDs) can be calculated (e.g., for standardised mean effect sizes) by multiplying the SEs by the square root of the sample size. However, adjusted skewness and kurtosis are not provided. Thus, skewness and kurtosis could be provided instead for the unadjusted scores in a table (or appendix). Plots can also be requested via the ANOVA dialogue boxes, which will provide line graphs of the adjusted means. Alternatively, a line graph could be drawn using other software (e.g., a word-processor) based on the adjusted means.

When should post-hoc tests be conducted?

1. When a main effect with three or more levels is significant.

General feedback

Overall, performance was slightly better than on previous reports (67.9%).

1. Word count
   1. This was generally spot on!
2. Formatting
3. APA style
   1. APA style has improved considerably across the reports, although there is still room for improvement in almost every case.
4. Title
   1. Sometimes struggled to succintly provide a clear description of the study
   2. Were all main variables mentioned?
   3. Avoid referring to the specific statistical techniques used; focus on the research question and/or findings
5. Abstract
   1. Did not always describe the method and sufficient detail in results/discussion.
   2. Avoid abbreviations in the Abstract.
   3. Avoid references in the Abstract.
   4. Avoid reporting detailed statistics in the Abstract.
6. Introduction
   1. Generally, succint
   2. Generally, proposed appropriate research questions/hypotheses
   3. Often could have been provided a literature which corresponded more closely to the RQs/hypotheses
7. Method - Materials
   1. Sometimes didn't cite the survey or explain sufficient detail about how the constructs of interest were operationalised
   2. Could often improve by making sure to include (for composite scores), the number of items in each factor, the response scale, and how composite scores were created
8. Results
   1. Generally, well done.
   2. Some reports exhibited a MLR/correlational conceptualisation of the study rather than an ANOVA/mean difference conceptualisation.
   3. Generally, an appropriate table (including M, SD, Skewness and Kurtosis for each cell and marginal totals, with an accompanying figure (e.g., multiple line graph) were provided.
   4. Figures should be presented so that they can be readily interpreted in black and white printing.
9. Discussion
   1. Generally, well done.
   2. A greater depth of understanding can be demonstrated when:
      1. an explanation if provided for how cited limitations may have affected the results
      2. more specific recommendations are made
10. References
    1. Generally, well done.
    2. APA style - do not include issue #s.