Survey research and design in psychology/Assessment/Lab report/Feedback/2011

General feedback about the lab report (2011)

This page provides a summary of commonly-repeated feedback about lab reports. This should be read in conjunction with the a priori lab report guidelines and the marking criteria.

Marking distribution

• N = 71
• Mean = 70, SD = 19.5, Skewness = -1.3, Kurtosis = 2.0
• Median = 75
• Mode = 72
• Min = 0
• Max = 99

Title

1. Did it clearly and unambiguously communicate the main content of the report? (Weaker titles tended to be more vague and to lack specific reference to the main variables analysed.)
2. Longer titles generally provide more appropriate detail than shorter titles (try to mention the key variables or questions).
3. A few reports didn't provide an APA style title page.

Abstract

1. Should be one paragraph in length.
2. Typically too much focus on the Intro/Method and not enough on the Results and Discussion.
3. Strength and direction of results were often not indicated.
4. Often there was no mention of implications or recommendations.
5. Statistical results (i.e., with symbols and numbers) should not generally be reported in the abstract unless they are particularly pertinent (e.g., a notable effect size).
6. References should not be reported in the abstract unless they are particularly pertinent (e.g., to draw attention to a key theory which is being tested).

Introduction

1. There is generally one major issue → Did the introduction lead directly to clearly expressed, logically-derived hypotheses?
2. For example, weaker introductions might have reviewed some literature but this wasn't necessarily related directly to justifying each of the hypotheses
3. Present one hypothesis or research question for each statistical test to be conducted. This wasn't always done.

Method

1. There was sometimes a tendency towards plagiarism of the sample lab report.

Participants

1. Often very basic profiles of the sample were provided, e.g., N, and n and percentage of males and females, with the average age (and SD and range; what about the median?)
2. Better sections provided more thoughtful description of the sample e.g,. more description of the cultural context (so that an outside/naive reader can better understand the sample).
3. Further description of the sample could have been provided for example by using other demographic information (many reports simply reported on gender and age).

Measures

1. Generally the instrument purpose, development and structure was well explained.
2. Sometimes the descriptions were overly based on the sample report.
3. Weaker sections tended to lack sufficient description of the proposed factors, e.g.,
   • Include a table summarising the proposed factor names, definitions, with example items.
4. Put more emphasis on the measurement of variables used in the Results than on other aspects of the survey.
5. Make sure to provide a citation and reference to the survey.

Procedure

1. What kind of sampling technique was used? (Hint: It was not random - this was a common mistake - it was convenience sampling, with systematic selection.)
2. Make sure to provide a citation and reference to the administration guidelines (otherwise, how can someone replicate the study?).
3. How did your administration process go? (response rate, reasons for refusal, anomalies)
4. Information about data entry, conventional statistical software (e.g., SPSS), data collation, etc. is not necessary.

Results

1. There's no need to mention what software you used for well-known and commonly available data analysis techniques, such as used in this study.

Factor analysis

1. Generally this was well done.
2. Some reports seemed to remove items and factors simultaneously. The recommended approach is to work out how many factors first, then work out which items to retain/drop.
3. Borderline plagiarism of the sample lab report was not uncommon.
4. Assumptions
   1. How many cases per variable were there? (Sometimes not reported)
   2. Despite what some books say, normality is not an assumption; linear relations are an assumption. Normality tends to enhance linearity. As factor analysis is based on correlations between variables, if r does not accurately measure the strength of a relationship between two variables then the analysis is degraded (Francis, 2007).
5. Use item descriptions rather than item names (item names are arbitrary and unfamiliar to an outside reader).

Qualitative analysis

1. Approx. 80% of reports adopted a quantitative approach to the qualitative data (i.e., categorising and counting). About 20% used thematic analysis. The thematic analyses were generally very good. The quantitative analyses which used multiple response analysis were generally also very good. The best multiple response analyses also provided some thick description of the main themes.
2. Good analyses tended to identify/suggest more themes/categories than weaker analyses. However, categories with very small sample sizes (esp. < 5) are likely to be unreliable and would have better subsumed into broader categories.
3. Some QAs did not suggest any additional categories to those identified in the factor analyses of the quantitative items, which generally did not indicate that an indepth analysis of the qualitative data had been undertaken.
4. Some content analyses did not use multiple response analysis (e.g., they summarised the multiple responses as separate variables).

Multiple linear regression

1. Often more care should have been taken to prepare the variables for MLR. For example, sometimes the IVs should have been recoded as dichotomous or dummy-coded. Also, GPA required some careful screening and recoding.

Discussion

1. The key task is to demonstrate depth of understanding about the results and the implications for theory, research, and practice.
2. Were the directions of relations between constructs clearly explained/understood?
3. New findings should not be presented in the results.
4. Limitations
5. The most common problem with recommendations and conclusions was their lack of specificity (vagueness).

References

1. Do not capitalise the titles of journal articles.
2. Where there are two references by the same author(s) in the same year, use a, b, c, etc. to indicate each reference uniquely e.g, Smith (2008a, 2008b, 2008c).
3. Use hanging indent

Appendices

1. A reader should not have to consult an Appendix to understand the report. Appendices are an optional adjunct (e.g., they aren't used much in journal articles). Any content specifically related to the marking criteria should be presented in the main body.

APA style

1. Remember, follow APA Style Manual 6th ed. (not 5th ed.).

Grammar

1. Sentences should not start with numerals, e.g., "Seventy-five people...." should be used rather than "75 people...".
2. Numbers under 10 which are used in sentences should be written in words.
3. In sentences, use words rather than symbols e.g., "<= 21" should be written as "less than or equal to 21". If used within brackets, symbols should be used.
4. Symbols such as equals (=) represent/replace words, therefore they should have a space before and after.
5. Statistical symbols which use English letters (such as M) should be italicised.
6. Write in the third person (not I, we, you, etc.).

Formatting

1. Heading styles have changed from the 5th ed. to 6th ed. of the APA style manual. The headings are now mostly bold, for example. See: http://blog.apastyle.org/apastyle/2009/07/five-essential-tips-for-apa-style-headings.html
2. Use page breaks rather than multiple blank lines to separate content onto new pages.

Tables & Figures

1. Unedited (default) output from statistical software (for tables and figures) is not acceptable as APA style.
2. Right align statistics in tables.
3. Two decimal points are generally sufficient - we don't learn much from the third.
4. Centre tables and figures horizontally on the page.
5. Using the Table feature of the word processing software is recommended because one cell per unit of information allows more powerful manipulation and formatting of columns and rows.

Capitalisation

1. All measured constructs should be referred to as proper names, i.e., with first-letter of the words capitalisation. This is mostly for the Method and Results and parts of the Discussion. In the Introduction and parts of the Discussion, where the more general concept (not the operationalised measure) is being used, this should be capitalised.

Written expression

1. Write in your own words: Directly copied or lightly rewritten versions of the sample write-ups and example reports is plagiarism :(
2. Avoid one-sentence paragraphs (try three to five sentences).
3. Avoid overly-long paragraphs (convey one key idea per paragraph).

See also

• Marking dispute process