Survey research and design in psychology/Assessment/Lab report/Feedback/2013

General feedback about the lab report (2013)

This page summarises commonly-repeated feedback about the lab reports in 2013. This feedback is supplementary to the a priori Lab report guidelines which include marking criteria. Feedback from other years is also available - General feedback about the lab report

Title

1. The title should clearly and unambiguously communicate the main content of the report (Weaker titles tended to be more vague and lacked reference to the main variables analysed and their relationship(s).)
2. Longer titles generally provide more appropriate detail than shorter titles (try to mention the key variables or questions).
3. A few reports didn't provide an APA style title page.

Abstract

1. Should be one paragraph in length. Sometimes multiple paragraphs were presented.
2. Typically too much focus on the Intro/Method and not enough on the Results and Discussion.
3. Strength and direction of results were often not indicated.
4. Often there was no mention of implications or recommendations.
5. Statistical results (i.e., with symbols and numbers) should not generally be reported in the abstract unless they are particularly pertinent (e.g., a notable effect size).
6. References should not be reported in the abstract unless they are particularly pertinent (e.g., to draw attention to a key theory which is being tested).

Introduction

1. There is generally one major issue/criteria → Did the introduction provide a review of literature which lead directly to clearly expressed, logically-derived research question(s) and hypotheses?
2. For example, weaker introductions might have reviewed some literature but this wasn't necessarily related directly to justifying each of the hypotheses
3. Present one hypothesis or research question for each statistical test to be conducted (EFA and MLR). Sometimes only one research question or hypothesis was provided.
4. Ensure that hypothesis include predicted direction of the relationship (where applicable). Also, try to be as concise as possible.

Method

Participants

1. Often only basic profiles of the sample were provided (e.g., N, and n and percentage of males and females, with the average age (and SD and range; what about the median?)
2. Better sections provided more thoughtful description of the sample e.g,. more description of the cultural context (so that an outside/naive reader can better understand the sample) and/or comparing with known statistics about the university poulation.
3. Further description of the sample could have been provided by using other demographic information (e.g., enrolment status, living status, completion).

Measures

1. Generally the instrumentation purpose, development and structure were well explained.
2. Weaker sections tended to lack sufficient description of the proposed factors, e.g.,
   • Include a table summarising the proposed factor names, definitions, with example items.
3. Put more emphasis on the measurement of variables used in the Results than on other aspects of the survey.
4. Provide a citation and reference to the survey.

Procedure

1. What kind of sampling technique was used? (Hint: It was not random - this was a common mistake - it was convenience sampling, with systematic selection.)
2. Make sure to provide a citation and reference to the administration guidelines (otherwise, how can someone replicate the study?).
3. How did your administration process go? (response rate, reasons for refusal, anomalies)
4. Information about data entry, conventional statistical software (e.g., SPSS), data collation, etc. is not necessary.

Results

1. Do not mention what software was used for well-known and commonly available data analysis techniques, such as being used in this study.
2. Avoid referring to SPSS variable names - these arbitrary. Refer instead to the construct, capitalised e.g., Stress or University Student Satisfaction
3. Round to 2 decimal places, except when reporting p values where 3 decimals are recommended.

Factor analysis

1. Generally this was well done.
2. The Time Management survey questions included some negatively worded items (i.e., items about time-wasting and procrastination). These items probably should be reverse-coded so that high scores represent better time management (i.e., lack of time-wasting). Weaker reports may not explain or understand or interpret the direction of scoring and meaning for TM particularly well.
3. Some reports seemed to remove items before deciding on the number of factors - decide # of factors first, then consider which items to remove
4. Assumptions
   1. How many cases per variable were there? (Sometimes not reported)
5. Use item descriptions rather than item names (item names are arbitrary and unfamiliar to an outside reader).

Multiple linear regression

1. Often more care should have been taken to prepare the variables for MLR. For example:
   1. Sometimes the IVs should have been recoded as dichotomous or dummy-coded.
   2. Average Mark was a commonly misunderstood variable because data enterers entered both marks (/100) and grades (GPA; ordinal values). Instructions were provided about how this variable could be converted to fully continuous data or fully ordinal data (see Survey research and design in psychology/Assessment/Project/Lab report/Data screening|data screening). Fully continuous data is needed if Average Mark is to be used as a DV in MLR. And continous or dichotomous data is needed if using this variable as an IV in MLR.Average Mark required some careful screening and recoding in order for it to be screened and to have an appropriate level of measurement for MLR.
2. Report on the individual predictors in-text, as well as the overall model.
3. It is important that understanding of the direction, size, and significance of MLR relationships are shown for each of the predictors.

Discussion

1. The key task is to demonstrate depth of understanding about the results and the implications for theory, research, and practice.
2. Did the discussion show that the directions and strength of relations between constructs were clearly explained/understood?
3. Avoid disregarding the study. Remember that even finding no significant relationship tells us something useful.
   1. If suggesting a greater sample size to increase power, make sure to report power for the study. (It turns out the studies power was probably pretty good). -->
4. The most common problem with recommendations and conclusions was their lack of specificity (vagueness).

References

1. Generally, APA style for references was quite good, but only rarely was it perfect.
2. APA style: Do not include issue numbers for journals which provide consecutive page numbering through the issues within a volume (i.e., most of the major academic journals)
3. Use hanging indent via paragraph styles (rather than manually tabbing the 2nd and subsequent lines of references inwards)

APA style

Common problems were:

1. Formatting of title page
2. Formatting of headings: Heading styles have changed from the 5th ed. to 6th ed. of the APA style manual. The headings are now mostly bold, for example. See: http://blog.apastyle.org/apastyle/2009/07/five-essential-tips-for-apa-style-headings.html
3. Use of running head
4. Do not include the heading for "Introduction" but do repeat the title before the introduction
5. Where more than 2 references are cited consecutively, ensure they are in alphabetical order.
6. Numbers under 10 which are used in sentences should be written in words.

Spelling and Grammar

1. Use Australian spelling (e.g., hypothesise instead of hypothesize)
2. Start sentence with words not numerals, e.g., use "Seventy-five people...." rather than "75 people...".
3. In sentences, use words rather than symbols e.g., "<= 21" should be written as "less than or equal to 21". If used within brackets, symbols should be used.
4. Symbols such as equals (=) represent/replace words, therefore they should have a space before and after.
5. Statistical symbols which use English letters (such as M) should be italicised.
6. Write in the third person perspective (i.e., do not use I, we, you, our, etc.).

Formatting

1. Use page breaks rather than multiple blank lines to separate content onto new pages.

Tables

1. Unedited (default) output from statistical software (for tables and figures) is not acceptable as APA style.
2. Right align statistics in tables.
3. Two decimal points are generally sufficient - we don't learn much from the third.
4. Centre tables horizontally on the page.
5. Using the Table feature of the word processing software is recommended because one cell per unit of information allows more powerful manipulation and formatting of columns and rows.

Capitalisation

1. Measured constructs should be referred to as proper names, i.e., with first-letter of the words capitalisation. This is mostly for the Method and Results and parts of the Discussion. In the Introduction and parts of the Discussion, where the more general concept (not the operationalised measure) is being used, this should be capitalised.

Written expression

1. Avoid one-sentence paragraphs (try three to five sentences).
2. Avoid overly-long paragraphs (convey one key idea per paragraph).

See also

• Marking dispute process