Instructional design/User testing of e-learning courses/Organizing the Data

Back to Topic: Instructional Design > User Testing of E-Learning Courses > Analyzing the Results

Qualitative Data

Quantitative data is best consolidated and managed by using a spreadsheet or datatbase. This way you can more easily perform descriptive and/or inferential statistical analysis on your data. A spreadsheet also makes it easier to spot patterns in the data.

Here are some examples of what you might calculate using a spreadsheet:

Determine reliability of a testing instrument using the test results
Analyze survey responses (e.g. Likert scale) individually or summatively
Average time to complete a lesson or topic
The frequency of specific problems

If your course is hosted on a learning management system (LMS)w:Learning_Management_System, then you may have access to reporting capabilities. Reporting might include testing scores, completion times, and survey results. The example below is an excerpt from a LMS report on a satisfaction survey of an e-learning course:

Quantitative Data

As with your qualitative data, you'll want to consolidate the quantitative comments and then review them collectively. This will make it easier to see trends in the results. For example, let's say you have ten testers. After consolidating their comments, you might see that eight out of the ten reviewers had comments for a particular page of the course. Even without analyzing their comments, you know right away that this is a potential trouble spot in the course.

Depending on how you conducted your user test, you'll be working with written data from the users themselves, and/or you'll be working from notes taken by an observer/interviewer while watching individual users take the course. Regardless of testing style, you need a method to organize the documented information. It makes sense to group the comments according to the course's location information.

The following is one method of organizing your qualitative data. The scenario is five testers of an e-learning course each using the worksheet (created in MSWord) provided to them for documenting their comments. Each tester dutifully submitted his/her worksheets at the end of the user testing period. The following are excerpts from each of the five worksheets.

WORKSHEET #1, TESTER: B. WALKER:

WORKSHEET #2, TESTER: K. EBY:

WORKSHEET #3, TESTER: V. THORNTON:

WORKSHEET #4, TESTER: T. RANDALL:

WORKSHEET #5, TESTER: A. MCCALLUM:

The next step is to take all five worksheets and create a single compiled version. The results look something like this:

Note that a color key has been created for ease of identifying the individual making the comment:

The above described method can become labor-intensive if you have a large number of testers. Some e-learning companies develop automated solutions to compiling their user test results. For example, you could create a macro in MSWord that allows you to feed the information directly into a customized database program (e.g. FileMaker Pro).

Click Next to learn how to analyze your qualitative data.

Back to Topic: Instructional Design > User Testing of E-Learning Courses > Analyzing the Results