Instructional design/User testing of e-learning courses/E-learning User Test Plan

Back to Topic: Instructional Design > User Testing of E-Learning Courses > Review Evaluation Plan

One of the key objectives of this module is to be able to perform a user test in an e-learning environment. Before you can perform the test, a test plan needs to be developed.

Recall the key reasons a plan is needed:

To determine the purpose of the test;

To outline tools needed to conduct the test; and

To determine if the testing goal was accomplished.

This plan was developed for user testing on the “Introduction to the Psychomotor Behaviors” Wikiversity page, setting the stage for you to actively participate in the Evidence, Data Gathering and Analysis sections of the plan in the next two lessons.

In “Conducting User Testing” you will be conduct a user test on the “Introduction to the Psychomotor Behaviors” Wikiversity page.

In “Analyzing the Results” you will review your results, judge the severity of your findings, and generate a list of recommendations.

Keeping this in mind, the following plan was created based on the elements listed in the previous topic.

The Plan

Purpose

User testing is a field trial that tests the course's usability and audience attitude about the course. For a user test, the course is delivered in a real setting to users in the target audience. These users will report any problems they encounter using the course. User testing is also used to evaluate the changes made as a result of the "alpha" review. The alpha is typically a small group trial, of SMEs where the course material and instructional objectives are tested.

Audience

The target audience includes the project owner, designer, and SMEs responsible for the instructional design and content. These people will receive the results of the user test. After the results have been reviewed and change decisions have been made, a report of changes will be sent to the developers.

Issues

The objective of user testing is to test the usability and attitude about the course. The following issues regarding the user testing in the selected Wikiversity page are:

Wiki Content: As noted earlier, this is an open forum, the content of the selected wiki page for this module may change over time.

Time Constraints: The evaluator will have a limited amount of time to test the internal wiki page.

Grading: The evaluation will not be graded. The evaluator will be able to compare his/her results with a completed evaluation that was performed for comparison.

Resources

The following resources will be needed to conduct a user test on the wiki page:

Members of the target audience;

Access to the wiki page; and

Evaluation tools: course evaluation document and attitude evaluation sheet

Evidence

The evidence needed to an to answer the evaluation questions and objectives noted above will be qualitative and quantitative in nature. Quantitative describes data in terms of quantity, whereas qualitative describes data in terms of quality.

Examples of quantitative evidence includes data regarding total seat-time for the course and scores on the knowledge checks. Examples of qualitative evidence includes data regarding tester comments about the instruction and problems with the delivery platform.

For a detailed discussion on the data that is conducted during a user test, see the topic titled The Data.

Data-gathering Techniques

The evidence collected within this module will be the completed knowledge checks. In a real setting, additional evidence will include the completion of the completed evaluation and attitude documents and the results of any knowledge checks or activities the tester completes during the user evaluation.

Analysis

Each completed evaluation document, knowledge check, activity, and any other data that's collected will be reviewed for clarity, trends, problems, and successes. This data will then be evaluated on the basis of priority from deal breaker, meaning this has to be corrected to not applicable, meaning no action will be taken. Actual analysis will not be available until after the user test is complete.

Reporting

After the analysis is completed, an evaluation report will be written to include the following information: executive summary, purpose of evaluation, methodology, results, and conclusions and recommendations.

Click Next to check your understanding.

Back to Topic: Instructional Design > User Testing of E-Learning Courses > Review Evaluation Plan