Instructional design/User testing of e-learning courses/Reviewing Evaluation Fundamentals

Back to Topic: Instructional Design > User Testing of E-Learning Courses > Formative Evaluation Overview

ADDIE ISD Model

ADDIE Model

The ADDIE Instructional Systems Design (ISD) model w:ADDIE_Model is a generic ISD model that is used traditionally by instructional designers and training developers.

The five phases — Analysis, Design, Development, Implementation, and Evaluation — represent a dynamic, flexible guideline for building effective training and performance support tools.

While the five phases may seem sequential because of the acronym, ADDIE, the phases can occur simultaneously or within other phases. A good example is evaluation.

Next, the evaluation phase and good evaluation practices are reviewed.

The E in the ADDIE Model - Evaluation

Evaluation is a critical quality assurance step. Evaluations must incorporate the good practices from the instructional field and the good practices from the software field to ensure the quality of their solutions. These include:

Practice	Description	Example
Quality Assurance w:Quality_assurance	The instruction is fully tested for quality assurance.	For example, forming interview or worksheet questions that ask about the reliability, maintainability and safety are ways to promote total quality control.
Correct Information	The information presented is correct.	For example, a typographical error may cause the learner to choose an incorrect response.
Correct Strategy Implementation	The instructional strategies are correctly implemented.	For example, does instruction follow Gagne's Nine Events as established in the design document or have events been rearranged or dropped?
Strategies Support Instructional Objectives	The instructional strategies meet the instructional objectives.	For example, does the use of a motivational strategy, ARCS, help achieve the objective of building learner confidence in the designed instruction?
Compatibility of Delivery Platform & Courseware	The delivery platform works as expected with the courseware.	For example, does Houghton Mifflin's Understanding Business, 8th ed., electronic course and textbook import successfully to Valparaiso University's CourseVU LMS for the course, "English for Business"?

These practices are standard for evaluation, regardless of its occurrence in the ADDIE model. The next section reviews a particular kind of evaluation - formative evaluation.

Formative Evaluation in the ADDIE Model

Don’t let the ADDIE model above fool you – evaluation doesn’t just take place at the end, you should weave it throughout. “Formative” means to shape or transform and formative evaluations should be conducted at regular review intervals. So as you knit your ADDIE model together, remember to weave in your formative evaluation thread.

The purpose of every formative evaluation, regardless of context or curriculum, is to validate the instructional design and strategies to ensure the instructional objectives are being met.

The following presents a customized ADDIE model in action. Note that formative evaluations are conducted at regular intervals.

The formative evaluation milestones seen in the above illustration are:

Design Signoff
- After a preliminary design is created, it is sent to key subject matter experts (SME)w:Subject_matter_expert for review, feedback, and acceptance.

Storyboard Signoff
- This feedback is used to revise the storyboard w:Storyboarding.

Prototype Review
- Next, a prototype w:Prototype is developed, providing another opportunities for feedback and correction.

Alpha Review
- After the changes are made, the prototype is now considered an Alpha version w:Alpha_version#Alpha where the instruction is fully developed and ready for review by the SMEs and client.

Beta Review
- Changes to the Alpha version gives rise to a BETA version which is piloted to field users/SMEs. This is also known as user testing or usability testing. Here users with no history in the development of the course and who have similar characteristics of the target learner complete the instruction and provide feedback.

As the design of the course progresses, the concentration of the testing moves from content specifics to implementation and achievement of instructional strategies and verifying the objectives have been met.

As you can see, the final formative evaluation culminates in user testing, also know as BETA testing or usability testing. This user testing component is done before implementation for instructional use and is what this module is all about…

What is User Testing?

User testing typically follows alpha testing in the development of web-based instruction. User testing occurs when the courseware is released in real settings to users who are expected to report back any problems they find with the program. The testers should be learners who mirror the target audience. User testing can also used to evaluate the changes made as a result of alpha testing. User testing is typically the final formative assessment.

In e-learning solutions, designers not only need to assess the efficacy of their instruction, but also the quality and performance of the systems that deliver the instruction. For the purpose of this module, user testing in e-learning courses is formative evaluation conducted at the end of the development cycle. User testing of e-learning courses evaluates instructional objectives, software quality assurance, and the overall user experience.

Setting up, conducting and prioritizing information gathered during the user testing process is usually organized as follows . . .

Identify participants
- For user testing of this module, ISD experts and graduate school students were invited to test the module as a sample population of the target audience.

Set up the user testing
- Prepare for user testing by identifying any SMEs, the participants, the instructional materials, and equipment.

Conduct user testing
- Managing the process during the actual user testing. (Check out the Conducting User Testing lesson.)

Evaluate the results
- Check out the Analyzing the Results lesson

Implement the results
- Once you've evaluated the user testing results, you'll want to make the final revisions to your e-learning course.

These are the basics needed to describe formative evaluation and user testing. The next lesson, Review Evaluation Plan will expand on this knowledge. But before going to the next lesson, test yourself yourself on this topic first - Check Your Knowledge 1.

Click Next to check your knowledge.

Back to Topic: Instructional Design > User Testing of E-Learning Courses > Formative Evaluation Overview