Web-Based Training

Standards

Division of Judicial Education

Office of State Courts Administrator

Version 2 (June 26, 2007)

Table of Content

I.Introduction

1.1. What is WBT?

1.2.Roles and Responsibilities

II.Development Process

2.1. Analysis

2.1.1.Project Timeline (Appendix C-1)

2.1.2.Content Analysis (Appendix C-2)

a.Overall Course Goals

b.Learner Profile

c.Pre-requisite Courses

d.Content Outline

e.Learning Objectives & Resources

2.1.3.Evaluation: ET Review/SME Validation

2.2. Design (Content Development)

2.2.1.Content Development (Appendix C-3)

2.2.2.Evaluation: SME Review/Course Content

2.2.3.Instructional Design (Appendix C-4)

2.2.4.Storyboard (Appendix C-5)

2.2.5.Evaluation: SME Review/Storyboard

2.3. Development (Course Development)

2.3.1.Evaluation: SME Review/Weekly Status Report (Appendix C-6)

2.3.2.Evaluation: Functional / Quality Review

2.4. Implementation (Pilot-Testing)

2.5. Evaluation

2.6. Final Course Activities

III.Course Structure

3.1. Course Main Structure

3.2. Course Content

3.2.1.JIS Procedure

3.2.2.Summary

3.3. Check Your Knowledge (Pre-Test)

3.4. Check Your Learning (Post-Test)

3.4.1.Learner Mastery Level

3.4.2.Feedback

3.5. Glossary

Appendix A – Instructional Writing Styles

Appendix B – Design Specifications

Appendix C - Templates

C-1. Project Timeline

C-2. Content Analysis

C-3. Content Development

C-4. Instructional Design

C-5. Storyboard

C-6. Project Weekly Status Report

I.Introduction

1.1. What is WBT?

Web-based Training (WBT) is a form of computer-based training delivered over a public or private network and accessed via the World Wide Web (WWW). Judicial Education’s WBT courses are delivered through Judicial Education Web Learning System (JEWELS). JEWELS is accessed through the Missouri Courts Website ( currently under Services – Judicial Education – JEWELS.

WBTs are designed around learning objectives or expected outcomes and content is presented in a structure that allows self-directed, self-paced instruction for any topic. Media elements used are text, graphics, audio, video or other animations. A course assessment is essential to measure learner performance and course effectiveness.

WBT standards are necessary to ensure the consistency of training delivered to court staff and to maintain the product integrity of courses delivered on JEWELS. WBT development requires significant resources and Judicial Education strives to deliver effective training using sound instructional methods.

The underlying purpose of any Judicial Education WBT or e-learning project is to:

  • Increase the quality, consistency, and availability of training
  • Maximize content sharing and reuse
  • Reduce classroom time by providing alternative training and prerequisite materials
  • Increase training effectiveness through best practices or standards

Roles and Responsibilities

The following are roles and responsibilities of project collaboratorsand Judicial Education development team during the process:

Court Services / OSCA Internal Clients / Judicial Education

Project Sponsor

  • Identify training need for OSCA or the courts
  • Provide strategic direction for outcome of the course/training
  • Assign Subject Matter Experts (SME)
  • Oversee SME activities with development team
  • Approve or delegate approval of course objectives and the course
/

Judicial Ed Division Director

  • Judicial Ed Director can also act as a project sponsor.

Project Manager

  • Responsible for the overall quality of the project and overall management of Judicial Ed team members – Education Technology Coordinator
  • Develop project timelines and distribute project plan
  • Analyze learning outcome goals and training needs
  • Plan and manage resources for the project
  • Coordinate all phases of development
  • Lead communication and project liaison
  • Guide the review and approval process
  • Develop formative and course evaluation process
  • Conduct or direct final course implementation activities

Court Services / OSCA Internal Clients / Judicial Education

Project Coordinator

  • Manage and coordinate SME activities with development team
  • Communicate with Development Coordinator for day-to-day activities
/

Development Coordinator

  • Responsible to project manager for day-to-day production activities – Judicial Ed Team Lead
  • Track development time
  • Conduct or direct functional testing – including LMS functional testing
  • Track review comments and revision implementation
  • Provide review feedback and revision status to project manager, SMEs, and development team
  • Draft progress and status reports

Subject Matter Expert (SME)

  • Contribute to learner profile and course objectives
  • Contribute and validate core course content
  • Identify source materials and other content references
  • Review storyboard and draft course for accuracy
/

Content Developer

  • Review content analysis
  • Develop course outline
  • Document course content for SME review
  • Create and revise script for storyboard

Instructional Designer

  • Structure and organize course content
  • Design instructional strategies including interactive exercises and activities
  • Create assessment / evaluation questions
  • Finalize storyboard content delivery
  • Supervise pilot testing and review

Course Developer: Graphic Designer

  • Responsible for course design development, including layouts, images, backgrounds, etc.
  • Design graphical presentation of course content
  • Develop storyboard
  • Design, develop, and incorporate interactive activities for the most effective delivery

Course Developer: Programmer

  • Responsible for overall development of course based on the approved course design
  • Responsible for programming of course
  • Design, develop, and incorporate interactive activities for the most effective delivery

Course Developer: Functional Tester

  • Responsible for functional testing at each stage of development
  • Responsible for LMS integration

Course Developer: Others

  • Copy Editor
  • Audio/video Producer
  • Narrator

II.Development Process

Development process of a WBT course follows a systematicprocess which consists of analysis, design, development, implementation, and evaluation.

  • Analysis: Define the needs and constraints of the course
  • Design: Specify learning content & activities
  • Development: Begin production, conduct quality & SME review, and revise
  • Implementation: Test the course for use by the desired audience
  • Evaluation: Evaluation occurs throughout the course development

2.1. Analysis

The following are completed and determined during the Analysis phase:

2.1.1.Project Timeline(Appendix C-1)

The overall timeline is determined, including concurrent activities and person in charge of each stage

2.1.2.Content Analysis(Appendix C-2)

The content analysis is drafted by the Subject Matter Experts (SMEs).

a.Overall Course Goals

To set the general direction and need for the course in analyzing the training need, SMEs should review the job responsibilities of the intended audience, prior training or knowledge, and identify any new concepts or terminology that will be introduced to learners.

b.Learner Profile

Significant characteristics of the target audience and their learning environment are determined at this stage. These characteristics may include but are not limited to:

  • Prior knowledge or experience with the proposed training material
  • Job responsibilities
  • Reading and education level
  • Computer and Web skills
  • Learning environment and supervisor support

c.Pre-requisite Courses

It is important to determine any pre-requisite courses at this stage to access prior knowledge and ensure the course being designed will augment the overall training curriculum for the learner.

d.Content Outline

The course outline includes a list of course content or topics to be included in the course to achieve the overall goals.

e.Learning Objectives & Resources

Learning objectives are detailed statements of what the learners will be able to achieve or demonstrate as a result of completing a course. Thus, objectives are statements of learner behavior. They describe the result of the learning process rather than what or how the learner will be taught.

Example:

  • Given a uniform citation, learners will be able to initiate a citation using CVAQMVI with over 75% accuracy.
  • Given a uniform citation, learners will be able to identify required information for CVAQMVI from a uniform citation.

2.1.3.Evaluation:ETReview/SME Validation

ET reviews and sends it to SMEs tovalidate the content analysis prior to the design stage of the project.

2.2. Design (Content Development)

The Design phase creates the "blueprints" for the instructional experiences. During this phase, the following questions are addressed:

  • What content will be included
  • How the content will be presented

2.2.1.Content Development(Appendix C-3)

The contentdevelopment document describes whatcontent will be included in the course. The purpose of this document is to provide the content in a clear, concise format or process steps so the SMEs can validate the accuracy and completeness of the content.

Content Development Tool: Word file –Content (Appendix C-3)

2.2.2.Evaluation: SME Review - Course Content

SMEs review and validate the accuracy and completeness of the course content. It is important to ensure the content is as complete as possible at this stage because it will define the scope of the project. Content added after this review will impact the development timeline.

2.2.3.Instructional Design(Appendix C-4)

During the instructional design phase the course objectives, course outline, and content documents are reviewed and the following questions are addressed:

  • How the learning materials will be structured
  • What instructional strategies will be used
  • How learning will be assessed

a. How will the learning materials be presented? - Is content structured in a way to help learners assimilate new information and concepts?

In order to maximize the effectiveness of the course the course content will be organized and structured into a framework of lessons which best facilitate learning. The course structure should ensure learners know at all times what they are trying to learn, what is expected of them in order to achieve that learning, and when they have reached that goal. Learning objectives and the content are sequenced according to the learner’s needs so that the instruction begins with developing knowledge and then progresses to applying that knowledge after it has been understood.

A logical course structure might be:

  • Proceeding from what learners know to what they do not know
  • The logical or historical development of a subject
  • Proceeding from concrete experiences to abstract reasoning

b. What instructional strategies will be used?

Based on the learning objectives the instructional strategiesare determined at this stage. Instructional strategies, such as tutorials, lecture, drill and practice, demonstration and situations, topical reviews, etc. provide an approach to organizing and presenting information.

c. How will learningbe assessed?

Learning assessments measure the competence or capability of learners in order to determine whether or not they have met the course objectives. Based on the learning objectives, assessment strategies are decided at this stage and will be used to measure learning outcomes and the effectiveness of the course. These strategies may include, text entry, multiple choice, drag & drop, hotspot (click on), true/false, yes/no questions as well as practical simulations of process.

2.2.4.Storyboard(Appendix C-5)

A visualized storyboard provides the overall look of the course, including the graphical presentation of the course content. The storyboard is developed by the instructional designer, the copywriter, and the graphic designer and will serve as the prototype of the course.

The storyboard should provide a combination of text and graphics or graphic descriptions that convey all the necessary information about the delivery of course content. Storyboards describe in detail all images, animations, movie segments, sound, text and navigational paths. The more complete, detailed, and accurate they are, the fewer the assumptions, questions, delays, confusion, and errors that occur later, during the development phase.

Storyboard Tool: PowerPoint file withText, ResourcesDeveloper’s Note(Appendix C-5)

2.2.5.Evaluation: SME Review - Storyboard

SMEs review and validate the storyboard prior to the development stage of the project. It is important to note that this review finalizes the course design and the content should be complete.

2.3. Development (Course Development)

Development is the production phase of the course.In addition to the team members involved in the design phase; course production includes all of the course developers; graphic designer, audio/video, narrator, copy editor, programmer, quality reviewer, etc.

Designers and developers work to incorporate the visual design of the instructional materials to best support and facilitate learning and determine appropriate interactions which engage learners and encourage additional learning.

The following processes are included in the Development phase:

  • Develop a preliminary course
  • Conduct quality / functional review
  • Share weekly updates / changes between SMEs and developers

2.3.1.Evaluation: SME Review/Weekly Status Report(Appendix C-6)

Communication is an important part of the evaluation/review process. Team members, particularly cross-divisional teams, should remain in contact with each other throughout development in order to quickly resolve issues. Team meetings and informal communication such as phone calls, e-mail, instant messaging are effective ways for developers to communicate, but a more formal collaboration is necessary with SMEs and the project sponsor.

Throughout the development phase a weekly status and issue log will be posted and will be accessible to all team members. As a section or component of the WBT is complete it will be posted and a communication sent to SMEs so they can review that progress and make comments. Status reports and course components for review will be posted on Fridays and SMEs will have one week to review and respond to the issue documented in the status report.

2.3.2.Evaluation: Functional / Quality Review

Functionaland quality reviews are conducted to ensure the functionality and quality of the course. The quality review team usually consistent of the development coordinator, a functional tester, and copyeditor.

2.4. Implementation (Pilot-Testing)

During the Implementation phase, the course is tested for use by a sample group of users representing the target audience for the course. Pilot-testing ensures that the course components will function as designed for the intended learners.

SMEs identify a potential test group of reviewers based on the learner profile for the course. Judicial Education will conduct the pilot-testing and recommended changes from the pilot-testing will be presented to SMEs.

2.5. Evaluation

The Evaluationoccurs throughout the course development.

  • ET review / SME validation for Course Goals,Learner Profile, Pre-requisite CoursesContent Outline, Learning Objectives & Resources
  • SME review and approval for Content Development
  • SME review for Storyboard
  • Functional / Quality review in the development process
  • SME review for PreliminaryCourse(weekly)
  • Pilot Testing
  • SME review for Final Version
  • Project Sponsor & SME approval for Final Version

2.6. Final Course Activities

The final course activities are performed once the sponsor validates or approves the course.

  • WBT published on JEWELS
  • Course announcement sent to learners

III.Course Structure

3.1. Course Main Structure

The basic structure of a Web-Based Training course will include an introduction to the course (“Getting Started”), lessons, course assessments, and resources that include course tips, glossary, and learning guide.

A pre-test (“Check Your Knowledge”) will allow learners to measure their entry knowledge on the subject. No score will be given at the end of the pre-assessment. It is a tool that gives the user clues about the course content and piques their interest and involvement. The pre-test is not required for completing the course, but a comparison score is provided at the end of post-test (“Check Your Learning”).

Main Menu Page – SAMPLE

3.2.Course Content

Course content is organized into lessons that include topical sections. A topic section may include interactive topical review questions to emphasize key learning objectives.

3.2.1.JIS Procedure

A procedure using a new JIS form is introduced using a DEMO with audio followed by a hands-on exercise to reinforce the learning.

3.2.2.Summary

Review activities will be included depending on the complexity of the lesson.

A summary of each lesson will restate the key learning points.

Summary Page – SAMPLE

3.3.Check Your Knowledge (Pre-Test)

The pre-test includes randomized questions from the post-test (“Check Your Learning”).The learner is given only one try with no correct or wrong answer feedback during the pre-test.

The pre-test is not required to complete the course. If the learner completes the pre-test, the comparison scores will be presented at the end of post-test (“Check Your Learning”).

3.4.Check Your Learning (Post-Test)

Post-test questions are developed based on the learning objectives of the course.

The post-test is active when all lessons have been completed. The learner is given only one try for each question. They will receive correct answer and incorrect answer feedback which reinforces the learning objective to be mastered.

3.4.1.Learner Mastery Level

The recommended level of mastery for a course is a score of 75% on the post-test.

Pass on the Post-Test - SAMPLE

Fail on the Post-Test - SAMPLE

3.4.2.Feedback

Feedback is given for bothcorrect answer and incorrect answers. Feedback indicates whether the learner answered the question correctly or incorrectly and states what the correct action was or should have been.

Example:

Correct. Clicking on the Exit icon exits the form and returns you to the previous form.

Incorrect. The correct answer is A because in order to exit the form and return to the previous form you should, Click the Exit icon on the toolbar.