Developing and Reviewing Online Courses: Items for Consideration

From:

Developing and Reviewing Online Courses: Items for Consideration

Clayton R. Wright, crwr77 @ gmail.com

Posted on 07 November 2011.

(This document is a revision of “The Criteria for Evaluating the Quality of Online Courses” that the author released in 2003. This revised 2011 document is based on the author’s experience since the original publication date and the suggestions of various e-learning professionals. The document focuses on effective online course components rather than on the qualities of effective instructors, online facilitation skills, or learning outcomes such as the 21st Century Skills developed by the League for Innovation and the Pew Charitable Trusts.)

Quality is an elusive concept, as all who measure it can define it differently. Although quality is affected by the context in which the measurement is taken, the real measure of a quality course is whether it helps learners achieve the stated learning outcomes. In order to do so, the course should comprise a number of components and attributes that will lead to learner success. This document outlines components and attributes of courses that will help you help learners succeed in the online learning environment.

From an instructional design viewpoint, a quality online course is one that enables students to achieve learning outcomes in a flexible and effective manner. Ideally, the learning process will also be efficient, but online activities such as discussions, web searches, and collaborative assignments frequently take time, and efficiency is sacrificed for effectiveness. A quality online course, from a design perspective, takes into account the following factors:

general considerations,
accessibility,
organization,
language,
layout,
goals and objectives,
course content,
instructional or learning strategies,
learning resources, and
evaluation.

Quality is usually achieved through the efforts of a team that may comprise a number of professionals, including the academic program head, project coordinator, subject matter expert, instructional assistant, course writer or developer, course reviewer, instructional designer, graphics or multimedia designer, copyright officer, editor, and computer and information systems specialist. You may decide to extract tasks and roles from this document and assign them to each member of your development team. Even if one person creates an online course, this individual must apply most or all of the skills of the professionals listed above. Often, quality courses take into account the contributions of stakeholders such as learners who will take the course and employers who may hire the graduates of the course.

You should not consider the list of components and attributes as a definitive list but rather a starting point when considering what determines a quality course in yourcontext. When developing a new course or reviewing an old one, you may want to review the items listed here and select those that are more appropriate for your situation. For example, if you are creating an online course from a constructivist point of view or want to implement problem-based learning, items listed under the heading of instructional and learning strategies may need to be modified or be deemed as inappropriate; but, items listed under other sections, such as language and copyright, would be useful to you. You may consider that a few of the items are more applicable for online programs (especially those listed under General Information) rather than for developing and reviewing a single course. Thus, you may want to make a list of items that only applies to programs. If you are an e-learning provider, such as an instructor in a continuing education department who is offering a standalone course not associated with other e-learning courses in a program, you may need to address all the items included in this document in your course. Use this document as a guide when considering what constitutes an effective online course in your situation. Whether you are constructing or reviewing distance or blended courses, this document will help you ascertain the components and features that should be included in your Web-based teaching and learning environment.

Initially, the list of items for consideration may seem overwhelming, but each item was selected because it identifies specific course components, qualities, or procedures proven to be helpful to learners and/or instructors. When using this list, focus first on the major question or questions under each heading. Then, as you review or develop a course, refer to the dotted (or bulleted) items to ensure that these items have been addressed. As you become more familiar with the items on the list, you will need to refer to this document less frequently.

Many courses are evaluated by internal and external reviewers. Consider making up your own list of evaluative items based on the items outlined in this document. Then, ask each reviewer to evaluate a specific aspect of the online course by focusing on the list of specifications under one heading.

1.0 General Information

At the beginning of the course, are learners provided with general information that will assist them in completing the course and in understanding its objectives and procedures?

A brief description of the course includes a goal and learning objectives and/or outcomes as well as the specific skills that must be mastered and exhibited at the end of the course.
A program map and/or rationale indicate how this course is related to other courses in the program and the credit value of each course.
Prerequisites and/or corequisites are identified, including any requirement to be a fluent user of the Internet.
Learners are directed to a website that helps them determine whether online education is appropriate for them.
If the course is intended for international participants, the level of reading proficiency (such as a Test of English as a Foreign Language or TOEFL score) necessary to complete the course successfully in the language in which it is written is specified.
The instructor is identified including expertise and research interest. Learners are provided with a variety of ways of contacting the instructor.
The availability of the instructor is specified, as well as what learners can expect from the instructor. For example, the turnaround time for responses to learners’ questions should be stated clearly.
The developers, reviewers, and facilitators or tutors of the course are listed. Brief biographies may be provided to assure students of the developers’ and facilitators’ knowledge and expertise.
A list of required and recommended resources includes all textbooks, courseware, and online resources necessary to complete the course. If learners must access online resources, instructions are provided for locating and accessing these resources, including password information. If the materials are located in a library, learners are told whether the items are available by e-reserve or physical reserve and the limitations on these reserves. If your student cohort comprises individuals who may be geographically distant from a library, refrain from relying on physical resources that may be found in only one location.
Special requirements are identified, such as recommended modem speed or Internet bandwidth, hardware (computing speed and storage capacity), software (including an e-mail program and file formats for handling assignments and other attachments), and plug-ins. If learners do not have access to a reliable Internet connection or suitable bandwidth, can they receive videos and other large files via compact discs (CDs), digital video discs (DVDs), or universal serial bus (USB) keys or flash drives?
The estimated time (i.e., hours per week) required for completion of the module and/or course is stated, along with its overall duration. If feasible, a timeline is provided that outlines dates by which specific activities must be completed.
Guidelines for participating in online discussions, also referred to as “netiquette,” are provided, as well as suggestions for handling incoming e-mail, e-mail attachments, viruses, and e-mail filters. If there is an online discussion area or chat room, the procedures or expectations for language, quality of discourse, and length of postings should be delineated. Participants must be informed as to whether they must use their real name or whether they can use a pseudonym during discussions.
Learners are informed about group-work activities, the guidelines for forming groups, grading criteria for group participation and assignments, and their responsibilities as group members. If possible, the grading rubric/grid and/or criteria for each grade are outlined.
The introduction to the course takes into account the learners’ backgrounds, ability levels, and expectations, including their personal learning goals and objectives, or specifies the attributes of the learners for whom the course is designed. Learner attributes may include the level of prior knowledge of the subject matter, experience with related materials, and skills already developed.
The availability of technical support is stated, and links to online technical information are provided. The hours during which technical support is available are clearly identified, including the time zone.
Links to institutional learner support services such as a mathematics help centre, writing centre, library, and counseling services are provided and learners are encouraged to use these resources. Off-site learners should be entitled to the same services provided to on-site users.
Learners are directed to a source for answers to “frequently asked questions” pertaining to online learning. This source may provide information covering many of the items listed above, as well as items related to plagiarism, virus protection, and firewalls.
Learners are informed about their right to privacy and the conditions under which their names or online submissions may be shared with others.
Learners are informed about the institution’s or course provider’s information and technology policy as well as online or e-learning policy which outlines acceptable behaviour and the responsibilities of the learner in upholding the procedures and rules that are included in the various policies.
Learners are informed if information about their course webpage visits, participation in discussions, and other online activities are gathered for institutional research purposes.
Some online courses require learners to create their own online profiles that can be viewed by the instructor and those registered in the course. Thus, learners must be informed about the type and size of documents, files, and folders that they are able to upload and how this operation can be done.
A copyright statement or disclaimer clearly identifies the owner(s) of the course and the source(s) of the material students are about to use. Learners should also be informed about their responsibilities in terms of use, distribution, and access to the copyrighted materials. The ethical and legal ramifications of using copyrighted works should be discussed or links provided to such a discussion.

2.0 Accessibility

How accessible is the course material? Can learners find information quickly?

Has the instructor ensured that the infrastructure and server can handle the number of learners enrolled in the course and the size of the documents or folders students will be working with and saving?

The function of each icon or button is explained and/or is naturally evident to the learners.
A detailed table of contents includes objectives, learning outcomes, topics, and assessments.
Every section of the course or module begins with a preview, overview, or summary.
Every page is linked to the previous page, the start of the module, the beginning of the course, and to e-mail links so that learners may contact instructors and other learners for clarification and discussion. These links must be checked to ensure that they are working and it should be possible for the learners to return to the original spot where they clicked on the link. Some course management systems do not allow designers to place a link on every page, but links may be placed on the screen frame or interface page.
Links within the course are provided to other parts of the course.
Links are provided to special editors (e.g., an equation editor or special language characters) that learners can use when completing their assignments.
Page headers or footers identify where the learner is in the course.
An index lists key words or topics.
A glossary defines uncommon or technical terms used in the course and may provide links to sources of supplementary information.
Consideration is given to learners who may have visual or auditory challenges.
The online course complies with accessibility laws and follows the Principles of Universal Instructional Design (UID).
The material can be accessed conveniently on a handheld device or alternatively, platforms on which the material is intended to be accessed are clearly stated.

3.0 Organization

Is the material organized in such a manner that learners can discern relationships between parts of the course?

The table of contents gives an accurate indication of how the material is arranged. Instead of a table of contents, a graphic organizer or chart may be used as this may help learners see the relationships between topics and modules.
The organization or sequencing of the content is appropriate for the subject matter and the intended audience. For example, content may be sequenced in several ways including chronological, hierarchical or pyramid, utilization-related or step-by-step, spiral, inquiry-related, and generic or isolated blocks.
Units of instruction or topics are divided into subunits or subtopics.
Subtopics are related to main topics.
The organization of components is consistent throughout the course. For example, each module may have the following sections:

introduction,
objectives,
pretest,
directions,
explanatory text including learning activities such as case studies,
suggested answers for learning activities and links to additional information,
module summary,
self-test,
self-test answers,
references,
additional readings, and
module assignment(s).

6. Required course elements. References and links to other parts of the material or external sources are accurate. The number of external links should be considered carefully as too many links may cause learners to lose their place within the course.

4.0 Language

Is the level of the language used appropriate for the intended participants?

The items below not only apply to written material, but also to video and multimedia content.

The writing or presentation style is clear and direct.
Instructions are stated simply and are easy to understand.
Clear directions are given.
Familiar or common words are used.
A conversational tone employs the second person: you, not the learner.
Verbs are in the active, not passive, voice; for example, Maslow developed the theory, not the theory was developed by Maslow.
Sentences are short.
Paragraphs are brief.
Numbers are used to identify sequential steps in a task or process.
Dots or bullets are used to list items that are not prioritized or sequential. However, when a list of non-prioritized or non-sequential items contains more than eight or ten items, it may be easier for the learners if the items are numbered.
Dots, dashes, and numbers are used consistently.
The tone of the writing or presentation is supportive and encouraging.
Terms are used consistently.
Abbreviations and symbols are defined.
Conventional technical terms relevant to the level of the learner or geographic location are used where appropriate. For example, which term is best for your learner – USB key, thumb drive, flash drive, or portable storage device?
Spelling and grammar are consistent and accurate. If the information is presented in an audio form, the pronunciation of terms is consistent.
The writing or presentation conveys no explicit or implicit bias relative to age, culture or ethnicity, race, gender, or sexual preference. However, biased opinions may be included as examples if they are relevant to the course content. For example, you may require learners to discuss different views of a topic after you have presented opposing or biased views.

Has the course material been edited for grammar, language, and content verification? Content verification can refer to the accuracy, breadth, and depth of the information to be presented to a specific target audience.

5.0 Layout

Does the layout facilitate learning? Is the material attractive and appropriate for the course content and the intended audience?

Note that it may be impossible to design a page that is displayed in the same manner on every computer screen; a simple layout is best if the equipment and software available to the learners are unknown.

The layout is appropriate for the content and intended audience.
Navigational icons or cues are used consistently. Navigation is explained or demonstrated, especially when it may differ from navigation in other courses in the program.
Links to external materials open in a new tab or window. This method is used consistently and explained to students. If this method is not used, students may lose their place in the module and may find it difficult to return to the original page of departure. As the permanence of external materials may be in doubt, one option is to cache the documents and link to the cache.
The typeface is appropriate for the content and common to all programs and computers, such as Times Roman or Arial.
Bold-face type is used sparingly, to highlight important terms, for example.
Capital letters and underlining are not used for emphasis. Underlining is used only for hyperlinks.
Key words are highlighted, especially when they are first used.
Headings and subheadings are used to organize content.
Consideration has been given to the numbering of sections and subsections and to the logical naming of files, so that the name of a file saved locally or visible as a URL gives useful clues about its contents.
The format is uncluttered and includes white space.
Ragged right margins are used or letters are kerned (evenly spaced on the line). The use of ragged right margins may lead to uneven line lengths, but the spacing between words and letters are consistent thus making the words easier to read.
The contrast between text and the background material makes the text legible. If learners are expected to print the materials, the text may be a shade off of deep black so as to save on ink cartridge consumption.
Colour is used effectively. Note that the colours used should not be indicative of any cultural or political bias or meanings. This can be difficult to achieve when the learners may have different cultural backgrounds. However, in most cultures “red” is used to grab attention and/or signal a warning. “Green” is used to signal safety or identify environmental factors.
Graphic elements such as diagrams, tables, and photographs illustrate or clarify information presented in the text. All these illustrative objects are consistently and appropriately captioned.
Illustrations can be seen easily on a computer screen, and image files are used that can accommodate different download speeds. Care is taken to minimize the file sizes so that those with limited bandwidth can obtain the information that they need.
Text explaining a graphic is aligned with the non-textual material.
Labels within a diagram are fixed so that their relative position does not change as the size of the image is altered.
To assist those who may use screen readers, a description of each illustration and graphic is available as an ALT tag. (An ALT tag is an HTML tag that represents an object. It provides alternative text when non-textual elements cannot be displayed.) Note also that if learners use a software reading program, such as Natural Reader, the text should be in a plain format as the software may not read italicized text or text presented in boxes.
Videos have the option of turning on or off closed captioning.
If frames are used in all web pages, their use is consistent.
The material is displayed attractively.

6.0 Goals and Objectives