Sample Requirements Definitions of Program Evaluation Criteria
Criteria / Possible Requirements Definition Considerations
Vendor instructional philosophy / §  Persuasive, well-articulated strategy for meeting objectives
§  Practice vehicles
§  Emphasis on applying content to communities of practice
§  Use of behavior modeling
§  Focus on reflective practice
Vendor design process / §  Continuous interaction with user community
§  Extensive real-world beta-testing
§  Needs assessment
Vendor customer references / §  Customers would recommend
§  Customers would rank as best vendor
§  Average length of customer relationships
Vendor services / §  Technical support availability and cost
§  Implementation planning consulting availability and cost
§  Hosting services
Vendor course library / §  Breadth or depth of library
§  Compatibility with other WBT vendor programs or Learning Management Systems
§  Pricing structure for library access
Prior knowledge assessment / §  Knowledge type-appropriate prior assessment
§  Program can be customized for different levels of prior knowledge
Post-testing / §  Knowledge type-appropriate post-test
§  Useful feedback and/or review
Learning objectives / §  Match with your needs
§  Emphasis on conceptual understanding
§  Emphasis on behavioral objectives
Target audience / §  Experience level
§  Prior Knowledge assumed
§  Acceptance of content
§  Functional expertise
Customizability / §  Your company name and logo
§  Your company policies and procedures
§  Your company forms
§  Messages from your senior executives
Content application to community of practice / §  Tools available to take back to work (forms, templates, etc.)
§  Ease of sharing content with co-workers
§  Emphasis on “doing work” in the program
Usability / §  No training required to use the system quickly and easily
§  Useful help system
§  Lack of fatal errors
§  Ability to leave and return to a lesson
Readability / §  Text “chunked” for easy reading
§  Lack of scrolling
§  Printable versions available for large amounts of text
Ability to use as a reference / §  Course structure is flexible for reference and/or “bookmaking”
§  Provides printable “cheat sheets”
§  Provides other current resources outside of course itself
Use of multimedia / §  Engaging, promotes attention
§  Memorable representations
Interactivity / §  Electronic page turns
§  Simulated decision-making process
§  Learner drives the learning activities
Personalization / §  Recognizes the user and their progress through the course
§  Presents different material based on the profile of the learner
§  Organizes material differently based on the learner profile
Technology infrastructure requirements / §  Server specifications
§  Network requirements
§  Client specifications
Online accessibility / §  Modem-friendly
§  Reliability statistics
§  Security
Offline accessibility / §  Downloadable for anytime, anywhere learning
Tracking, prompting, and reporting capabilities / §  Tracking of use by learner profile
§  Email prompting capability
§  Notifications of progress to HR or management
§  Compatibility with HRIS data
Help for administrators and users / §  Help systems
§  Technical support
§  Internal IS support requirements

Janette Cullinan

Stanford university