Assignment 8 - Creating My Own Evaluation Criteria

Jessica Foley (Hingle)

November 27, 2007

Checklist Description / Mnemonic or Evaluative? / Why?
Customer Satisfaction Reviews / Evaluative / Evaluates our effectiveness at satisfying the customer on a number scale (judging value or merit). If the score is low, changes are required.
Packing list for conference / Mnemonic / Simply helps us remember all of the items required for the conference. Does not mention value or merit of items.
Internal Product Review Sheet / Evaluative / Used by our team to internally review products prior to release to the customer with written judgmental comments next to each item.
Award Nominee Review Sheet / Evaluative / Used by our company to screen the nominees for yearly awards. Numeric scale with comments is included.
ACTD Process Flow Chart / Mnemonic / Simple flow chart that details the ACTD process. No discriminating criteria, simply move to the next step when the current step is complete.

Activity 1
Activity 2

Software Evaluation Checklist Part 1

  1. Acquisition & Technical Logistics
  2. Cost
  3. Availability Date
  4. Deployment timeline
  5. Minimum hardware requirements
  6. Minimum software requirements
  7. Intended Audience
  8. Education level
  9. Specific Pre-existing knowledge/skills
  10. Dedicated learning time
  11. Community of Interest
  12. Content
  13. Accurate
  14. Can it be updated?
  15. Comprehensive
  16. Methods & Activities
  17. Objectives stated
  18. Variable levels of difficulty
  19. Logical division of information
  20. Adequate Review of sections
  21. Assessment for sections
  22. Free-play section
  23. Overall assessment
  24. Utility
  25. Relates to mission
  26. Has been through MUA
  27. Requested?
  28. Sponsored?
  29. Product already deployed?

Software Evaluation Checklist Part 2

I chose to create a checklist which would determine which software should be considered for an expert review. This checklist would narrow down a large group of software packages to a small group.

Please see the next two pages for the Evaluation Checklist.

Reviewer Name:Review Date:

Product Name:

Product Description:

Intended Audience:

Criteria / Yes / No / Comments
Acquisition & Technical Logistics
  1. Is the cost within the projected budget? Note approximate cost.

  1. Is the product available on an acceptable date? Note approximate date.

  1. Is the deployment timeframe reasonable? Note approximate length.

  1. Do the current systems meet the minimum hardware requirements? Note any discrepancies.

  1. Do the current systems meet the minimum software requirements? Note any discrepancies.

Section Total: / Each “Yes” answer is worth 5 points.
Audience
  1. Is the education level of the material appropriate for the our learners? Note any discrepancies.

  1. Are pre-existing skills or knowledge on par with our learners? Note any discrepancies

  1. Is the time length of the program appropriate for our audience? Note any discrepancies.

  1. Is there a pre-established community of interest for this product within our learners? If so, note.

Section Total: / Each “Yes” answer is worth 4 points.
Content
  1. Is the content accurate? Note any discrepancies.

  1. Can the content be updated? Note any discrepancies.

  1. Is the content comprehensive? Note any discrepancies.

Section Total: / Each “Yes” answer is worth 5 points.
Methods & Activities
  1. Are the objectives clearly stated? Note any discrepancies.

  1. Are there variable levels of difficulty? Note any discrepancies.

  1. Is there a logical division of information? Note any discrepancies.

  1. Is there an adequate review of each section? Note any discrepancies.

  1. Is there an assessment for each section? Note any discrepancies.

  1. Is there a “free-play” section? Note any discrepancies.

  1. Is there a capstone assessment? Note any discrepancies.

Section Total: / Each “Yes” answer is worth 3 points.
Utility
  1. Does the product relate to the Agency’s mission? Note any discrepancies.

  1. Has the product been through a Military Utility Assessment? If so, note date and results.

  1. Was the product specifically requested? If so, by whom?

  1. Is the product already sponsored by a Combatant Command? If so, which Command?

  1. Is the product already deployed elsewhere in the Armed Forces? If so, where?

Section Total: / Each “Yes” answer is worth 3 points.
Grand Total:

Final Comments:

Reviewer Signature:Date:

Software Evaluation Checklist Part 3

The categories and subcategories that I chose for my software evaluation checklist are a result of examining the undocumented steps that my division takes when evaluating software for use in the military arena, combined with the knowledge I have gained throughout this course. We start with over 100 candidates and narrow the field to approximately 16 that are closely scrutinized. In the end, only a few make it to acquisition.

To start with, I listed the main categories that my division discusses when software candidates: logistics, content, and utility. The logistics category consists of the “make it or break it” items. We do not have the authority to adjust our budget, nor the finances to alter the hardware or software that the warfighter has out in the field. The content of learning applications has always been high on our list because it is important that the warfighter learn the exact needed skill, not a similar skill. However, this category can be supplemented with additional information if needed, so it was not weighted as high as the logistics category. The utility category contains optional items that make software approval easier within our field, but those items are not absolutely essential. Therefore, they are weighted less.

After having gained knowledge from my ITMA courses, I added in the two other categories: audience and methods. I have learned the importance of fully understanding your audience and how the methods that are utilized to impart knowledge to that audience can affect how much knowledge is transferred. These are two items that have not been considered in the past within my work environment, but I am going to work towards officially incorporating them.

Overall, I am pleased with my checklist. It is comprehensive, easy to use, and successfully discriminating. I feel that it will assist us with selecting high quality software candidates for delivery to the warfighters.