/ Government of Newfoundland and Labrador
Office of the Chief Information Officer
Corporate Services & Projects: Project Management Office

Client Test Strategy

999999 - Widget ProductionControl System Project

This page intentionally left blank

Client: / Production Coordination Division, Department of Widgets
Author: / Walter Weis
Status / Final
Date: / 2014-05-30
Revision Log
Date / Version / Description of Revision / Author
2014-04-07 / 0.1 / Initial Draft. / Jane Juniper
2014-04-20 / 0.2 / Updated with feedback from Sponsor / Jane Juniper
2014-05-09 / 0.3 / Updated with feedback from Delivery Manager and Director of AD / Jane Jumpier
2014-05-25 / 1.0 / Final version / Jane Juniper
Prepared By
Project Manager / Jane Juniper 2015-04-25
(name) (signature) (yyyy/mm/dd)
Reviewed By
Delivery Manager / Morgan Freebird 2014-05-26
(name) (signature) (yyyy/mm/dd)
Approved By
Project Sponsor
<include title> / Peter Puddin 2014-05-30
(name) (signature) (yyyy/mm/dd)

Table of Contents

Acronyms and Glossary

Introduction

Purpose

Objectives

Scope

Stakeholders

Document Repository

Training

Test Phases

Approach

Test Environment

Test Data

Software & Defect Release Plan

Defect Tracking

Client Unit Testing

Software Deployment (Smoke Testing)

Functional Testing

Performance Testing

Web Standards Testing

Regression Testing

Business Process Testing

User Acceptance Testing

Test Schedule

Client Test Strategy
Template Version 1.0 2014-05-29 / Page 1 of 15
/ Government of Newfoundland and Labrador
Office of the Chief Information Officer
Corporate Services & Projects: Project Management Office

Acronyms and Glossary

The following table includes definitions for any unique symbols or notations that are used in the document, which may cause confusion with the intended message, or may result in multiple interpretations of some key terms.

Term / Definition
WP / Widget Production
WPCS / Widget ProductionControl System
NLWPD / Newfoundland and Labrador Widget Production Division
AFS / Application For Service
POS / Point of Sale – this is the order processing module in WPCS
UAT / User Acceptance test – tests carried out by the business users to verify that the software product meets the expectations of the users and the business.
NWC / Nation Widget Centre Software

Introduction

Purpose

This document identifies the overall approach for the Widget ProductionDivisionto test the overall solution as delivered by Nation Widget Centre Software for the Widget Production Control System,to ensure the client’s requirements have been met.

Objectives

The objective of testing is to confirm that the Widget Production Control System performs as intended and meets the requirements as outlined in project deliverables as noted below and the Business Requirements;

Document / Owner / Contact
999999 - WPCS Test Plan v1 0.doc / NWC Software / Jane Jupiter
Newfoundland Test Plans-V1 0.doc (test cases) / NWC Software / Morgan Freebird
999999 WPCS Test Cases / OCIO / Jane Jupiter
999999 - WPCS Requirements / OCIO / Jane Jupiter
999999 - WPCS Web Detailed Specification / NWC Software / Jane Jupiter
999999 WPCS Project Charter / OCIO / Morgan Freebird
999999 WPCS Detailed Architecture Design / OCIO / Jane Jupiter

Scope

In Scope:

The Test Strategy scope includes testing the completed solution for Widget Production Control System to ensure all requirements have been fulfilled. The following are types of testing to consider;

Client Unit Testing

  • Software Deployment (Smoke Test)
  • Functional Testing
  • Interface Testing
  • Performance Testing
  • Web Standards Testing
  • Business Process Testing, including Financial Processes
  • Regression Testing

User Acceptance Testing (UAT)

  • End to End Business Process Testing , including Financial Processes
  • Regression Testing

Out of Scope:

Testing the functionality within the ePayment Broker solution is out of scope.

Testing the FTP of files to National Widget Center is out of scope.

Stakeholders

The Stakeholder groups for testing are as follows:

Stakeholder / Role
Widget Production Division / Test execution and management.
  • Tommy Toes –Tester / Backup Test Coordinator
  • Mary France – Tester
  • Frank Fleming - Tester
  • Susie Quisie - Tester
  • Charles Brown - Tester
  • Peter Puddin – Testing Business Prime

OCIO / Test planning, execution, management and support.
  • Teri Toberone - OCIO Test Coordinator.
  • Nancy Webmaker – OCIO Web Standards Prime
  • Paddy Payer– OCIO ePayment Broker Prime
  • Jane Jupiter – OCIO Project Manager

Nation Widget Centre / Test support, defect resolution and software release management.
  • Donald Cow - NWC Software Project Manager
  • Nick Hogg – NWC Software Test Support Prime
  • Jonathan Frogger – NWC Software Test Support Prime

Document Repository

A working folder for the OCIO and Production Coordination Division to store and control testing documents is \\psnl.ca\ocio-dfs$\OcioShare\WPCS Project Share\

Training

  • NWC Software will be on site June11th -June 15th, 2014 to complete Training and initial testing with the NLWPD test team. This session will be held off-site in the OCIO Training Lab, Higgins Line.
  • The planned activities for the week include;
  • June11th – Initial setup and planning , attendees include;

-Jane Jupiter

-Morgan Freebird

-Tommy Toes

-NWC Resource

  • June 12th – NWC Web Solution training for all NLWPDTesters; two sessions, morning and afternoon.

- Review the testing process and testing documents.

- Morning session includes the following resources;

-Tommy Toes

-Mary France

-Susie Quisie

-June Jupiter

- Afternoon session includes the following resources;

-Charles Brown

-Frank Fleming

-Peter Puddin

-Morgan Freebird

-Waldo Whereis

-June Jupiter

  • June 13th–15th - Initial Client testing;attendees include the following resources;

-Tommy Toes (60%)

-Mary France (60%)

-Charles Brown(60%)

-June Jupiter (100%)

Test Phases

The Widget Production Control Division will perform Client Unit Testing and User Acceptance Testing (UAT) of the Widget ProductionControl System. The following table provides an overview of each testing phase.

Level of Testing / What is it? / Who does it? / When? / Where?
Client Unit Test / Software Deployment (Smoke Test)
Functional Enhancement Test
Interface Test
Performance Test
Web Standards Test
Business Process Test
Report Defects
Regression Test / OCIO EA Team
NWC Team
NL Project Team
OCIO Web Team / After NWC Functional Integration test is completed. / OCIO Test Environment
User Acceptance Test / End To End Business Process Test
Report Defects
Regression Test / NWC Team
NL Project Team
Business End Users (if needed) / After Client Unit Testing is completed. / OCIO Production Web Environment
Network Testing / Browser Version (current -3)
GNL Network
Wi Fi
VPN / NL Project Team
Business End Users (if needed) / After Client Unit Testing is completed. / OCIO Production Web Environment

Approach

Test Environment

The OCIO Enterprise Architecture Group is responsible for the set up and configuration of the Testing and Production environments with NWC Software being responsible for the installation and configuration of the NWC software and any other software required.

Test Environment

Client Unit Testing will be conducted in a complete and fully integrated test environment containing all software components required for the full solution set including:

  • Client / public facing web pages,
  • ePayment Broker test system, and
  • NWC test databases.
  • FMS (FMS Test and Staging)

Web Production Environment

UAT Testing will be completed in the Web Production Environment.

Test Data

Client Unit Test

The NWC Test database will contain a copy of data from the xyz-test environment as well as any necessary data configuration updates that maybe required to the reference table data. Test orders will be initiated for online AFS requests that are based on legitimate registration records. AFS requests will be based on the conditions defined in the NWC Test Case spreadsheet.

UAT Test Data

UAT testing will be conducted using a selection of actual mail AFS orders as well a selection of orders from the NWC Test Case spreadsheet.

Test Cases

The testing conditions and scenarios that need be executed during Client Unit Test and UAT have been documented in the NWC Test Case spreadsheet. The OCIO Test Coordinator will maintain and update test cases as needed.

Software & Defect Release Plan

NWC Software will coordinate each software release containing necessary system updates with the OCIO Project Manager to ensure the test environment remains stable during the testing cycle. The upgrades will be based on resolved defects as required.

With each new release, NWC Software will update the status of submitted issues to indicate which issues have been resolved and are ready to be re-tested. Any other relevant information will be provided with the release.

Defect Tracking

The following describes the approach for the NWC Software and Production Coordination Division to manage defects encountered during testing.

NWC Software

  • NWC Software uses Abbacus Helpdesk to manage defects.
  • The existing testing queue will be used to manage web testing defects. Any previously reported defects related to the core implementation will be hidden from view.
  • Vital Statistic’s testers will report defects to the OCIO Test Coordinator via email.
  • The OCIO Test Coordinator will log all reported defects in Abbacus Helpdesk.
  • Abbacus Helpdesk is available online at NWC Softwarewill assign usernames and passwords for applicable NLWPDTesting resources.
  • NWC Software aims to review and response to all defects within one business day.
  • NWC Software will resolve defects which significantly impede test execution as timely as possible, including critical or high severity defect.
  • NWC will participate in weekly status meetings with NLWPD Project team to review and discuss resolution for all outstanding defects.
  • Defect severity levels include:
  • Critical: Show stopper; system locks up or shuts down; there is no work around.
  • High: Major functionality not working; there is no work around.
  • Medium: Functionality not working as defined but there is a work around.
  • Low: Minor problems, such as typos or look and feel problems, or minor validation issues.

Production Coordination Division

  • WP will use the “999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet to execute and manage the outcome of each individual test case.
  • The test case spreadsheet provides the detail to execute and maintain the status of each test condition.
  • The Client tester will execute the test and update the test case status (Pass, Fail, Remaining or Delayed) and report any defects found to the test coordinator.
  • The Test Coordinator will update the test case spreadsheet with the defect id obtained from NWC and the defect status.
  • The OCIO Test Coordinator will coordinate and manage test activities and consolidate test cases spreadsheets completed by testing resources.

Defect Tracking Overview

  1. WP testers will report defects they encounter during testing to OCIO Test Coordinator via email as follows:
  1. Create a new email to log each defect detected during testing.
  • Only one defect per email.
  1. In the email subject line add a brief summary of the defect and the severity level.
  2. In the body of the email message clearly document the defect including sufficient details for NWC Software to reproduce the error. Attachments may be added to further explain the defect, such as screen shots and any helpful steps to reproduce the defect.
  3. Send email to OCIO Test Coordinator,
  1. OCIO Test Coordinator will review defect details and forward to .
  1. NWC Software will automatically reply to this email and provide a unique defect id number in the subject line.
  2. The OCIO Test Coordinator will record the unique defect id number in the “999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet and update the defect status to ‘outstanding’.
  3. NWC will provide subsequent software releases and once the defect is ready to be retested, NWC Software will change the ticket status to ‘fixed’ and send notification via email that the issue is ready to be retested.
  4. The OCIO Test Coordinator will change the defect status from ‘outstanding ’to ‘fixed’ for the applicable defect id in the “999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet.
  5. The Test Coordinator will re-issue test case spreadsheets to NLWPDtesters for the new release ready for retesting.
  1. The NLWPDTester will retest all test cases that have a status of ‘fixed’ and complete any necessary regression testing that is determined by the testing coordinator.
  1. The Tester will re-execute the appropriate test cases and notify the test coordinator of defect status (Resolved or Fail).
  1. The OCIO Test Coordinator will reply to the fixed notification email received from NWC indicating the results of testing stating whether the fix was a “pass” or “fail” and update the defect status in the test case spreadsheet.

Client Unit Testing

The objective for this test phase is to conduct testing on a complete, integrated system to verify that the system meets all of its specified requirements (Contract, Business Requirements Document, Detailed Specification and Detailed Architecture Design documents). Client Unit Testing will be completed by NLWPD Project team, OCIO Web Team and support provided by NWC Development team. The test coverage will include:

  • Navigation through web pages;
  • Availability of linked pages;
  • Field validations, edits and lookup lists;
  • Screen functionality (including error and message responses);
  • Informational content correctness and field level help;
  • Web transaction processing - information entered makes it to the shopping cart database in the

proper way and information returned to the user is correct; and

  • Interactions with external systems, ePayment Broker.

NWC will be responsible for supporting all WP testing including defect logging, management and resolution.The following types of testing will be used to ensure the desired test coverage is obtained.

Software Deployment (Smoke Testing)

The objective for this type of test is to determine if the NWC Web Software solution delivered is adequately stable and reliable for WP to proceed with more thorough testing.

  • This test will occur once for each new software release received and deployed.
  • Testing will be limited and include executing some basic AFS orders to confirm all parts of the system are up and running and confirming user access. A small selection of test cases may be obtained from the “999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet.
  • Any defects found should be immediately reported to the Test Coordinator, to determine if the release is deemed inadequate to continue with Client Unit Testing.
  • Defects are to be recorded as previously outlined in the Defect Tracking section in this document.

Functional Testing

The objective of Functional Testing is to ensure the solution functions as intended according to the project’s defined requirements and specification documents. This type of testing will validate that the software coded functions are invoked and behave as expected. It includes a broad range of testing and will include coverage for the following:

Navigation through web pages;

Availability of linked pages;

Field level validations, edits and lookup lists;

Screen functionality (including error and message responses);

Informational content correctness and field level help;

Web transaction processing - information entered makes it to the shopping cart database in the

proper way and information returned to the user is correct;

Interactions with external systems, ePayment Broker - sending and receiving transaction data to and from ePayment Broker;

Transfer/Loading of AFS order data into the POS Module;

POS Order processing; and

Order Status database updates.

  • The OCIO Test Coordinator will lead Functional testing, delegating test cases required for execution to the applicable Production Coordination Division’s testers.
  • Testers will be assigned a selection of test cases from the “999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet.
  • Defects are to be recorded as previously outlined in the Defect Tracking section in this document.

Performance Testing

Performance testing is required to ensure that acceptable timeframes exist within the application for pages to load and message responses to end-users. This type of testing will also ensure efficient processing of daily maintenance jobs.

  • Testing will include a selection of test cases obtained from the Usability Checklist tab in the

“999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet.

  • Defects are to be recorded as previously outlined in the Defect Tracking section in this document.

Web Standards Testing

The Web Standards Testing will review and evaluate items contained on OCIO Website Usability Checklist including:; architecture and navigation; layout and design; content and readability; forms and interaction; Website Accessibility Guidelines and Web Application Guidelines.This type of test also takes into consideration the operating system platforms used, the type of network connection, internet service provider type, and browser used (including version) ensuring NWC Software package configuration meets all NLWPD specification requirements.

  • Web Standards Testing will be completed by OCIO Web Standards Prime, Necie Mouland and NLWPDTester.
  • Testing will include a selection of test cases obtained from the Usability Checklist tab in the

“999999 - WPCS Project Test Cases v.1.0.xls” spreadsheet.

  • Defects are to be recorded as previously outlined in the Defect Tracking section in this document.

Regression Testing

The objective of Regression Testing is to ensure subsequent changes to the NWC Web solution, necessary to correct defects identified in earlier stages of testing are fixed and did not introduce new defects to the application.

Every fixed defect must be retested in the applicable new release, as per the release notes.