Karumanchi Sravanthi

Karumanchi Sravanthi

CS 458

Mid Term

Due Date: 11/01/01

Question # 1

Certification is the endorsement of the correctness of a system by some outside source using some predetermined standard of performance.

For example, the DOD certifies Ada compilers against over 2000 functional specifications. Do you consider such certification a function test, a performance test, an acceptance test or an installation test? Explain why or why not for each of the four types of test.

Answer # 1

Certification is the business of evaluating a product against a standard. A product that meets certain requirements based on the standard may be awarded a certificate of conformance. Once verification, inspection and usage testing have been completed the increment is certified as ready for integration. Certification is one of the ways in which we can provide quality assurance.

Certification is thus a function test and a performance test.

Function Test: Function testing is also called as black box testing. This is because no knowledge of the internal logic of the system is used to develop the test cases. Function test is performed to ensure that the product functions the way it is supposed to according to design specifications and documentation. It aims towards knowing the specified function that the product has been assigned to perform. This test is in general applied to all applications. Certification is function testing as it involves testing done using a standard tool as well as tests are done for testing functionality, stress and race conditions. Thus certification has to check if the product functions the way it is supposed to according to design specifications and documentation, in this case with respect to a standard. This gives an assurance to the buyers and end users as this check reveals that the system is functioning as desired. This promotes confidence in the technology, which is the main aim for certification.

Performance Testing: This is a term often used interchangeably with stress and load testing. Ideally performance testing is defined in requirement documentation or quality assurance or test plans. Performance testing is designed to test run time performance of software within the context of an integrated system. Performance testing occurs throughout all steps in the testing process. Even at the unit level, the performance of an individual module may be assessed as white-box tests are conducted. However, it is not until all system elements are fully integrated that the true performance of the system can be ascertained. Performance tests are often coupled with stress testing and often require both hardware and software instrumentation. External instrumentation can monitor execution intervals, log events as they occur and sample machine states on regular basis. By instrumenting a system, a tester can uncover situations that lead to degradation and possible system failure. Thus certification has to do performance testing as this will help to detect if there is any degradation in the system. The main objective of performance testing is to provide quality assurance. By using performance test we can find the true performance of the system, which is the main requirement when a certification agency certifies. It looks at the true performance of the system. Performance metrics of the system have to be performed as this tells us if the system behaves properly and performs the way it is supposed to.

Installation Testing: The purpose of installation testing is to validate that a system can be successfully installed on the target platform that once installed, the system is usable, and that the guidelines used to direct installation are accurate. Testing during this phase must ensure that correct versions of the programs are placed into production, that data if changed or added is correct and that all involved parties know their new duties and can perform correctly. This is not certification as a particular version of the product is certified and if there are any changes made to this product they need to be certified again. Thus certification has nothing to do with installation testing.

Acceptance Testing: When custom software is built for one customer a series of acceptance tests are conducted to enable the customer to validate all requirements. This is conducted by an end user rather than a system developer. An application test can range from an informal test drive to a planned and systematically executed series of tests. If software is developed as a product to be used by many customers, it is impractical to perform formal acceptance tests with each one. Most software product builders use a process called alpa and beta testing to uncover errors. Acceptance testing views the interrelationship of all aspects of the system in its entirety. As well as verifying the satisfaction of the requirements, it is a vehicle to ensure that training, on-line help, user guides, etc. corresponds with the program operations being delivered. Since certification agency may not certify the documentation and other things required for software like support it cannot be acceptance test. Certification is not acceptance test, as it is not looked at from each user’s point of view. Certification is to check a product with respect to the standard and then be used my many users so it cannot be an acceptance test.

Question # 2

A payroll system is designed using a record for each employee. Each record is updated weekly with the hours worked by that employee for that week. Every two weeks, a summary report of hours worked for the year to date is produced for each employee. Monthly, paychecks are issued for all employees. What tests should be administered to ensure the system functions properly?

Answer # 2

The tests that are required for the above system to function properly are:

Function Testing: Function Testing is performed to ensure that the product functions the way it is supposed to according to design specifications and documentation. It aims towards knowing the specified function that the product has been assigned to perform. Tests are conducted that demonstrate if each function is fully operational while at the same time searching for errors in each function.

Function testing is very essential to our application as well as in general applied to all applications. Specific functions are tested to see if they function the way they are supposed to and also try and eliminate those faults that are missed when compiled. Functions especially like calculating the amount the employee gets, the amount of hours worked etc have to be checked completely so that no employee is over paid or under paid.

Stress testing: Stress tests are designed to confront programs with abnormal situations. In essence, the tester who performs the stress testing asks: “ how high can we crank this up before it fails?” Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. Essentially, the tester attempts to break the system.

Stress testing may or may not be required for the payload system. The number of employees is going to remain more or less constant so there should be no heavy load on the system. But it could be used on our system as the payroll system is very critical part and it should be checked regularly to see that some kind of overload on the system is not degrading the system. For example if new employees are taken into the company the load may increase. The payroll system is important as any small wrong operation could degrade the system and may not work fine so stress testing could be performed on the payroll system.

Volume testing: Volume Testing is a subset of stress testing that deals specifically with determining how many transactions or database accesses an application can handle during a defined period of time. Volume testing refers to the creation of specific types of test data in order to test predetermined system limits to verify how the system functions when those limits are reached or exceeded. The objective is to verify that the system can perform properly when internal program or system limitations have been exceeded.

Volume testing could be necessary if we assume that the database used is very huge that is the count of employees is huge. But since the count, how many users are allowed to send queries simultaneously etc are not mentioned volume testing is not necessary.

But if i assume the database is huge and number of users sending queries is also huge then volume testing is necessary.

Configuration testing: Configuration testing, also known as hardware compatibility testing, if it's not done correctly, the product may not work on a range of machines. Skimping on configuration testing can increase the number of technical support calls, destroy word-of-mouth advertising, and do irreparable damage to the company’s customer base.

Configuration testing may or may not be necessary for the payroll system depending upon whether the system is used on different machines or not. But since it is not mentioned that it is run on different machines this test may be redundant.

Compatibility Testing: Compatibility testing ensures that the application functions correctly on multiple browsers and system configurations. Compatibility testing may be performed in a test lab that contains a variety of platforms or may be performed by beta testers. Compatibility testing also involves checking the backward compatibility of the system to its previous versions or any other system interacting with the new system.

If the payroll system were interfacing with some other out-sourced software or ERP software, then compatibility testing would be necessary. Since that information is not given this test is redundant in the system.

Regression Testing: Each time a new module is added as a part of integration testing, the software changes. New data flow paths are established, new I/O may occur, and new control logic is invoked. These changes may cause problems with functions that previously worked flawlessly. In the context of an integration test strategy, regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects.

Regression testing is necessary whenever any changes done to the system e.g. when pay scales are changed for the system to function properly. Any changes in the system should not generate undue effects on other parts of the system, which are not changed.

Security Testing: Any computer-based system that manages sensitive information or causes actions that can improperly harm (or benefit) individuals is a target for improper or illegal penetration. Security testing attempts to verify that protection mechanisms built into a system will, in fact, protect it from improper penetration. This testing technique is imperative to help prevent and test against unlawful entries and illegal data manipulation. e.g. some outsider trying to access employee records / company information or some employee trying to access records from outside the company or manipulate his records. To help achieve secure access, all employees need to have their own username and password to log in. Delegating authority to update records. Delegating authority to read-only access to the system.

This is very necessary for the payroll system. The information handled by the payroll system is of high value to the organization and tampering with the system can create havoc in the organization. Given enough time and resources, good security testing should ultimately penetrate the system. The role of the system designer is to make penetration cost more than the value of the information that will be obtained.

Timing Testing: Timing testing becomes critical in real-time systems or systems which are highly interactive in nature. Timing testing aims at evaluating response times and time to perform a function.

This is not necessary for the payroll system, since it is not a time-critical system.

Environmental testing: This kind of testing tolerances for heat, humidity, motion, portability. Since no clarifications are provided regarding the location of the system and its need for portability, assuming that our system will be in a controlled environment and there is no issue of physical portability, this kind of testing is not really necessary.

Quality Testing: Quality testing involves reliability, maintainability, and availability of the system. This type of testing becomes redundant after all the previous functional and performance tests are conducted. It would amount to the same thing.

Recovery Testing: Recovery testing is a system that forces the software to fail in a variety of ways and verifies that recovery is properly performed. If recovery is automatic (performed by the system itself), re-initialization, check pointing mechanisms, data recovery and restart are evaluated for correctness. If recovery requires human intervention, the mean-time-to repair (MTTR) is evaluated to determine whether it is within acceptable limits.

For our system, it is a must to perform recovery testing. We cannot afford to have erroneous wages e.g. on recovering; a system faultily interchanges the pay scales of different job positions. A lot more information needs to be given regarding the rules the system must use to recover. This is required for the Payroll system. The information handled by the system is very critical and it has to be recovered in case of system failures. It has to be checked if the system returns to a stable state after experiencing an erroneous or an unstable state. Information is required specifying what the ways or rules for system recovery.

Maintenance Testing: This testing technique is divided into three categories viz. Corrective, Adaptive, and Perfective. Corrective testing includes the diagnosis and correction of errors, which were not found prior to software release. Adaptive testing modifies software so it properly interfaces with its environment as that environment changes. This is already taken care of in Compatibility testing. Hence Adaptive testing can be avoided.

Perfective testing adds new capabilities, modifies existing functions and incorporates other general enhancements, which would eventually result in a more stable, fault-tolerant and user-friendly system.

Hence this technique is essential as changes for the better are always welcome.

Documentation Testing: This is one of the most important testing techniques for any system. The purpose of documentation testing is to ensure that applications documentation accurately reflects the most current state of the application. Hence we should not neglect this form of testing for our system.

Human Factors Testing: A human factors testing incorporates testing the interface of the system with actual system users. The test cases will reveal the understandability of the labels, terms, remarks or information messages displayed on the user-interface. The test cases aim towards determining if software interface is satisfactory to an end-user or customer.

The payroll system is an in-house system and not a product to be distributed. This makes the technique not an essential one for the payroll system. Again this will depend on the following information:

• Has the user been previously exposed to similar systems
• How extensive is the user interaction demanded by the system

This kind of testing holds importance in conventional commercial markets and when the system is web based or if the final system is to be used by customers not part of our company, since this deals with checking for ease of use by user and the user interaction with the system.

Installation Testing: Problems installing or uninstalling an application can cause frustration, even among the most patient people. The test cases make sure that the application is free of user installation pitfalls. The test cases also aim towards testing of full, partial, or upgrade install/uninstall processes. The payroll system is an in-house system. It gets updated time-to-time when the payment policies change or the business strategies change. Newer versions of the system have to be installed and older versions have to be un-installed. This scenario makes Installation testing a required testing procedure.

Acceptance Testing: This is a final testing based on specifications of the end-user or customer, or based on use by end-users/customers over a period of time. This is a required test to make sure the system is working correctly with respect to each employee.

Requirements Testing:Though this method has not been mentioned in the list of tests, it is recommended that requirements testing be performed on the system. This type of testing verifies that the system can perform its functions correctly and that the correctness is sustained over a continuous period of time.

Error-handling Testing:It is also recommended for our system. This kind of testing determines the ability of the application system to properly process incorrect transactions. It implies that all reasonably expected error conditions are recognizable by the application system. For example, the system should be able to recognize discrepancies in hours worked per week etc.

Load Testing: This is to test applications on heavy load. This is a required test to check to see if the system can handle heavy loads. This is similar to the volume testing.