What I have researched so far - 1
Major Topics (my outline from his work)
Software Testing
Differentiating between OO Testing and Traditional Testing
Object-Oriented Testing
Miscellaneous Stuff
======
Software Testing
Khatri et al
· Objective of software testing: find as many errors as possible with minimum effort & cost; successful test should show bugs rather than that a program works
o 40-50% of costs testing
Johnson
“Testing is the process of executing a program with the intent to yield measurable errors. It requires that there be an oracle to determine whether or not the program has functioned as required, with comparison of performance against a defined specification.”
- M. Smith & D. Robson “A framework for Testing Object Oriented Programming” Journal of Object-Oriented Programming, June 1994, p. 45-53
· Testing is all about minimizing effort, maximizing findings
Gu et al
· Effective test case – involves both the changing class state and a sequence of operations
· Dynamic testing – process of running a program with the purpose to get measurable errors
· Software Testing components:
o Test case design (data selection)
o Test case execution
o Comparison of the result of the execution with expected ones (test oracle)
· Software fault is triggered only by certain input values in a certain operational environment (for any practical program input space is large – if not infinite) -> exhaustive testing is infeasible
o Need to select proper testing data (expose maximum number of faults) is of utmost importance
· Test Data Selection Techniques
o Program-based (white-box) – select test data based on code
o Specification-based (black-box) – equivalence partitioning, boundary value analysis, cause-effect graphing
Differentiation between Object Oriented Testing and Traditional Testing
· Traditional testing involves testing inputs and seeing what the output is.
· For object-oriented systems, the process is not always the same. Sometimes you can get the same output, but the state of a class (i.e. its fields) is in an incorrect state. This means that not only does the output of methods need to be tested, but a class’s state needs to be correct as well in order to make sure that future processes are correct.
Khatri et al
“During this comparative analytical study we have found that object oriented systems may reduce some kinds of errors however they increase the chances of others.”
· An encapsulated class is usually the focus of unit testing for object-oriented testing
o More complicated than traditional testing, but more effective in the overall system than procedural unit tests
· Most interactions occur through complex state interactions through inheritance and polymorphism
· Developers need to put more emphasis on integration testing than on unit testing (testing methods)
Johnson
· Standard testing methodologies may not be useful
o Process of testing OO programs is more difficult because programs are not executed sequentially as in a traditional approach
o OO programming can be combined in many orders and testing becomes the search for the order of routines which cause errors
o State-based manner of OOP is that an object’s state is not only defined by its internal values, but also in its association with other objects
o Object-Oriented testing is iterative and requires regression tests between iterations
§ Must test class again if a method changes in a superclass, affecting those that inherit from it
§ Testing first requires classes to be “fresh”, that is, that the classes are proven to be correct themselves before interactions can be tested between them
· Traditional – manner in which the components and interfaces, and the relations between them, are defined structurally rather than behaviorally
· The concept of a central program does not exist in an object oriented system, there is not a clearly defined structure to use Traditional techniques for integration testing
Gu et al
· Traditionally
o Software program viewed as implementation of a function mapping input domain to output range
o Input -> process -> output
§ Execution path is decided by input
· Object-Oriented Programs
o Centers on objects as instances of classes
o HOWEVER, no input-process-output correspondence (can’t talk about input to/output from a class necessarily)
§ There is not and should not be a predetermined flow control from class to class. The classes’ methods can be used in any order. This means that traditional methods are ineffective.
Object Oriented Testing
· Used primarily when objects are heavily interconnected and run in no predetermined order. If an object-oriented language program is still written like a traditional, method-driven program then traditional testing methods can be used.
· Many of the features of object-oriented programming which tie into its ease of use - reusability, data protecting, and dynamic use – also lead to problems in testing
· Inheritance leads to redundancy in testing, i.e. if methods are changed in the superclass, you must make sure that method is working in the superclass and all subclasses
· Object-Oriented testing is also used when classes are dependent on one another, which is usually the case. Traditional testing cannot be used because the flow of OOP is ever changing depending on the state of each class.
Khatri et al
· Features of object-oriented systems which cause issues with testing:
o Encapsulation – difficult to understand object interactions and prepare test cases to test interactions
o Inheritance – does not guarantee that a method that is tested to be “correct” in the context of superclass will work in subclass
o Polymorphism – an attribute of an object may refer to more than one type of data, and an operation may have more than one implementation which results in lack of controllability
o Coupling – the dependency between modules; undesirable because it prevents changes of components independently of the whole (need to understand the whole coupling set)
o Cohesion – if methods serving a given class contribute to a single, well-defined class; contributes to reliability, reusability, and understandability. High cohesion leads to low coupling
o Dynamic binding – introduces decidability concerns because exact data type and implementation cannot be determined statically
o Abstract classes – cannot be instantiated, which affects base testing
Johnson
· Testing Levels for OO
- Unit Testing – Testing class methods/routines independently of other methods
- Similar to unit tests for traditional testing
- Class Testing – testing the class methods as they relate to and interact with one another
- New to object oriented programming (integration steps)
- Cluster Testing – extending class testing to verify that a group (cluster) of cooperating classes interact correctly
- New to object-oriented programming (integration)
- System Testing – Testing the system using scenarios of normal customer usage in a configuration that resembles the customer’s environment as accurately as possible
a. Similar to system tests for traditional testing
· Errors can hide within different hierarchal levels and defects can be closely related to other object’s states rather than just a specific series of steps.
· Every system should include at least a class-based testing effort and a system test that modules how the system will be used in the end, along with automated testing tools
Gu et al
· A class is an implementation of an abstract data type (APT)
o ADT specifies intended behavior
· A class may behave abnormally by reaching an invalid state or by malfunctions at the method-level
· Method types – affect class state
o Constructors – change state; malfunctions by putting class into invalid or inappropriate state
o Transformers – “
o Observers – only report values, malfunctions by changing state or returning incorrect values
· CLASS IS SMALLEST UNIT TO TEST – should be focal point of testing because of high potential and frequency of it
· Test data is 2-dimensional rather than 1-dimensional:
o Test data should be selected based on both the state space of a class and the operation space of the class (i.e. all possible sequences of operations and all possible states of C)
§ This test data may be infinite so a good test suite (effective) contains: properly selected sequences of operations and properly selected class states. Exhaustive testing is often not possible, so determining which sequences and states to test is important.
Miscellaneous Stuff
· Most authors seem to start off by saying how little work has been done in testing object oriented systems in comparison to object oriented analysis and design
· There are some methods which I have found articles about, which I have not read in depth yet: Flow-graph-based, Graph-Based Approach, ASTOOT (Gu et al has some comparisons on these, but I have not had a chance to read them yet), COTT, etc.
Sources
1) Mrs. Sujata, Khatri, Chhillar Dr. R. S., and Sangwan Mrs. Arti. "Analysis Of Factors Affecting Testing In Object Oriented Systems." International Journal On Computer Science And Engineering 3 (2011): 1191. Directory of Open Access Journals. Web. 5 Sept. 2012.
2) Morris S. Johnson, Jr.. 1996. A survey of testing techniques for object-oriented systems. In Proceedings of the 1996 conference of the Centre for Advanced Studies on Collaborative research (CASCON '96), Michael Bauer, Karen Bennet, Morven Gentleman, Howard Johnson, Kelly Lyons, and Jacob Slonim (Eds.). IBM Press 17-.
3) Dechang Gu, Yin Zhong, and Sarwar Ali. 1994. On testing of classes in object-oriented programs. In Proceedings of the 1994 conference of the Centre for Advanced Studies on Collaborative research (CASCON '94), John Botsford, Ann Gawman, Morven Gentleman, Evelyn Kidd, Kelly Lyons, Jacob Slonim, and Howard Johnson (Eds.). IBM Press 22-.