Software Enterprise Practice: Software Testing Spring 2010
Shared Lab, CST316 and CST416 April 14, 2010
Objective: Understand functional testing using an automated tool
For this lab you should work with a single partner from your team. CST416 students should work with a CST316 student wherever possible.
Note the lab instructions ask you to use JUnit4, but the tool we are using, Selenium, actually lets you work with scripts generated in a number of languages including Ruby and C#. I don’t care what language you actually use in those steps, but keep in mind there is no inherent advantage in web testing in having the test script language be the same as the language the web application is actually implemented in.
Submit a zipfile that is named <asurite1>.<asurite2>.zip which contains 2 zipfiles from parts 1 and 2.
Part 1: Web Testing Using Selenium
Testing web applications usually involves scripting or click-replay functionality, or both. Selenium is a freely available functional testing tool for web, console, and traditional desktop applications that supports both click-and-replay and scripting (in a variety of languages). For this activity you will use Selenium to execute, create, and log test cases.
1. Download the Selenium-RC and Selenium-IDE from the Selenium site (http://seleniumhq.org/)
a. RC is a tool that proxies your test executions. Expand the zipfile into an empty directory. Note it will create 8 directories when expanded.
b. IDE is a Firefox plugin that does click-and-replay. Install in your Firefox.
2. Download the Stest zipfile from the class website. This contains an Eclipse project with a single selenium test file and a generated Java file.
a. Create a new Java project rooted at the directory where you expand this zipfile.
b. You probably will need to go into the Build Path settings for the project.
i. Select “Add External Jar” to add the selenium-java-client-driver.jar from the selenium-java-client-driver-1.0.1 subdirectory under where you expanded the Selenium-RC zipfile in step 1a.
ii. Select “Add Libraries”, select JUnit, and choose option JUnit4 to enable JUnit4.
3. Next, we’ll run a sample script as a JUnit test. (Note you should capture logs for steps a and c)
a. Startup up the Selenium RC server in a console window. In a shell window, cd to the selenium-server-1.0.3 directory and type “java –jar selenium-server.jar”. You should see some startup messages and then the program sits and waits (if you get security warnings from Windows, do what you need to do to allow port 4444 traffic. If that doesn’t work, throw your winders machine in the trash and go get a Mac ;)).
b. In your Eclipse/RTC, open the file Stest.java. Near the top you will see a call to “setUp("http://localhost:8080/CoffeeMaker/index.jsp", "*chrome");” please change localhost:8080 to the hostname:port given to your team.
c. Now right click on the Stest class and select “Run As…JUnit test” from the context menu. You need to have JUnit4 added as a Library in your Eclipse Build Path above in step 2.b.ii.
i. You should see Eclipse change to its JUnit context
ii. After a minute, Firefox should start up automagically, and start scrolling through screens.
iii. When the test is done, Firefox will close on its own, and your JUnit pane in Eclipse should show green for 1/1 successful tests.
iv. Inspect the JUnit test file Stest.java. For help in understanding the calls and what is going on see the Selenium docs on page http://seleniumhq.org/docs/05_selenium_rc.html. Note as well that Selenium can generate and run tests scripted in a variety of languages.
v. Note: after running each test you should navigate to the CoffeeMaker application’s “Delete a Recipe” page and delete all recipes you see in the list. Do this after any time you run a test script during this lab.
4. Now, we’ll use the Selenium-IDE to run a test case.
a. Bring up your Firefox browser. Under Tools, open the SeleniumIDE.
b. In the plugin, select File à Open and navigate to the file name Stest.html in the root of your Eclipse project directory. You should see script language in the main pane of the SeleniumIDE.
c. Click the green arrow just above the main pane in the SeleniumIDE. You may want to move the slider toward “Slow” before doing so. You will see the browser rotate through user interactions as the test executes. Save the log info that appears in the lower pane.
5. Next, we’ll create a small script using Selenium’s click-and-replay facility
a. Select “File à New Test Case”. You should see a new test case titled “Untitled” in the left pane.
b. Select this test case and click the red record button in the upper left.
c. Click around through the CoffeeMaker application, exercising at least 3 functions from the main menu ending with a submit on page “Make Coffee”.
d. Click the red button again when done.
e. Highlight your new script and click the green button in the menubar (the one to the right). You should see your script execute in your Firefox browser. Did it pass? Save a log.
f. Right-click on the Untitled title in the left pane and rename your script to “StestLab”.
g. Use “File à Save Test Case As” to save your test case file. Name it StestLab.html.
h. Use “File à Export Test Case As…” to save as a Java/JUnit4 test script named StestLab.java.
i. Copy both StestLab.html and StestLab.java to your Eclipse project. Refactor the Java file so it resides in the web.stest package with the existing Stest.java file.
j. Run the StestLab.java file as a JUnit4 unit test. How did it do? Save the log output.
6. Finally, document the script as a test plan.
a. Translate the steps of the script (from the HTML file) to an ordered natural language list of user actions (things the user had to do and in what order).
b. Can you identify a goal for your script? Was there a specific thing you set out to verify when you started clicking around, or was it random?
c. What other procedures must the test team consider? Do they have to seed/remove data? Have a user account? Any other pre or post conditions that need to be specified?
7. Save all the files from this part (test script, StestLab.html/java, all logs) into a zipfile named part1.zip.
Part 2: Constructing Test Cases from Requirements
The requirements for the CoffeeMaker application are available in the file CoffeeMakerExample.html in the root directory of the downloaded Eclipse project. Note that the requirements are given as a combination of User Stories and Use Cases. You will use the Use Cases to define two test cases in Selenium.
1. Review use case 7, Purchase Beverage.
2. Devise a test plan for the Main Flow (ignore Alternative Flows for now). Document the test plan in a numbered list format like you did for Part 1, step 6.
3. Remember the topic of Equivalence Partitioning and Boundary Case Evaluation – do you need to record the script with more than one set of data to be certain the test goal is achieved?
4. Devise a test plan for a 2nd use case. You may choose one of UC2, UC3, UC4, or UC5. Do not choose UC1 or UC6. Repeat steps 2 and 3 for this use case.
5. Now, generate test scripts for both test plans using SeleniumIDE’s record functionality. Save the respective scripts as UC7Test.html and UCXTest.html (where X is the use case you selected).
6. Export the scripts as you did in Part 1 as JUnit test cases, run them, and save the output.
7. Open the UC7Test.java JUnit generated testfile, and review the Alternative Flows for UC7 in the requirements. One thing Selenium cannot record is conditional behavior like this in a single test case. Instead, you would have to use multiple test cases. However, you are able to manually edit and add calls to the Java code to encode these conditions in a single test case. Modify the Java code to incorporate the 2 alternative flows.
8. Finally, when running the tests, did you encounter any defects? Were they defects of “verification” (correctness) or “validation” (behavior meets requirements). Log any defects, and their exact conditions to a file defects.txt.
9. Save all files from this section – scripts, Java, test plans, and logs, to a file named part2.zip.
Part 3: Journal Entry
Answer me these questions 3. You know the drill.