Rapid fire performance testing of 250 websites

Abstract

Leading Auto spare parts supplier in US was undergoing an organizational IT upgrade and decided to take advantage of Cloud

So it was decided to move a set of 250 different Static IT applications over to Cloud environment from current on-premise facilities.

The requirement was to performance test all the 250 websites, in cost effective way, which majorly contained static contents, in 6 months to have optimum time to market.

This was not possible with standard performance testing approach and hence out of the box arrangement was thought of creating custom performance testing tool. Tool created was using the MS Excel and industrial leader open-source performance testing tool - JMeter, to reduce Performance testing time.

Business Requirements & Challenges:

•Exhaustive performance testing of 250 websites in 6 months

Performance testing needs to cover all the static URLs along with their respective stated child urls in the 6 months timeframe to have optimum time to market.

•Multiple cycles of Performance testing

Based on the outcomes of the testing, there can be multiple trial & error method for single websites for testing and the results should be in comparable form.

•Non-feasibility with normal performance testing tool cycle

Creating & maintaining individual scripts for performance testing was not feasible option as the time & cost constraints won’t be feasible. Also maintenance might become issue.

•Cost Effective performance testing

If licensed performance testing tools are used then that would significantly increase the overall cost of the project.

•Extension to non-technical users

Tool should be extended to the non-technical users, in case it needs to be tested under specific conditions / cross verification

•Lower maintenance efforts

Tool should be easy to maintain and re-usable in nature.

Approach

Considering short duration & cost constraints, the traditional performance testing cycle/tool will not be able to achieve the required results.

Creation of Tool, which would sit on top of Industrial Leader Open source Performance testing tool – Jmeter with following features was conceptualized:

Tool life Cycle:

Creates IE handle of the base URL and extracts all the child URLs and domains available

URLs and domain can be configured to be executed

Validates all the URLs by executing all the URLs sequentially.

The Page title of all the URLs are captured and can be used to validate the URL functionally.

User can configure the load settings like:

Number of Users

Proxy settings

Wait Time

Test Duration

Generates a Jmeter script based on User configuration

Jmeter script can be directly executed from Utility and live results can be monitored

Benefits:

•Automatic & Dynamic script generation of the application under test.

•Usage of Open source tool to reduce license cost.

•Easy Configuration along with execution and result viewing through Excel file

•Option of URLs to be configured for execution on the basis of URL Domain / specific URL.

•Feature to validate all the URL Navigations before actual load test

•Capturing Single user response times with all User content objects.

•Storage of multiple version of single test to be stored for comparison

•Error details for analyzing load test errors

Tool Snapshots:

Step1: Extraction

References & Appendix

Author Biography

Dharmit Shah is the Performance Test Manager at LTI

Has nearly 12 years of IT experience and has extensively worked on different areas of Software Testing including performance testing, performance engineering, software test automation, test consultancy, etc.

He has authored multiple papers & participated in various conferences. Whitepaper on batch performance test & Auto Framework is listed on various sites

After completing his Computer Engineering from Gujarat University he has worked with multiple leading IT Companies.

He loves playing cricket & watching movies.

THANK YOU!