Topic:Use of device farm for mobile device testing

Introduction:

In digital era of testing we are facing many challenges with delivery to customer with seamless quality. With emergences of different mobile devices and platform we are facing many compatibility issues. However, this has emerged as great challenge for testers to keep pace with every release. The aim of this paper is to provide a solution for faster and reliable delivery of product while using device farm. We integrate our systems with device farm setup available through open source channel or service providers. We can book and access these devices through our systems and run our tests manual and automation. Going forward we can create our own device farm with mobile device available.

Problem statement:

Procurement of every mobile device is not a feasible and faster solution. We often require many approvals for attainment. Maintenance and managing devices is also a big overhead since the device market is changing rapidly. Its costly to purchase a device every time a user reports a device specific issue. Certain test scenarios many times due to devices non-availability are not executed and lead to user reported bugs at forums hampering ratings of application. Moreover, big organizations need to purchase devices for each of their geographical located testing centers which is a redundant cost.

Proposed Solution:

We can use services of device farm available by various service providers or our self-integrate and create a device farm with available devices which can be used across organization location. We can run automated test as well on many devices simultaneously thus saving time and helping reaching deadlines.

Implementation using OpenSTF:

  1. Installing OpenSTF
  2. Installing NVM but feel free to install node directly from the repository
  3. Android SDK
  4. Install Libsodium
  5. Install Zeromq
  6. Finally install open stf
  7. Install and configure Rethinkdb Service
  8. Install the Jenkins Plugin
  9. install the plugin
  10. Configure the Jenkins Plugin

3.2.1 Go to Jenkins -> Configure

3.2.2 Go to the Open STF Configuration

3.2.3 Enter in the Open STF url

3.2.4 Enter in the Open STF token (you can generate one in the open stf ui under Settings -> Keys) If everything went well you should see no errors, go ahead and save the configuration

3.2.5 Go to an existing job

3.2.6. You should now see “Use a STF device during build” under “Build Environment”

  1. Open configured URL from your connecting device.
  2. Select devices from dashboard.
  3. Drag and drop APK files.
  4. User should be able to access application and run tests.

Brief overview of how communication occurs:

  1. Browser sends a WebSocket message
  2. One of thewebsocketunits receives the message and translates it to Protocol Buffers
  3. Thewebsocketunit sends the message via PUSH to the app sidetriproxy's PULL
  4. The app sidetriproxytakes the message as-is and randomly forwards it via DEALER to one of theprocessorDEALERs
  5. Theprocessordoesn't do anything yet (although it could if necessary), it simply forwards the message via DEALER to the device sidetriproxy's DEALER
  6. Thetriproxytakes the message as-is and uses PUB to publish the message the device's channel (which is included in the message)
  7. Thedevunit that is listening on that channel receives the message via its SUB.
  8. If the device is in a group, it leaves that group (and clears its in-memory state)
  9. Thedevsends a GroupLeave message and sends it via PUSH to the device sidetriproxy
  10. Thetriproxytakes the message as-is and randomly forwards it to one of theprocessor's DEALEREs
  11. Theprocessorreceives the message, reads it, and modifies the database to remove the owner from that device
  12. Theprocessorforwards the message via DEALER to the app sidetriproxy's DEALER
  13. Thetriproxyreceives the message and PUBs it to all thewebsocketunits
  14. All thewebsocketunits receive information that the device no longer has an owner
  15. Thewebsocketunits emit a WebSocket message to all the connected browsers
  16. The browsers receive the message, update their internal state in JavaScript and re-render the UI if necessary

Benefits:

•Faster turnaround time as there is no need to get approvals for Devices.

•No need to worry about the device shelf life.

• Since device is booked and used on hourly basis its faster and cheaper than purchasing different variety of devices.

•No need to buy separate devices for separate geographical locations

•Automated test cases can be run simultaneously through Appium on connected devices (Both android and IOS)