How To Automate Mobile Application Testing
The average attention span of a human being is less than 9 seconds! Have you ever wondered why? Most mobile applications that have sprung up in the recent past require you to scroll down a feed where the brain is fine-tuned to not look at something beyond a few seconds. This has led to significant reduction in the attention span of an average user.
With the plethora of mobile applications that are available in the app store, comes the great responsibility for mobile app testing and scaling the regression testing efforts. That’s where the mobile testing tools come in handy.
I have been one of the early adopters of the mobile testing frameworks and in my experience, focusing on automation, form factors and performance testing have played an equally important role to scale and increase impact in the mobile world. In this article, I'll share the most effective techniques for automating your mobile application tests.
Mobile Application Testing: Components To Consider
Testing on any specific platform has its own challenges but with mobile, you have these additional parameters that you need to take into account. Whether you do manual testing or test automation, in addition to functional testing, your test cases need to focus on:
Connectivity
It’s important to address what happens if a user is using their mobile device in flight mode or offline, what happens if bandwidth fluctuates, etc, to ensure the app works.
Location
Especially for mobile applications that depend on GPS or location-based services, testing has to be performed by simulating location using tools like MobileSpy, Location Spoofer, etc. otherwise it would be hard to test—say for example: If you want to simulate a test scenario for a ride-sharing mobile application, where someone is requesting a ride from NYC and there have been changes made to the backend microservices that are specific to finding the driver in that location. Without simulation, the tests wouldn’t give you accurate results.
Operating System
Especially with the number of android phones that we have in the market today, testing on all available OS versions is nearly impossible - then how do you prioritize? With iOS, you have additional challenges that Apple puts forth related to Xcode versions associated with the OS version changes.
These days companies resort to using 3rd party providers like BrowserStack for providing simulators/emulators for various OS, device testing, and cross browser testing.
The UI icons are different based on the operating systems platforms: Windows, iOS, and Android and when it comes to test automation, most of us end up using simulators—the Android ones take much longer to load whereas the iOS devices are faster.
The different mobile key combinations to use based on different operating system types for taking screenshots for debugging issues is a critical input for anyone testing mobile applications.
Form Factor
While developing mobile applications, there are 5 major form factors that need to be considered for the different OS versions in addition to portrait or landscape mode on each of these device types:
| Device | Screen size |
| Small mobile devices | 3.5 inches or smaller |
| Medium mobile devices | 3.5 to 5 inches |
| Tablets | 5 to 7 inches |
| Small tablets | 7 to 8.5 inches |
| Full size tablets | 8.5 inches or larger |
How To Choose The Right Framework For Mobile Automation
Considering form factors while building mobile applications refers to ensuring responsiveness when the same app is opened across different mobile devices and OS types. When the app usage spans across the world and if companies want to increase their user base, they will need to make this decision either early on in the development process to resort to using React Native frameworks over native android or iOS development.
If not, companies will end up spending time and effort later for refactoring and redesigning the apps to switch to React Native frameworks. This directly impacts the number of teams and maintenance overhead as React Native frameworks support development of both iOS and Android applications.
When mobile applications are developed using React Native frameworks, it also reduces the burden on test automation, as you only need to maintain a single automation testing framework for both iOS and android.
There are different ways to develop mobile applications. Some companies resort to using native programming languages like Java and Objective C for Android and iOS applications respectively. The advantage of using native languages helps them provide rich functionality to their applications.
On the other hand, the element ids will be different on both platforms—which means we will need to have separate native test frameworks to test the same like Espresso on Android, XCUITest on iOS, etc.
React Native is another commonly used programming language for developing responsive mobile applications that was open-sourced by Facebook. The objective was to unify mobile development efforts on both the platforms so there can be a single team and so the element ids on both the apps will be the same—which only needs a single test framework like Appium to help test the mobile applications.
When it comes to automating your mobile application tests, choosing the right QA automation tools can make a significant difference in your testing efficiency.
How To Automate iOS Mobile Application Testing
While testing manually there are a bunch of criteria that need to be adhered to: most of the companies that develop iOS apps clearly specify which operating systems and sdk versions are supported. This helps to limit your testing efforts.
In order to scale software testing efforts, we need to invest in automation. There are a lot of popular frameworks that support iOS automated mobile app testing, out of which let’s limit the scope of our discussion to the following 2 automation tools:
- XCUITest
- Appium
| XCUITest | Appium |
| Native apps framework that can reside within the iOS source codebase. | Silo’d test framework that is built outside of the source codebase. |
| Written in Objective C or Swift | Written in Java, Python, etc. |
| Explicit waits that apply a condition. | Implicit waits which involve usage of sleep(), which makes tests flaky. |
| Mocking backend API responses using open source libraries like Mockingjay | Supports mock server usage. |
| Since this is a native apps framework, it can be used to test iOS applications only. | Since the tests reside outside of the source codebase and can be written in languages like Java, python, etc it can be used to test cross-platform applications like Windows, iOS, and Android app so long as the element ids are the same on both platforms. |
I have leveraged mock servers to help mock out the backend calls and test the UI exclusively as follows by creating a base class:
Now that the base class is ready, I add the actual test by leveraging the page object design pattern as follows:
How To Automate Android Mobile Application Testing
There are over 2 billion active android devices in the world. Testing your android application across different device and operating system types is nearly impossible! So, companies explicitly call out the versions they support. This ensures better user experience as well.
Android automation frameworks will definitely help accelerate testing processes like Robotium, Selendroid, etc and for the scope of our discussion, let’s limit to the automation tools: Espresso and Appium:
| Espresso | Appium |
| Native framework that can reside within the Android source codebase. | Silo’d test framework that is built outside of the source codebase. |
| Written in Java and Kotlin | Written in Java, Python, etc. |
| Explicit waits, asynchronous operations supported by idling resources capability | Implicit waits which involve usage of sleep(), which makes tests flaky. |
| Mocking backend responses using open source libraries like Mockito and Retrofit builder | Supports mock server usage. |
| Since this is a native framework, it can be used to test android applications only. | Since the tests reside outside of the source codebase and can be written in languages like Java, python, etc it can be used to test cross platform applications on both iOS and Android applications so long as the element ids are the same on both platforms. |
One of the advantages of using native app frameworks is to use mock servers to mock out the backend api calls and thereby test the UI exclusively. Here’s an example code snippet on how I have made it work:
Depending on how your microservice architecture is developed, you either grab the response from the network tab of your browser or from the developers directly and reference them as follows within your test:
As you can see in the above example, HomeTabPage is being called within the test, instead of having to explicitly assert each element—I have leveraged page object pattern in the framework as follows:
Mobile App Testing Tools
There are various testing tools in the industry and the choice depends on multiple factors:
- Team skillset
- Purpose of automation
- Audience for automation: Product Managers/Sales/Engineers/Testers
- Compatibility of test framework and native source code
- Number of hybrid apps
Different tools that are popularly used in the industry include:
- Selendroid
- Robotium
- Appium
- Testdroid
- UiAutomator
Don't feel like doing your own in-house testing? You can also employ mobile app testing services to do it for you.
Mobile Application Performance Testing
While testing mobile applications, apart from focusing on automation to scale testing efforts, we should also prioritize performance testing as it plays a crucial role in determining if a user is going to continue to use our application.
If the screen takes more than 2 seconds to load, people become restless—goes back to our attention span research, and hence to retain users, we need to invest in performance testing.
While testing app performance, we will need to identify the KPIs and benchmark against them:
- Maximum response time
- Average response time
- Average throughput
- Peak number of active users for each OS and device
Using a mobile app testing tool can simplify performance testing. For example, Apptim can track end-user performance metrics—it provides comprehensive performance reports for all key performance parameters.
For monitoring the mobile application performance, we can use Firebase Performance monitoring which provides control over performance data by sharing insights into how your app performs with a breakdown of trace and network data into dimensions like app version, country, device, and network type.
There are other tools like Appium Studio, Sauce Labs, Testdroid, etc which provide similar performance monitoring capabilities.
Hope this article gave you a primer into mobile application testing and the critical factors associated with it!
For more best practices and tips to shape your development process, subscribe to The QA Lead newsletter.
Related Read:
Check This Out:
- WHAT IS TESTGEAR? OVERVIEW & TOUR OF FEATURES
- SERVER MONITORING METRICS TO TRACK FOR SYSTEM HEALTH AND PERFORMANCE
Related List of Tools:
