Make Sure Testing Keeps up With Mobile App Evolution

By Jeffrey Hughes, Perforce

As financial mobile apps continue to evolve, testing these apps becomes increasingly challenging. For instance, the introduction of biometric user identification on a mobile app introduces additional aspects to test. This is why a comprehensive and targeted test strategy is essential, because inadequate testing can lead to performance issues, dissatisfied users, compliance problems, and even potential security vulnerabilities.

However, the sheer scale of testing requirements (let alone making sense of the results) can quickly become overwhelming, making it hard to know what to test or what is the impact on the business.

Today’s test environments need several fundamental building blocks that include: understanding the biggest challenges, what to test, and understanding the benefits of modern test techniques and tools available to development teams right now.

The toughest mobile test challenges

Let’s start with understanding some of the toughest mobile app testing challenges. Apart from the aforementioned biometric authentication, other challenges include image injection (in other words, using a camera for barcode scanning). Another example is testing for different network conditions, across 2G, 3G, 4G, and 5G, no cellular but WiFi access, or poor connectivity. If someone is using a banking application and moves from a lobby into an elevator, how well does the app handle that scenario?

Other test challenges that have increased in recent years include greater use of two-factor authentication, voice activated commands, and geolocation. These mobile app enhancements all bring extensive user experience benefits, but also add to the number of features that need to be tested.

What to test

To prevent testing from becoming overwhelming, it’s vital to build a mobile test strategy that is fit for today’s fast-changing app development environment. Clearly, security is of paramount importance, but thorough quality assurance as part of testing to ensure the best possible user experience is essential too. Increasingly, consumers expect perfection, and will not tolerate slow responses, inconsistencies and annoying bugs that impact performance.

While a customer is unlikely to walk away from a financial provider just because of the app’s quality, it could be a contributing factor, and at the very least, reduce overall customer satisfaction.

It is also vital to keep up to date on changes to security policies, data collection, and storage capabilities. For example, Google now requires developer teams to adhere to more strict privacy requirements. Another important requirement is to ensure accessibility of an app, across multiple aspects, such as text contrast, hit area size and how easy an app is to navigate.

Mobile testing best practice

So, what constitutes a strong mobile test environment? First, appreciate the need to test on both real and virtual devices. While some test scenarios are still best carried out using physical phones, there are also tests that come up earlier in software development that lend themselves to a more virtualised approach. By implementing this best practice, app teams can take on much larger volumes of testing while reducing overall costs. Issues can be pinpointed earlier, which is known as Shift-Left testing, a concept within DevOps whereby more testing is carried out earlier in the development cycle.

The well-established test pyramid helps teams work out what tasks can be performed virtually. For instance, unit tests (isolated pieces of code) at the bottom of the pyramid lend themselves to virtual test labs, while user interface or accessibility tests might best be carried out manually on real devices.

Test coverage priorities

In addition, apps need to perform well on an ever-increasing array of devices, as well as different operating systems (OS) and web browsers optimised for mobile. Plus, while still early, foldable devices are on the increase. While app developers will want to support legacy devices that are still in popular use, there comes a point when decisions must be made over what to prioritise.

So, it is important to know which devices, OS, and browser versions are being used by your customers, and that can vary according to geography: a phone that has been a big hit in Asia might have made little impact so far in Europe. Similarly, an update to an OS may be more widespread in one country compared to another. Make use of publicly available resources, such as Perfecto’s regularly updated and free Test Coverage Guide, to learn what devices, Oss and web browser versions are most used in a particular country or region.

Beta testing

It may sound old-school, but one of the best ways to verify how an app will be received is to carry out beta-testing before a release. Beyond engaging with beta testers who have been loyal in the past, look for people who may have engaged with other apps from the financial services provider, or who have left feedback to similar apps online. Another approach is to have a limited launch campaign, only releasing an app in one country to gain real user feedback. Consider designating certain areas of the app for different testers on which to focus.

Load testing

Apps and websites being overwhelmed by traffic hit the news more than a few times in 2022, and it was a reminder to developers worldwide that load testing is needed to make sure that the user experience is consistent across high and low traffic periods. So, load testing that simulates real world scenarios should be an integral part of any robust test strategy.

Mock services

An sometimes overlooked aspect of testing is that it often requires access or data from another service or API. If that service has not yet been built or the test data is unavailable during testing, then the entire development project can get delayed.

An answer to avoiding the wait for test data is to use mobile mock services, which are a facsimile of real services. By simulating responses from a mobile gateway, for example, mock services can allow testing to proceed as if the mobile gateway is actually operational. Mock actions receive mock responses, providing the tester with a realistic idea of how tests will work once the real services are available.

Mock services are useful when a team is early in the development of a new app or feature, or when there are firewall or other network constraints. For instance, using a mock API could be used to test an app against lifelike account services (user log-in), transaction services (visibility of where transactions are stored), or to locate nearby ATMs.

Low-code/no-code, AI/ML

In recent times, there has been an exponential growth of automated and continuous testing tools. Furthermore, there are a growing number of initiatives from a variety of testing technology vendors to make testing more accessible to a wider range of people, including those with no test experience whatsoever. Technologies such as low-code/no-code testing and those using artificial intelligence (AI) or machine learning (ML) help a greater throughput of testing to be performed.

The growing use of AI and ML, existing tools and techniques, plus others yet to become apparent, all contribute to a continuously changing test environment. And that leads to a final point: as software evolves, so does the need to review and embrace the best that the world of testing has to offer. Therefore, testing strategies should not be ‘fit and forget’, but instead, be continually re-evaluated to best fit the requirements of modern financial services providers and their customers.