The impact of software releases and testing on customers’ mobile and app ratings

By Eran Kinsbruner, Chief DevOps Evangelist at Perfecto by Perforce 

Download a free copy of the ebook here

Web and mobile apps are the front-line for many customers’ interactions with banks and other financial services organisations, so they have an essential role in customer satisfaction. These digital shop-fronts are the equivalent of a good or bad experience in the traditional world of teller-based high-street banks, where regular customers would strike up a relationship with the staff. That experience is harder to reach in the digital world, which is why any ways in which to improve the performance of mobile and web apps has rapidly become a market differentiator. 

However, as a benchmarking exercise performed by Perfecto among various banks’ web and mobile apps, there is variation in their quality. High app store ratings and excellent web performance matter, plus the research also found a correlation between the frequency of app releases and quality. To provide a view of both ends of the spectrum, Perfecto also compared the highest rated app with one that scored below average. 

We will take a closer look at the results in a moment, but first, here is some background about the data collected between mid-June and mid-July 2021 across 100 medium-to-large financial services companies worldwide. Web benchmarking data was gathered with Lighthouse and mobile benchmarks with App Annie, both widely regarded as reliable sources of current and independent insight.  

Mobile app performance

The good news for banks is that they are above average compared to other market sectors. For instance, the average IoS app store rating for a business is 3.47, but the average score of the 100 financial services companies evaluated is 4.3, rising to 4.5 for the top 25. Furthermore, those top 25 have no reviews with less than three-star ratings, at least now. In a competitive world of customers with high expectations, that is an impressive achievement. Furthermore, over half of the top 25 received scores of 4.5-5 stars. 

Web apps told a slightly different story. For the top 100, performance — around various speed-related checks — scored 36.8 per cent, dropping to just 33 for the top 100. It is important to view this in the move towards progressive web applications (PWA), which brings opportunities and a learning curve for banks. 

However, banks’ web accessibility faired better (around 85 per cent for both) and best practices over 70 per cent in both categories. Accessibility criteria included adherence to WCAG guidelines and ARIA to ensure that everyone can access a website, including those with disabilities. Best practices related to common web development errors could affect website security.

App release frequency and impact on quality

The data shows a strong correlation between the frequency of app releases (in other words, updates and new features), which is not surprising because update strategies affect user feedback and overall app success. Of course, quality is everything: teams need to be quick to sort out bugs, but trying to transform features too frequently can be challenging and bring risk if not executed thoroughly. Across both Android and IoS, top 25 and top 100 banks, the average number of releases between January and mid-July 2021 was between 5.58 and 6.45.

High versus below-average performance

One of the best ways to illustrate the impact of all this data is to look at two banks, comparing the one that scored highest across all data versus one that fell below average. With an average rating of 4.8 out of 5, the leading bank receives minimal complaints about performance issues and functional bugs, with positive reviews saying ‘well-organised, intuitive and user-friendly’ with ‘great features and updates’. In stark contrast, the below-average bank, with a lower rating of 3.15, has multiple reviews relating to poor performance. Users report errors and other complaints such as ‘cannot log in’ and ‘glitchy’.

Of course, different organisations have different resources at their disposal: the leading bank has a $5 billion turnover and 200 million customers, whereas the other bank has an estimated $250-500 million turnover and 500,000 customers. However, given the disparity in these results, it is clearly in the interests of any bank to do whatever it can to improve web and mobile app performance.

Best practice advice

So how can banks improve their mobile and web apps? The starting point is, of course, a well-thought-out design with an intuitive user experience that is in tune with the banks’ customers. Increasingly, organisations are investing in user experience teams and consultancies to make their apps more appealing and easier to use with less friction. Issues such as support for multiple languages and highly visual but simplified layouts can make a big difference, making it easy for users to quickly explore and find what they need.

Performance issues, including speed, eliminating bugs and reducing security risks, lie with efficient software development practices, especially improvements to test strategies. For example, it is essential to test an application on the right mix of end-user devices (mainly mobile phones but also tablets) as well as web browsers. The permutation and volume of those can vary hugely depending on geography. Fortunately, there are regularly published sources of insight into those geographic variants, including Perfecto’s own free and frequently updated Test Coverage Index, which also looks at upcoming device and browser releases.

Use cases

It is also vital to test apps in real-world conditions to see how they handle competing apps, poor network coverage and other challenges. For instance, many banking apps include a ‘find an ATM near me’ feature, but if this is not tested across every place where a bank has an ATM, then there are gaps in the test coverage. In turn, this could mean a risk of error messages when a user in untested geography tries to locate an ATM. This may not sound the end of the world, but given the volume of test scenarios against which to test, the amount — and hence the app rework and damage to customer satisfaction — can accumulate. Similarly, since accessibility features so highly in an app’s rating, it must be incorporated into test strategies.

Test automation

There are several benefits to reducing the volume of manual testing and moving more towards automated testing. First, automated testing for security features such as touch ID, face ID and 2FA helps to reduce manual error and ensure basic security configurations are working. Second, with test automation, more tests can be carried out and faster. Plus, with the introduction of codeless testing — which does not require skilled test engineers and QA managers — more people within an organisation can take on test tasks, including software developers and even non-technical business teams.

Test automation also helps banks address a range of complex scenarios, such as depositing a cheque on a mobile device, as well as support increased volumes of parallel testing across different platforms, so that test coverage is broader, tests happen faster, and bugs are fixed earlier. Likewise, virtual testing — also known as simulators and emulators — eliminates dependency on having access to physical devices so that tests are more scalable and faster in a cost-effective way. Test automation can also contribute valuable data for analysis and reporting purposes, for example, identifying recurring patterns and help assess which tests have the most significant impact on the business.

Test strategies, together with other elements critical to mobile and web app quality, usability and security, will all contribute towards higher customer satisfaction and better app store ratings. Moreover, as Perfecto’s benchmarking report demonstrates, if smaller banks want to compete with larger ones, then focusing more on mobile and web app performance can make a valuable contribution to their brand and market appeal.

Download a free copy of the ebook here

This is a Sponsored Feature