Today’s mobile app development cycle demands short sprints to enable faster deployments. Time allowed for development is shrinking, with even less time for testing. Testing and development must also be scalable as teams face high pressure deadlines, while focusing on quality throughout the pipeline.
These dynamics are very well known, and the industry now includes a variety of quality offerings to ensure mobile apps perform great while delivering value to the customer. However, as testing and/or QA teams increasingly test against the varying device types, OS versions, phone sizes and languages, visual testing platforms are emerging to help ensure mobile apps look great and provide the best customer experience possible. This new dimension to mobile testing addresses the downside and impatience end-users have with mobile apps that are not formatted, sized or able to present a visually pleasing experience.
Consider These Visual Test Scenarios
For a mobile app, testers and QA teams must determine how they are going to test each case. For example, apps today often support multiple languages. Visual testing helps validate the different local languages to make sure they look correct each time within the mobile app. It ensures that the design fully supports multiple languages by accounting for right-to-left reading in languages such as Arabic or Hebrew, as well as left-to-right like the Western languages. App designs also must support varying lengths of text strings for different languages. If testing is done only in the native language, this can create a visually unpleasant and unusable application for anyone who uses it in a different language.
What also makes visual testing vital is how all mobile manufacturers do many things differently. Visual testing can help account for the visual presentation of a single app across each OS version, along with varying resolutions, screen-sizes and types of devices – to name a few. If one of these elements is overlooked, glaring issues are going to surface that will negatively impact the user experience. For example, it is easy to ignore the landscape orientation on a mobile device, because portrait is more commonly used and tested. This is surprisingly overlooked and some apps are completely unusable in landscape – this won’t fly in today’s customer experience-driven world.
What are the solutions?
Theoretically, a developer could try a single-threaded manual approach to visual testing, but the amount of effort required in a short time frame makes that neither effective or efficient, nor fair to the tester. Another option is a multi-threaded manual approach, but that is not going to scale with modern application release demands. This is where automation comes into play, and today there are options for mobile web and native apps. For example, there are visual testing platforms that automatically validate the look, feel and user experience of an app while allowing the tester to run their test locally and scale on a cloud service. This is a developer’s dream – to gain immediate feedback for all code changes and that they don’t adversely impact the overall application in terms of “look and feel.”
These solutions, while new to the industry, are designed to integrate with existing automated test tools and platforms, rather than introducing new tests that are not integrated to the overall quality and performance of the application.
Bringing it All Together
To get started with visual testing and monitoring for iOS, for example, the tester would install Xcode, command line tools, and create a simulator or connect a real device. For Android, they would use the Android SDK and create an emulator or connect a real device and enable debugging mode. Then they would install the automation testing framework for visual testing.
The new setup will check a mobile web app to ensure that it functions and looks correct on every single release, on every single responsive design, and it will all be available in a visual format. Some visual testing frameworks come equipped with a variety of SDKs and libraries. The tester can then set a full-page screenshot which can navigate to a cloud service to run the test. Now, any time a change is made to the application and the visual test runs, it captures the page to validate that it looks correct.
New mobile automation tools with a dashboard allow you to gain visual assertion of where and how the design, copy and images are laid out – and confirm that everything is in the right place. There are dashboards that offer radar features that highlight all the discrepancies that are caught during testing. Then, the tester is able to drill down to the baseline of the visual anomaly and find the issue. For example, perhaps it was caused by a spelling mistake. This automated process aims to catch errors that would be easily overlooked or difficult to find during manual testing. With the technology and tools in place to make this type of testing possible, the next step is to scale.
Some visual test automation platforms can run all on a local computer with multiple devices and emulators connected, however, it would need to be a substantial machine or multiple machines to work. Alternatively, the tester can leverage an internal device grid or cloud test services. Cloud services are the future of visual test automation that can minimize or even eliminate the need for running a local test. Instead, everything will run in the cloud. That will offer ease of setup and integration, because essentially that is already done for the tester.
Today, cloud services can be run parallel with the visual test, offering varying levels of analytics (depending on the service selected). Some cloud services integrate easily with CI services to allow the tester to create a dictionary of different device and OS versions, split them into different threads, and test them, whereas this would be very expensive if done locally.
As automation helps visual testing teams scale their efforts to meet demanding release pipelines, more testing functionality will continue to shift to the cloud. As testing time decreases, teams must ensure the visual integrity of the mobile app. The must confirm that it delivers an aesthetically pleasing experience to the end user in any environment, language or device. This emerging approach demonstrates the importance of visual testing, as automated validation that the application looks and works correctly becomes more mainstream.
Want to learn more about mobile test automation? Watch the exclusive expert session we hosted with Appium Architect and Project Lead Jonathan Lipps, as he walked us through the state of mobile testing and the future of Appium.
