Learn about common localization bugs, the traditional challenges involved in finding them, and solutions that can make localization testing far easier.
What is Localization?
Localization is the process of customizing a software application that was originally designed for a domestic market so that it can be released in a specific foreign market.
How to Get Started with Localization
Localization testing usually involves substantial changes of the application’s UI, including the translation of all texts to the target language, replacement of icons and images, and many other culture, language, and country-specific adjustments, that affect the presentation of data (e.g., date and time formats, alphabetical sorting order, etc.). Due to the lack of in-house language expertise, localization usually involves in-house personnel as well as outside contractors, and localization service providers.
Before a software application is localized for the first time, it must undergo a process of Internationalization.
What is Internationalization?
Internationalization often involves an extensive development and re-engineering effort which goal is to allow the application to operate in localized environments and to correctly process and display localized data. In addition, locale-specific resources such as texts, images and documentation files, are isolated from the application code and placed in external resource files, so they can be easily replaced without requiring further development efforts.
Once an application is internationalized, the engineering effort required to localize it to a new language or culture is drastically reduced. However, the same is not true for UI localization testing.
The Challenge of UI Localization Testing
Every time an application is localized to a new language, the application changes, or the resources of a supported localization change, the localized UI must be thoroughly tested for localization and internationalization (LI) bugs.
Common Localization and Internationalization Bugs Most Testers can Catch
LI bugs which can be detected by testers that are not language experts include:
- Broken functionality – the execution environment, data or translated resources of a new locale, may uncover internationalization bugs can prevent the application from running or break some of its functionality.
- Untranslated text – text appearing in text fields or images of the localized UI is left untranslated. This indicates that certain resources were not translated or that the original text is hard-coded in the UI and not exported to the resource files.
- Text overlap / overflow – the translated text may require more space than available in its containing control, resulting with the text overflowing the bounds of the control and possibly overlapping or hiding other UI elements.
- Layout corruption – UI controls dynamically adjust their size and position to the expanded or contracted size of the localized text, icons or images, resulting with misaligned, overlapping, missing or redundant UI artifacts.
- Oversized windows and dialogs – multiple expanded texts and images can result with oversized tooltips, dialogs and windows. In extreme situations, expanded dialogs and windows may only be partially visible in low screen resolutions.
- Inadequate fonts – a control’s font cannot properly display some characters of the target language. This usually results with question marks or glyphs being displayed instead of the expected text.
Localization and Internationalization Bugs Requiring Language Expertise
Other common LI bugs which can only be detected with the help of a language expert include:
- Mistranslation – translated text that appears once in the resource files, may appear multiple times in different parts of the application. The context in which the text appears can vary its meaning and require a different translation.
- Wrong images and icons – images and icons were replaced with wrong or inappropriate graphics.
- Text truncation – the translated text may require more space than available in its containing control, resulting with a truncated string.
- Locale violations – wrong date, time, number and currency formats, punctuation, alphabetical sort order, etc.
Localization and Internationalization Bugs are Hard to Find
An unfortunate characteristic of LI bugs, is that they require a lot of effort to find. To uncover such bugs, a tester (assisted by a language expert) must carefully inspect each and every window, dialog, tooltip, menu item, and any other UI state of the application. Since most of these bugs are sensitive to the size and layout of the application, tests must be repeated on a variety of execution environments (e.g., different operating systems, web browsers, devices, etc.) and screen resolutions. Furthermore, if the application window is resizable, tests should also be repeated for various window sizes.
Why is UI Localization Testing Hard?
There are several other factors that contribute to the complexity of UI Localization testing:
- Lack of automation – most of the common LI bugs listed above are visual and cannot be effectively detected by traditional functional test automation tools. Manual inspection of the localized UI is slower than with a non-localized UI because it is unreadable to the tester.
- Lack of in-house language expertise – since many of the common LI bugs can only be detected with the help of external language experts, which are usually not testers and are not familiar with the application under test, LI testing often requires an in-house tester to perform tests together with a language expert. In many cases, these experts work on multiple projects for multiple customers in parallel, and their occasional lack of availability can substantially delay test cycles and product releases. Similarly, delays can occur while waiting for the translation of changed resources, or while waiting for translation bugs to be fixed.
- Time constraints – localization projects usually begin at late stages of the development lifecycle, after the application UI has stabilized. In many cases, testers are left with little time to properly perform localization tests, and are under constant pressure to avoid delaying the product release.
- Bug severity – UI localization bugs such as missing or garbled text are often considered critical, and therefore must be fixed and verified before the product is released.
Due to these factors, maintaining multiple localized application versions and adding new ones, incurs a huge overhead on quality assurance teams.
Fortunately, there is a modern solution that can make localization testing significantly easier – Automated Visual Testing.
How to Automate Localization Testing with Visual Testing
Visual test automation tools can be applied to UI localization testing to eliminate unnecessary manual involvement of testers and language experts, and drastically shorten test cycles.
To understand this, let’s first understand what visual testing is, and then how to apply visual testing to localization testing.
What is Visual Testing?
Visual testing is the process of validating the visual aspects of an application’s User Interface (UI).
In addition to validating that the UI displays the correct content or data, visual testing focuses on validating the layout and appearance of each visual element of the UI and of the UI as a whole. Layout correctness means that each visual element of the UI is properly positioned on the screen, is of the right shape and size, and doesn’t overlap or hide other visual elements. Appearance correctness means that the visual elements are of the correct font, color, or image.
Visual Test Automation tools can automate most of the activities involved in visual testing. They can easily detect many common UI localization bugs such as text overlap or overflow, layout corruptions, oversized windows and dialogs, etc. All a tester needs to do is to drive the Application Under Test (AUT) through its various UI states and submit UI screenshots to the tool for visual validation.
For simple websites, this can be as easy as directing a web browser to a set of URLs. For more complex applications, some buttons or links should be clicked, or some forms should be filled in order to reach certain screens. Driving the AUT through its different UI states can be easily automated using a variety of open-source and commercial tools (e.g., Selenium, Cypress, etc.). If the tool is properly configured to rely on internal UI object identifiers, the same automation script/program can be used to drive the AUT in all of its localized versions.
So, how can we use this to simplify UI localization testing?
How Automated Visual Testing Simplifies UI Localization Testing
- Preparation – in order to provide translators with the context required to properly localize the application, screenshots of the application’s UI are often delivered along with the resource files to be localized. The process of manually collecting these screenshots is laborious, time consuming, and error prone. When a visual test automation tool is in place, updated screenshots of all UI states are always available and can be shared with translators with a click of a button. When an application changes, the tool can highlight only those screens (in the source language) that differ from the previous version so that only those screens are provided to translators. Some visual test automation tools also provide animated “playbacks” of tests showing the different screens, and the human activities leading from one screen to the next (e.g., clicks, mouse movements, keyboard strokes, etc.). Such animated playbacks provide much more context than standalone screenshots and are more easily understood by translators, which are usually not familiar with the application being localized. Employing a visual test automation tool can substantially shorten the localization project’s preparation phase and assist in producing higher quality preliminary translations, which in turn can lead to fewer and shorter test cycles.
- Testing localization changes – visual test automation tools work by comparing screenshots of an application against a set of previously approved “expected” screenshots called the baseline. After receiving the translated resources and integrating them with the application, a visual test of the updated localized application can be automatically executed using the previous localized version as a baseline. The tool will then report all screens that contain visual changes and will also highlight the exact changes in each of the changed screens. This report can then be inspected by testers and external language experts without having to manually interact with the localized application. By only focusing on the screens that changed, a huge amount of time and effort can be saved. As we showed above, most UI localization bugs are visual by nature and are therefore sensitive to the execution environment (browser, operating system, device, screen resolution, etc.). Since visual test automation tools automatically execute tests in all required execution environments, testing cycles can be drastically shortened.
- Testing new localizations – when localizing an application for a new language, no localized baseline is available to compare with. However, visual test automation tools can be configured to perform comparisons at the layout level, meaning that only layout inconsistencies (e.g., missing or overflowing text, UI elements appearing out of place, broken paragraphs or columns, etc.) are flagged as differences. By using layout comparison, a newly localized application can be automatically compared with its domestic version, to obtain a report indicating all layout inconsistencies, in all execution environments and screen resolutions.
- Incremental validation – when localization defects are addressed by translators and developers, the updated application must be tested again to make sure that all reported defects were fixed and that no new defects were introduced. By using the latest localized version as the baseline with which to compare the newly updated application, testers can easily identify the actual changes between the two versions, and quickly verify their validity, instead of manually testing the entire application.
- Regression testing – whenever changes are introduced to a localized application, regression testing must be performed to make sure that no localization bugs were introduced, even if no direct changes were made to the application’s localizable resources. For example, a UI control can be modified or replaced, the contents of a window may be repositioned, or some internal logic that affects the application’s output may change. It is practically impossible to manually perform these tests, especially with today’s Agile and continuous delivery practices, which dictate extremely short release cycles. Visual test automation tools can continuously verify that no unexpected UI changes occur in any of the localized versions of the application, after each and every change to the application.
- Collateral material – in additional to localizing the application itself, localized versions of its user manual, documentation and other marketing and sales collateral must be created. For this purpose, updated screenshots of the application must be obtained. As described above, a visual test automation tool can provide up-to-date screenshots of any part of the application in any execution environment. The immediate availability of these screenshots significantly reduces the chance of including out-of-date application images in collaterals and eliminates the manual effort involved in obtaining them after each application change.
Application localization is notoriously difficult and complex. Manually testing for UI localization bugs, during and between localization projects, is extremely time consuming, error-prone, and requires the involvement of external language experts.
Visual test automation tools are a modern breed of test automation tools that can effectively eliminate unnecessary manual involvement, drastically shorten the duration of localization projects, and increase the quality of localized applications.
Applitools Automated Visual Testing and Localization Testing
Applitools has pioneered the use of Visual AI to deliver the best visual testing in the industry. You can learn more about how Applitools can help you with localization testing, or to get started with Applitools today, request a demo or sign up for a free Applitools account.
Editor’s Note: Parts of this post were originally published in two parts in 2017/2018, and have since been updated for accuracy and completeness.
