How do you reduce code maintenance of your end-to-end tests?
You spend significant effort to develop end-to-end software tests of your web and mobile apps. Do you think you spend too much time on test maintenance?
We assume you are reading this because you have built automation for your end-to-end application tests. When organizations ran these tests with manual testers, test maintenance included informing the testers about the expected behavior change for enhancements and eliminated features, as well as the behavior of new features. Testers might have been informed about appearance changes. Organizations depended on the skills and detail orientation of manual testers to catch any errors.
Because you invested in test automation, you specify your tests as part of your software development process. Your test construction, however, becomes dependent on developers, because your test controls depend on artifacts your coders create.
Web element locators identify the fields for entering data to set a test condition, as well as the elements to click and initiate action. You might also use locators to validate your test output. Your end-to-end test maintenance effort depends, in large part, on ensuring that locators remain consistent.
Locators And End-To-End Test Maintenance
Application updates can cause your locators to fail. How do you uncover locator changes that can impact your ability to set test conditions or correctly validate output?
Ideally, developers agree not to touch locators. Once they’re set, that’s how they’ll be used. Realistically, locator changes occur and testers get surprised by failing tests.
Some advanced development teams set locators programmatically. You might not depend on statically-set locators. You might use a look-up table or other data structure to uncover the locator you need for a given task. Some programmatic approaches can reduce the failures caused by changing locators.
You incur maintenance costs when you upgrade your application and discover failing tests. If you have failing tests after an application upgrade, you will need to debug the test code. Ultimately, you have to figure out if the test failed due to broken application code or broken testing code.
Actually, you cannot count on the passing tests, either. When you upgrade your application, your test code might continue to pass, but parts of your application unusable. It is difficult and cumbersome to write locators that uncover visual errors.
Visuals As An Alternative To Locators
Visuals provide a great alternative to coded locators. Some test approaches require visual images to apply test conditions. Other approaches use visuals to validate the output of a test.
When you use visuals to control the test conditions, you need a way to ensure that your visual-based controls update consistently. If you update the application, your visuals might change, and you will lose the ability to control a test.
For visuals as outputs, you need a way to identify and approve expected changes and identify unexpected changes as bugs. Since visuals can apply to multiple pages (think an app header and footer), you also would like to group common changes across multiple pages and accept them in a bulk action.
For configuring test conditions using either visuals or locators, your will need to do maintenance work when your applications change the layout or naming of the control interfaces. For validation, however, visual is much more efficient. Visual validation can capture all the changes that affect a given page. When you
Applitools For End-to-End Tests
At Applitools, we focus on visuals to validate the output of any test condition you apply. You can use Applitools along with coded locators to ensure you catch visual errors. Or, you can choose to remove coded locators and replace them with a visual capture and validation. step
We recommend using Applitools to validate your end-to-end tests for several reasons.
- End-to-end tests validate the user experience, which is inherently visual.
- You likely already have functional coverage for your unit and system tests. Using those tests verbatim for end-to-end tests gives you no additional knowledge of your application behavior.
- End-to-end test code with visual assertions requires much less code maintenance.
- Applitools Auto-Maintenance and newly-announced Smart Assist make it easy to deploy and maintain your visual assertions.
Applitools uses an artificial intelligence (AI) engine to recognize individual visual elements on a page. We call this capability Applitools Visual AI. Rather than use pixels to determine visual differences, Applitools identifies and compares visually-distinct versions of elements. Visual AI distinguishes Applitools from all other visual comparison solutions because
By using Visual AI, you capture an entire page with a single assertion. Your captured page gets loaded into the Applitools Eyes server. The output comparison relies on fewer lines of code and reduces any fragility associated with using locators.
Applitools Reduces End-to-End Test Maintenance
Applitools makes it easy to maintain the visual data it collects. A great Visual AI feature is called “Automated Test Maintenance”, or Auto Maintenance. With Auto Maintenance, Applitools identifies common visual differences found on multiple pages. When you inspect a difference found by Applitools, you can choose to accept it or reject it. Once you have made the choice, Auto Maintenance lets you know about the other pages with the identical difference. With one click, you can choose to accept or reject the change for those pages as well.
For instance, you update the color scheme of your application logo. Ths logo is found in the same place on every page. You might think you have to go page by page and accept the change. Auto Maintenance makes it easy to accept all the update logos. You accept the first found logo change, then you are prompted to accept the changes everywhere else.
Simplifying Deployment With Smart Assist
Another useful feature in Applitools Visual AI is the recently-announced Smart Assist. Applitools allows you to choose among three useful visual comparison levels: Strict, Content, and Layout.
- In Strict mode, Applitools highlights any user-identifiable visual change.
- In Content mode, Applitools inspects content, not the color, font, or other visual difference. For example, you change CSS and want to ensure that nothing else has changed.
- Layout mode handles dynamic content on a page. Applitools compares the structure around the dynamic content, and that the dynamic content has no errors.
On its own, Applitools provides you lots of flexibility. You can choose the comparison level for a given page. If necessary, you can apply different comparison level to a specific region on a page. For example, say you have third-party ads on your pages. You might define those regions as “Layout”, while the rest of the page is Strict.
With Smart Assist, Applitools suggests the appropriate comparison level as well as exceptions for individual regions within the screen images. Even if you choose Strict as the default page comparison level, Applitools will suggest appropriate exceptions for you. Strict, content, layout, or even ignore – Smart Assist can help you make the right call.
Conclusion
Visual AI eases test deployment and maintenance. You eliminate the need to validate pages based on DOM locators, which reduces your total lines of test code. You eliminate the need to update DOM locators in your test code, reducing test debug and maintenance time. And, your resulting test code evaluates your application based on what a user sees – not simply inspecting the DOM.
Applitools Smart Assist helps you ensure that you use the right level of comparison for your application visual tests. You can become an Applitools expert by letting Applitools Smart Assist be your guide.
Cover Photo by Peter G on Unsplash
For More Information
- Visual Testing: A Guide for Front End Developers
- Modern Cross Browser Testing with Cypress and Applitools
- How Do You Catch More Bugs In Your End-To-End Tests?
- Sign up for a free Applitools account.
- Request an Applitools demo.
