As teams get bigger and mature their testing strategy alongside the needs of business, new challenges in their process often arise. One of those challenges is that the analysis and maintenance of tests and their results at scale can be incredibly cumbersome and time-consuming.
While a lot of emphasis gets put on “creating” new tests and reducing the time it takes to run them across different environments, there doesn’t seem to be the same emphasis on dealing with the results and repercussions of them.
Let’s say you have a test that validates a checkout experience and you want to expand that testing to the top 10 browsers. Just two bugs along that test scenario would produce 20 errors that need to be analyzed and then actioned on. This entire back and forth can become untenable in the rapid CI/CD environments present in many businesses. We basically have to choose to ignore our test results at this point if we want to get anything productive done.
This is where Auto-Grouping and Auto-Maintenance from Applitools come in, as it allows AI to quickly and accurately assess results just as an army of testers would!
What Is Automatic Analysis?
Applitools Auto-Grouping helps group together similar bugs that occur in different environments like browsers, devices, screen sizes, etc. Applitools even allows you to group these bugs between entire test runs, test steps, or even specific environments allowing you to really fine-tune your automation.
In the above scenario, let’s assume we found 2 bugs across our 20 browsers for a total of 40 bugs! When we enable Auto Grouping, our errors are grouped together to present only 2 bugs – making it much easier to analyze what actually is going wrong in our interface and cutting down on chasing repeat bugs.
What is Automatic Maintenance?
Auto-Maintenance builds on Auto-Grouping by automating the process of updating tests based on their test results. Auto-Maintenance also enables users to set granular controls over what gets updated automatically between checkpoints, test runs, and more.
Again, taking a look at the above example, if we accepted a new baseline on one browser, we’d have to accept it on the other 19 browsers manually – taking up a ton of time. When a new baseline is accepted, Auto-Maintenance can apply that acceptance across all similar environments saving you hours of writing new tests that would accommodate those new baselines.
How Sonatype Saves Money & Time With Automatic Maintenance
Jamie Whitehouse and everyone on the development team spent time on each release working to uncover and address new failures and bugs across different browsers. Often, this work occurred as spot checks of the 1,000+ pages of the application during development. In reality, this work, and the inherent risk of unintended changes, slowed the delivery of the product to market.
Now, if Sonatype engineers make a change in their margins across a number of pages, all the differences show up as highlights in Applitools. Features in Applitools like Auto-Maintenance make visual validation a time saver. Auto-Maintenance lets engineers quickly accept identical changes across a number of pages – leaving only the unanticipated differences. As Jamie says, Applitools takes the guesswork out of testing the rendered pages.
Start Saving Your Testing Time
To get started with automatically maintaining and analyzing your tests, you can check out our documentation here.
You’ll need a free account, so be sure to sign up for Applitools.
