If you use visual testing tools, one of the biggest challenges you’ll face are displacement diffs: something gets wider, narrower, taller, or shorter and other visual elements shift vertically or horizontally. The problem of shifting elements affects all kinds of snapshot test tools. How do you inspect just the changes?

Visual element changes between builds can shift other elements up, down, or side to side. Applitools Eyes spots these changes but does not know if they are either intentional or acceptable. Should they be highlighted as changes? Or, should they be ignored as acceptable? In Applitools Eyes, 10.7, we make it easy for you to inspect just the changes. You can choose to turn off notification for displaced items.

In this third blog about the new Applitools Eyes 10.7 features, we cover a new feature, “Hide Displacement Diffs.” You can also read about our new visual testing insights feature, as well as our integration with Atlassian Bitbucket.

Automated visual testing compares the visible results of your applications easily across many test environments. Of course, keeping track of all the screen sizes and operating systems in their tool leaves Selenium or Cypress users with a massive maintenance headache. Snapshot testers get bogged down trying to figure out which snapshot differences are valid.

What Causes Visual Displacement?

Visual displacement occurs as visual elements on a page change size. One thing changes, and everything else is affected. You want to inspect just the changes, but everything has changed.

When a CSS change affects font sizes, large sections of the page move accordingly – both horizontally and vertically. This displacement causes the other elements on the page to shift down or up, right or left. It’s a big issue – one for which developers have invested energy in training to solve.

When one visible element of a page goes missing or gets added, other parts displace. For example, a bar across the screen in the baseline goes missing in the captured image. Everything else shifts upward. Effectively, the entire page from that element down changes.

How does your visual testing tool handle these kinds of differences? Yes, these displacements result in shifts across the page – rendering plenty of potential discrepancies. Can you inspect only the changed elements and ignore elements that remain the same except for displacement?

An Example: One Change Affects Everything Below It

Let’s look at an example comparison between a baseline and a checkpoint image.

Here are two screens – a baseline on the left, and a checkpoint on the right. How many differences can you spot?

Next let’s take a look at the two screens in the Test Editor with differences identified, using “Strict” comparison. In “Strict” mode, Applitools Eyes highlights any visible change.

Wow, look at all the pink highlights. Starting from the top, you spot a difference with the wifi icon. The clock on the top right shows a different time. Further down, one page has the status of “Open” in green, while the other has the status “Closed” in red. Further down, starting from the horizontal line, Applitools highlights everything as pink and different? Why?

Compare the two lines. On the checkpoint image on the right, the thinner horizontal line shifts all the other bits further up the page. In Strict mode, Applitools notes everything down the page as different.

The screen on the left has the baseline image, and if you look just below “Opening time,” you’ll notice a thick horizontal line. The screen on the right has a thinner line, which affects all the pixels below it.

Inspect Just the Changes in Applitools Eyes 10.7

In the latest version of Applitools Eyes, we can easily distinguish between images that are the same but displaced. What we cannot do is answer that question about the developer’s intention. So, we leave it up to you. And, we make it easy to record your preference and keep it going forward.

Once you are testing results, you’ll see a new icon on the screen. It looks like this:

When you hover over this icon, it tells  you it means “Hide Displacement Diffs.” This icon shows up in both the Step Editor and the Test Editor in Applitools Eyes.

]

The Hide Displacement Diffs button is in the View Toolbar. When you click it for the images above, Hide Displacement Diffs will toggle to the following view:

Here, Hide Displacement Diffs ignores all the elements that would render the same on both the baseline and checkpoint. From the narrower horizontal line, every displacement difference loses its highlight. So, the only other highlighted change shows the added text for hours showing Saturday-Sunday as “Closed” in the checkpoint.

Hide Displacement Diffs lets you eliminate highlights for all the elements whose relative size and content remain unchanged. Once set, the Hide Displacement Diffs setting for a given page will persist for that pair of captured page and baseline.

Look For Changes That Impact Everything

If you use traditional snapshot testing tools today, you will tell us what our customers tell us about their time using those tools – that you hate wasting time. We know that you spend all this time storing baselines and capturing snapshots, only to discover a large number of false positives. You want to know what makes Applitools different – and more efficient for you.

Applitools Eyes runs on a simple premise – if something looks different to the human eye, show it. Hide Displacement Diffs lets you choose what to highlight. You can choose to show all the visual changes. You can choose to ignore displacement of the elements that have remained entirely the same and focus on the changed items only. This new feature helps you become more productive by focusing solely on the cause, and not also the effects of visual change.

If you’re using Applitools Eyes today, get ready to try Hide Displacement Diffs.

For More Information

• Learn about another 10.7 Feature – Insights into your visual testing.
• Read about another 10.7 Feature – our Bitbucket integration.
• Understand why screenshot image comparison tools fail with dynamic visual content.
• More from Applitools: Request a Demo, Sign Up for a Free Account, or visit our Tutorials.

The post Shifts Happen – Inspect Just The Changes That Matter appeared first on Automated Visual Testing | Applitools.

In our State of Automated Visual Testing report, we share our research and insights about the cost of visual bugs found in production in companies like yours. Based on the number of builds you release a month, and the total number of unique screens on which your application runs, we can help you estimate the number of visual issues you might expect to find — and their cost to you.

Now, we can help you get real data on your own development processes. Applitools Eyes 10.7 introduces a new set of built-in metrics, called Insights. The new Insights feature provides you with a visual display showing your current test status, including passing tests out of all tests run, failing tests, as well as new results to inspect and evaluate. You can view your overall results, as well as filter your results by batch run or by a named team – as well as report results over a specific date range. With Applitools Eyes Insights, you can track the progress of your visual testing projects.

Engineers measure, because we quantify everything. We want to know how much faster we can complete process A compared with process B. Or, we might want to know the resource requirements to complete a given task. Other times, we want to know the quality of our processes. We are measuring constantly.

When you subscribe to Applitools Eyes, you do so with an implicit assumption that visual testing with Applitools makes your team more productive. You expect to run more tests more easily. And, you expect to uncover a greater number of visual regressions that require a fix. Like most of our customers, you get a sense that your code behaves much better. And, like most of our customers, you want a way to quantify the improvement.

Introducing Applitools Insights

Applitools Insights reports on the tests you run with Applitools. You access Insights from the Applitools Test Manager page – part of the set of reports that show your results from your test runs.

The Insights page provides you with a number of charts that give you visual testing insights related to the performance and effectiveness of tests that teams execute on your account. Reviewing the charts allows managers to identify ongoing trends or risks. The scope of data that you can view is role based. Account admins can view data at the account and team level. Users that do not have admin rights can only view data that relates to the teams for which they are a member.

You generate the charts by specifying the value for two settings, the Scope, and the Date Range. The Scope determines if data is taken with respect to the entire account or one or more teams that are assigned to your account.

To set the scope, select your scope from the “SCOPE” pull-down. Your choice for scope can be your entire company, or a specific team or teams you identified previously that you select.

The Date Range sets the time frame boundaries for which the Insights data is collected. You set the date either in a preset of the past 7, 30 or 90 days, or a custom range you specify in the date selector.

The Insights page layout

The Insights page is divided into several sections:

• Insights toolbar.
• Tests Analytics tab.
• Batch Statistics tab.
• Tab toolbar.

The Insights toolbar

From the Insights toolbar, use the Scope selector to select the account or teams for which you display visual testing insights. Next, use the Date Range feature to set the time frame used for extracting information from the database.

The Test Analytics tab

This tab displays a set of charts that provide at-a-glance insights into all your executed visual tests. The statistics reflect the scope and timeframe you set in the Insights toolbar.

The Batch Statistics tab

Displays a list of Batches that were executed over the selected time frame. Each row in the list displays a separate batch and basic batch run and status information such as Last run, Pass, Fail, Unresolved and Total runs. Clicking a row expands the row to display a set of charts and analytics for that Batch.

The Tab toolbar

Provides various tools for arranging the display or extracting and sharing information presented on the tab. The available toolset varies depending on which tab is active.

Other Tab toolbar controls include:

• Arrange display: when you click the Arrange display icon, the page changes to allow you to reposition the charts. Drag and Drop the charts into a new position and then click the Arrange display icon again to return to the regular display.
• Print PDF: to create a PDF of the current report, click the PDF icon to capture the screen, ready to save in PDF format.
• Share link: To share the current report with someone, click the Share link icon to copy a link for the current page to the clipboard. From that point, you can share the link.

Looking at Test Analytics

The Test Analytics section includes five basic reports:

• Overall test results status – pie chart showing Passed, Failed, and Unresolved for scope and date range.

• Results of test execution – This report takes the form of a vertical bar chart. When you open it,  you see bars showing the number of tests and horizontally broken up by time. Test Execution shows overall tests run over the entire date range. You can find Passed, Failed, and Unresolved and shown in bars broken up in time. If instead, you choose Test Execution by Team, you see tests run in total by a given team or teams over time. The Test Execution by Team gives you visual testing insights for a team or set of teams over time.

• Defined tests over time – a line chart for a selected scope of teams showing the number of defined tests run over each period. A separate line is shown for each team
• Detected issues over time – a line chart for a selected scope of teams showing the number of issues discovered over time, broken down by team
• Number of executed tests by – horizontal bar chart, broken up vertically by parameters. This report gives you insights into your visual testing, organized by an individual, by team, or by target test platform. For a given scope and date range, the number of tests across a selected set of test parameters. Parameters include Assignee, Browser, Device, OS, and Viewport. Within these, appropriate parameters segregate (e.g by a named browser). You can also view your most and least stable apps and tests. This way, you know which tests to focus on maintaining, and which apps to focus on testing. Here’s a chart of our least stable tests, for instance:

Looking at Batch Statistics

If batch runs interest you, we thought of you in our design. Applitools Eyes Insights uses the Batch sequence name to group batch data automatically. Insights collects, analyzes, and presents batch and test execution status information. You see just the visual testing insights for the batch tests meeting your scope and date range criteria.

Each Batch sequence is presented in its own row in the Batch sequence list. Within that row, you can see all the data for that batch sequence, including status, run time, and most recent result. For your selected scope and date range, the report shows you insights into all your visual testing runs. You collect insights into the visual testing for the batch, including the number of passed, failed and unresolved images, as well as total runs.

Clicking a row expands the row to show a set of charts and analytics that provide further visual testing insights and statistics on the batch sequence.

Next Steps

• Read our blog post explaining why screenshot image comparison tools fail with dynamic visual content.
• Read our blog post about another new Applitools Eyes 10.7 feature: How to integrate BitBucket into your Visual Testing
• Read Dave Haeffner’s Series on CICD Automation
• More from Applitools: Request a Demo, Sign Up for a Free Account, or visit our Tutorials.

The post How To Gather Visual Testing Insights appeared first on Automated Visual Testing | Applitools.

Why is Jira so hard to use for bug tracking? This seems to be a continual refrain heard on the web: “Jira isn’t suited for Agile development”. “Jira is complex and bloated”. “People cram too much information into Jira”. Another common complaint is that Jira tickets are simply unmanageable.

Organizing JIRA tickets pic.twitter.com/2YeDNc7tcf

— Mike Herchel (@mikeherchel) October 2, 2017

But, love it or hate it, Atlassian Jira is very popular for bug tracking and can be found within many software development organizations worldwide. If your workplace is one of the many that uses Jira, you might want to consider using Applitools Eyes to make your visual user interface (UI) testing easier.

First, some background: Applitools Eyes lets you automate visual UI testing of your web and mobile application front end. Using Applitools Eyes ensures that your application’s UI is displayed consistently and accurately across any device, browser, or viewport. Atlassian Jira manages your development workflow, task tracking, issue management, and reporting.

Integrating Applitools Eyes and Jira will save you time, especially if your visual tests detect many defects. Manually logging visual defects may take a lot of time: grab a screenshot, draw a circle around the defect, write some text explaining what the problem is, create a Jira ticket, attach a screenshot, then have a follow-up call with a developer because they didn’t understand your write-up. This may easily add three to five minutes just to report and explain one software bug.

Three to five minutes per bug? What a time sink…

There’s a better way: integrate Applitools Eyes with Jira. Doing this lets you directly report issues to your engineering team, and track those issues to resolution. Highlight a region, write a quick comment, tag your teammate responsible for investigating, and you’re done. A bug report with screenshots with highlights of all visual differences, along with your comments, filed in Jira and your teammate is notified. Total time: about thirty seconds to file a bug.

With the manual tasks of reporting and tracking out of the way, testing teams can focus on increasing test coverage (and leaving the office at a reasonable time). This post explains how to quickly report issues and defects captured by Applitools Eyes to a Jira project.

Configuring Applitools

First, open the Applitools Eyes Admin panel and locate your team. Find the Jira section on the page and click the checkbox to enable the integration. In the text box, type the URL of your Jira server. Next, specify the Project ID (PID) and the ID of the relevant issue type for the target project. If your project uses additional Jira fields, type the field name into the text box and press the Add button. In the drop-down menu, select a field type. Alternatively, you can type the name of a custom field into the text box. After you provide all the relevant configuration information, save it.

Applitools Eyes Jira Integration in Action

Now that you have configured the Applitools Eyes Jira integration, we will show you how to use it. In this example, we will simulate a defect in an existing web application (Gmail), and then use the integration to create a new Jira issue. The test inspects Gmail’s More button and detects a typo. When a test detects an issue, click the Create New Jira Issue button. This opens a new browser tab that displays the Jira Create Issue form. In the form, each field you specified during the configuration process contains the relevant Applitools Eyes test information. You can edit the information in the form and populate any empty fields. When you are ready to create the issue, click the Create button.

After you save the detected defect, Jira creates an issue that includes the name of the test, the batch in which it was running, the step description, the time when Applitools Eyes encountered the issue, the execution environment in which it was running, and any custom properties you defined for the test. The issue includes a link that lets you view all the defects in the Applitools Test Manager. This allows you to examine the recorded visual validation in context and compare captured screenshots with the test’s baseline images.

You can add existing visual regression test results that were not linked to a Jira project using the Link to Jira issue dialog box. The dialog box lets you enter an issues Applitools ID and add it to Jira. After you add an issue from Applitools to Jira, click the View in Jira button from an issue region or the Collaboration View to view the defect in Jira. You can also use the dialog box to link to a different Jira issue, or you can just delete the link.

Find visual UI bugs before your customers do

In this post, we explained how our Applitools Eyes and Jira integration can boost your team’s productivity. Integrating Jira with Applitools will help you streamline communication with your engineering team, which will allow for more efficient management and tracking of issues. As a result, your software development team will ship better quality software in less time.

But productivity is only part of the picture. It’s absolutely critical that you don’t let visual user interface bugs escape into production. When this happens, your customers will lose trust in your software and brand.

Even worse, you can have a visual bug that blocks customers from buying, such as this glitch on the Southwest Airline’s website. See how the text is overlapping the Continue button? Because of this, the Continue button doesn’t work, and customers can’t book a flight.

From a business perspective, this bug is disastrous since it is blocking revenue generation and is actively driving customers to competitors. It’s these kinds of revenue-killing production bugs that software QA and development teams need to be able to report and act on quickly, and 

So, once you’ve integrated Applitools and Jira, what will you do with all the time you’ve saved?

If you are a Jira user and want to read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post How to Make Jira Easier to Use with Integrated Visual UI Bug Tracking appeared first on Automated Visual Testing | Applitools.

Many people think of GitHub as a niche website that hosts open-source projects. In June 2018, that niche website with a mere 28 million users got more media attention than Apple’s Worldwide Developer Conference (WWDC). Somehow, a site that provides free services was so valuable that Microsoft acquired them for $7.5 billion US dollars.

Since its launch in 2007, GitHub has grown far beyond its roots as a web-based service that provides a hosted version of the Git-distributed version control system. The site hosts open-source projects and also offers paid plans for developers and companies. With the adoption of agile development and DevOps practices, it has become an essential part of the continuous integration/delivery/deployment pipeline for many companies. Not only do these companies host their source code in private GitHub repositories, but they also integrate their repositories with their continuous integration (CI) systems.

An important part of any CI system is the ability to run automated tests on any code committed to a source control repository. The automated tests help prevent new code from breaking the software built by the CI system. Now, with the release of the Applitools Eyes GitHub integration, you can include automated tests that perform visual UI testing as part of your build process. Visual UI testing lets you create tests that accurately simulate real users interacting with your software. The integration also takes care of uploading test results to Applitools’ advanced Automated Visual Management (AVM) system for analysis.

This post explains how you can use the Applitools Eyes GitHub integration to perform visual validations for pull requests. Integrating Applitools Eyes with GitHub prevents visual defects from merging into your production software. For each pull request, it merges the baseline screenshots of the branch along with your source code, and runs automated visual validation tests.

Applitools Eyes GitHub Integration

Applitools Eyes ensures your application looks and functions exactly as expected across all browsers, operating systems, devices, and screen resolutions. In addition to handling application testing, the Applitools Eyes GitHub integration lets you use Applitools Eyes as an integral part of your build automation process.

The Applitools Eyes GitHub integration automatically configures the baselines used for obtaining and saving test baselines. It displays the status of your tests directly in the pull request page, right next to the merge status. The integration lets you view the history of an application’s user interface (UI) and see how it has changed, what has been changed, and by whom. This will show you how your product’s UI has evolved over the product’s lifetime. The feature allows you to merge visual testing baseline branches in the same way you merge changes to your test and source code. For each branch in your GitHub project repository, you can see the history of all your test baselines, compare them to prior baseline versions, and if necessary, revert to an earlier version. This feature lets you reject and/or revert to any baseline you modified by accident or by design.

Visually Validating a Pull Request

In order to illustrate how to use the Applitools Eyes GitHub integration, we created a complete demo that includes all the relevant source code files and project-related assets. The demo is called merge-demo, and we made it available as a GitHub repository. The demo runs visual validation against the Applitools Hello World test page. The test navigates to the Hello World page, visually validates it, clicks the Click Me button, and visually validates the page. In this demo, the Hello World test code has been modified to test the page in two different viewport sizes.

Configuring the Integration

Before you can run the demo, you need to install and configure the Applitools Integration application. In the merge-demo repo, open the Settings > Integration & Services page. The page indicates that the Applitools Integration application has been installed. The page also indicates that the Travis CI service has been configured to build and run tests.

Creating the Pull Request

To create the pull request on your PC, open a terminal window and clone the merge-demo locally. Create a new branch, called mybranch, and modify the tests. Commit the changes and push them back to the remote branch. Now, create a pull request to merge the changes with the master branch.

Building the Request

After receiving the Pull Request, Travis CI initiates a build for the pushed pull request. For the sake of simplicity, we recommend you disable the Builds for the Pushed Branches option; otherwise, Travis CI will build two individual branches.

When the build is complete, open the Pull Request page. The Test Applitools status indicates the status of all visual tests that ran as part of the build. We can see the two tests detected differences. If you click the details link, it will take you to the corresponding page in the Applitools Test Manager. The batch contains two tests with different viewport sizes. The test name includes the branch name, the name of the pull request, the source branch, the commit message, and the SHA. Using the page, you can accept the changes made to the first test and reject the changes made to the second test.

Managing and Merging Baselines

Reopen the Pull Request page and you will see the Status Description now indicates that one of the visual tests has been failed. If you accept the changes in the second test, you can see the status changes to green to indicate that all of the visual tests were passed. By looking in the Test Details, you can see the tests were configured to run in the source branch. After reviewing the test results, you save them to the baselines of the source branch. The updated baselines will not affect any tests running in other branches.

After updating your baselines, you can check that other team members did not make conflicting changes to the baseline. You can do this by checking the SCM status of the Applitools request, which is currently pending. Next, open the Pull Request page. This page displays the details of all the changes in the baselines, compares them, and allows you to edit the source baseline by porting ignore regions and locations from the target baseline, or by adding new ones. The page shows that the Applitools SCM status changed to green and no baseline conflicts were found. This means the two changes in the source branch are ready to be merged with the target branch. As soon as your source code is fully merged, Applitools will automatically merge the changed baseline in the source branch with the baseline of the target branch.

Visual UI Version Control

In a very short time, GitHub has grown from an obscure developer-centric website into an essential tool for a wide range of enterprises. It serves the needs of freelance contractors, large corporations, and governments. GitHub may have started as a cloud-based version control system, but today, it provides additional services such as project management, online documentation, web hosting, and defect tracking. For many companies, GitHub has replaced their on-premises source control management systems, and it functions as an essential piece of their CD/CI tooling.

In this post, we looked at how to use GitHub as part of your DevOps stack. We explained how the Applitools Eyes GitHub integration automatically configures the baselines used for obtaining and saving test baselines, how to create pull requests, and how you can add visual test validation to your CI pipeline.

This solution gives you powerful tools that ensure each time a pull request triggers a build and merges your target and source branches, it also gives you control and insight into the process. The integration also simplifies the process of reviewing and managing visual test baselines. More importantly, it is a form of UI version control that shows how your application’s UI has changed and evolved over time. These Applitools Eyes GitHub integration benefits will ensure the quality of your products and will benefit your end users and customers.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post How to Test Your GitHub Pull Requests so You Don’t Break Your UI appeared first on Automated Visual Testing | Applitools.

We are rolling out a shiny new UI for Applitools Eyes, with a brand new look and feel, and improved user experience on all fronts of automated visual testing.

• The new test manager is designed to streamline large-scale baseline management.
• There is a new step thumbprints gallery for easier to search test validation.
• We have improved batch, test, and step searching and filtering for easier search and test analysis.
• We have improved automated maintenance capabilities, including grouping of similar differences flagged across entire batches of tests.
• There are substantially reduced wait times for tests to complete.

We also share the best practices for using the new UI in the test manager.

What the full on-demand recording of the webinar featuring Applitools CEO Adam Carmi sharing the new Applitools Eyes UI here:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Applitools Eyes is Launching a New Shiny UI appeared first on Automated Visual Testing | Applitools.