Overview

Applitools is used by the world’s top organizations to help expedite time to market with Visual AI-powered functional & visual testing. Applitools easily fits into existing toolchains, turbo-charging CI/CD pipelines with AI-infused cross browser testing reducing test maintenance and slashing build times, allowing developers to test more, earlier.

With GitHub boasting over 56 million users as of September of 2020, it’s no surprise that a deep integration with Microsoft’s popular source control solution is critical to our customers’ app-dev workflows and thus success.

This article provides an overview of all the ways in which Applitools integrates with GitHub & GitHub Actions.

Use cases covered in this article:

• What is UI Version Control? (read more)
• Integration with GitHub via GitHub app (read more)
• Integration with On-prem GitHub server (read more)
• [Legacy Integration] with GitHub cloud and on-prem via direct integration (read more)
• Integration with GitHub Actions (read more)

What is UI Version Control?

By enabling the Applitools / GitHub integration, you instantly unlock UI Version Control. With UI Version Control enabled, developers can easily correlate code changes with UI/UX updates for Visual UI version control across web and mobile apps. As branches are created in GitHub, Applitools UI Version Control automatically mirrors the branch and allows for a new set of baseline images. This process simplifies the automated testing of feature branches without impacting the trunk and its tests. Next, as PRs are merged back, UI Version Control automatically updates the target branch with the source branch’s baseline images while ensuring that no conflicts exist between branches, eliminating test maintenance overhead.

Integration with GitHub via GitHub app 

With Eyes 10.11, Applitools now officially supports visual source control via the GitHub app. This interface requires fewer permissions than the Applitools direct integration and is thus the officially recommended integration by the Applitools product team. For step by step instructions on enabling the GitHub integration via GitHub all, visit the integration documentation.

Integration with GitHub server which is installed on-prem

Another great addition to Eyes 10.11 is the ability for GitHub on-prem servers to leverage a new proxy service to eliminate the security challenges IT teams previously encountered with the legacy “direct connect” integration. Setup is quick and easy, simply enable the integration, get the GitHub app, and install the Applitools GitHub Proxy Service.

[Legacy Integration] with GitHub cloud and on-prem via direct integration 

Existing cloud & on-prem deployments who have our legacy direct integration already enabled can continue to enjoy backward compatibility. 

Integration with GitHub Actions

Microsoft describes GitHub Actions as an easy way to automate all your software workflows including with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.

The Applitools integration requires your Applitools API Key and for you to pass Git Commit SHA as Applitools Batch ID. Once it’s all configured, GitHub Actions workflow understands when changes are made to code, and it automatically kicks off the workflow complete with your Visual AI powered test suite.

For more information, check out the article “How Do I Link GitHub Actions with Applitools?” from Satish Mallela 

Additional Links on GitHub & Applitools

• Get a Jump into GitHub Actions
• How to Setup GitHub Actions with Cypress & Applitools for a Better Automated Testing Workflow
• Webinar – Intro to GitHub Actions for Test Automation — with Angie Jones and Brian Douglas
• How to Test Your GitHub Pull Requests so You Don’t Break Your UI

Additional Source Control Integrations to unlock UI Version Control

• GitLab
• BitBucket
• Learn more about rich Applitools ecosystem by visiting our SDKs and Integrations page

(heart icon in header from Kiranshastry at www.flaticon.com)

The post Everything You Need to Know About Integrating Applitools with GitHub appeared first on Automated Visual Testing | Applitools.

We’re excited to share that Applitools has once again been named to the CB Insights 2019 AI 100!

This list represents the 100 most promising privately-held artificial intelligence (AI) companies in the world. These are companies using AI to solve big challenges, and we’re honored to be recognized alongside so many well-respected and innovative companies.

How did we make the list?

Through an evidence-based approach, the CB Insights research team selected the AI 100 from over 3,000 companies based on several factors: patent activity, investor quality, news sentiment analysis, their proprietary Mosaic scores, market potential, partnerships, competitive landscape, team strength, and tech novelty.

Want to give Applitools Eyes a try? Enjoy a free trial of our easy-to-use visual testing SaaS solution

Why Applitools?

Today, if your company has a digital presence, it has to be compelling. It has to be visually flawless. Any less, and you’ll lose the trust of your customers.

After all, you wouldn’t shop in a store with a broken sign. “If they can’t fix such an obvious problem, what else is wrong?” you’d think to yourself.

It’s the same with your digital presence. Visual glitches erode trust in your web or mobile app, which makes customers less likely to buy from you.

But it’s not easy to fix all visual bugs. Software development teams focus on constantly, rapidly delivering a drumbeat of new features. So, if you’re in QA, it’s hard to keep up and check every new feature. Especially given the wide range of phones and web browsers your customers might use.

Traditional functional testing tools aren’t well-suited for catching visual bugs — there are simply too many visual properties to check. So many QA teams revert back to manual testing, which is way too time-consuming to check everything in time.

So visual bugs escape into your release, are seen by your customers, and your business suffers. Like what happened to Amazon during its Prime Day sale last summer.

So, we here at Applitools want to help. We built a visual AI to help you automatically test and monitor your mobile and web apps. This way, you can be sure they appear correctly across all the devices, operating systems, browsers, and screen sizes your customers use.

We’ve been busy

Since receiving this honor last year, we’ve been busy building out our platform, making life easier for developers and QA teams. Back in April, we released the world’s first UI Version Control system that lets you view the history of an application UI over time, and see how it has changed, what has been changed, and by whom.

We also created an ultrafast visual testing grid that lets you test for visual bugs in dozens of combinations of browsers, screen sizes, and mobile device orientations — in parallel, in seconds.

And we released a new root cause analysis feature, letting you pinpoint the cause of bugs in application code within minutes, not hours.

Our goal with all of these releases? To eliminate hours from traditional bug diagnosis practices, and help you keep your software projects and digital transformation initiatives on schedule and looking great.

Helping QA professionals grow

However, it’s not just about new features. Applitools recently launched Test Automation University, to provide educational training courses to help improve test QA automation skill sets for all. Provided free of charge, these online courses help address the lack of accessibility to the training and educational resources needed to fill a growing skills gap in IT, and are vetted by some of the leading test automation experts in the world.

Make sure to stay tuned in the coming year for some more exciting announcements, updates to our solutions and the latest trends in test automation. And, thanks again to the CB Insights team for the honor of being added to the AI 100 list!

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

How are you looking to use AI to improve your product development and test automation? 

The post Applitools Named to CB Insights’ AI 100 List for the Second Year in a Row appeared first on Automated Visual Testing | Applitools.

I’m pleased to announce that Applitools has released Root Cause Analysis, or RCA for short. This new offering allows you to instantly pinpoint the root cause of visual bugs in your front-end code. I’d like to explain why RCA matters to you, and how it’ll help you in your work.

https://dilbert.com/strip/2015-04-24

Well, maybe RCA doesn’t find THE root cause. After all, all software bugs are created by people, as the Dilbert cartoon above points out.

But when you’re fixing visual bugs in your web apps, you need a bit more information than what Dilbert is presenting above.

The myriad challenges of front-end debugging

What we’ve seen in our experience is that, when you find a bug in your front-end UI, you need to answer the question: what has changed?

More specifically: what are the differences in your application’s Document Object Model (or DOM for short) and Cascading Style Sheet (CSS) rules that underpin the visual differences in my app?

This isn’t always easy to determine.

Getting the DOM and CSS rules for the current version of your app is trivial. They’re right there in the app you’re testing.

But getting the baseline DOM and CSS rules can be hard. You need access to your source code management system. Then you need to fire up the baseline version of your app. This might involve running some build process, which might take a while.

Once your app builds, you then need to get it into exactly the right state, which might be challenging.

Only then can you grab your baseline DOM and CSS rules, so you can run your diffs.

But doing a simple diff of DOM and CSS rules will turn up many differences, many of them have nothing to do with your visual bug. So you’ll chase dead-end leads.

That’s a tedious, time-consuming process.

Meanwhile, if you release multiple times per week or per day, you have less time and more pressure to fix the bug before the next release.

This is pretty darn stressful.

And this is where Applitools RCA comes to the rescue!

AI-assisted bug diagnosis

With Applitools RCA, we’ve updated our SDKs to grab not just UI screenshots — as we always have — but also DOM and CSS rules. We send this entire payload to our test cloud, where we now perform an additional step.

First, our AI finds significant visual differences between screenshots, as it always has, while ignoring minor differences that your users won’t care about (also called false positives).

Then — and this is the new step with RCA — we find what DOM and CSS rules underpin those visual differences. Rather than digging through line after line of DOM and CSS rules, you’ll now only be shown the lines responsible for the difference in question.

We display those visual differences to you in Applitools Eyes Test Manager. You click on a visual difference highlighted in pink and instantly see what DOM and CSS rules are related to that change.

This diagram explains this entire process:

Even better, we give you a link to the exact view you’re looking at — sort of like a blog post’s permalink, which you can add to your Jira bug report, Slack, or email. That way your teammates can instantly see what you’re looking at. Everyone gets on the same page, and bugs get fixed faster.

Here’s a summary of life before and after RCA:

How much would RCA speed up your debugging process?

Making Shift Left Suck Less

If you’re in an organization is that is implementing Shift Left, you know that it’s all about giving developers the responsibility of testing their own code. Test earlier, and test more often, on a codebase you’re familiar with and can quickly fix.

And yes, there’s something to be said for that. But let’s face it: if you’re a developer doing Shift Left, what this means is you have a bunch of QA-related tasks added to your already overflowing plate. You need to build tests, run tests, maintain tests.

We can’t make the pain of testing go away. But with Applitools RCA, we can save you a lot of time and help you focus on writing code!

We intentionally designed RCA to look like the developer tools you use every day. Our DOM diffs look like your Google Chrome Dev Tools, and our CSS diffs look like your GitHub diffs.

All this means you have more time to build features, which is probably the part of your job you like to focus on.

ROI, Multiplied for R&D

This section is for the engineering managers, directors, and VPs.

Applitools RCA lets your team spend more time on building new features. It helps your R&D team be more productive, efficient, and happy!

It’s application features that move modern businesses forward. And RCA helps your team get bug fixing out of the way so they can focus on adding value to your company, and get kudos for adding more features to delight your customers.

So, RCA is good for your developers, for your business, but also for your CFO! Here’s a quick back-of-the-envelope you can share:

Let’s say you have 100 developers on your engineering team. How much money would you save if RCA can accelerate your development by 10%? The quick calculation shows: maybe $2m per year? Maybe more? That’s tons of money!

UI Version Control, Evolved

Applitools RCA helps your product managers too!

With RCA, our user interface version control now includes the DOM and CSS associated with each screenshot.

This means that not only can you see how the visual appearance of your web application has evolved over time, but also how its underlying DOM and CSS have changed. This makes it easier for you to roll back new features that turned out to be a bad idea since they hurt the user experience or decreased the revenue.

You Win Big

Applitools Root Cause Analysis is a major step in the evolution of test automation because, for the first time, a testing product isn’t just finding bugs; it’s telling you how to fix the bugs.

The evolution of software monitoring tools demonstrates a similar pattern. Early monitoring tools would find an outage, but wouldn’t point you in any direction of fixing the underlying problem behind the outage.

Modern monitoring tools like New Relic or AppDynamics, on the other hand, would point you to the piece of code causing the outage: the root cause. The market spoke, and it chose monitoring tools that pointed users to the root cause.

In test automation, we’re where monitoring was ten years ago. Existing tools like Selenium, StormRunner, Cypress, and SmartBear are good and finding problems, but they don’t help you discover and fix the root cause.

Applitools RCA, like New Relic and AppDynamics, helps you instantly find the root cause of a bug. But unlike those tools, Applitools RCA doesn’t force you to rip-and-replace your existing test automation tools. It integrates with Selenium, Cypress, WebdriverIO, and Storybook, allowing you to make your existing testing much more powerful by adding root cause analysis.

See for yourself

To see Applitools RCA in action, please watch this short demo video:

Start Using Applitools Root Cause Analysis Today!

If you’re not yet using Applitools Eyes, sign up for a free account.

If you’re an existing Applitools customer, a Free Trial of Applitools Root Cause Analysis is already provisioned in your account. To learn more about how to use it, see this documentation page.

A free trial of Applitools RCA is available until the end of February 2019. After that, it will be available for an additional fee.

The post Applitools Root Cause Analysis: Found a Bug? We’ll Help You Fix It! appeared first on Automated Visual Testing | Applitools.

Many people think of GitHub as a niche website that hosts open-source projects. In June 2018, that niche website with a mere 28 million users got more media attention than Apple’s Worldwide Developer Conference (WWDC). Somehow, a site that provides free services was so valuable that Microsoft acquired them for $7.5 billion US dollars.

Since its launch in 2007, GitHub has grown far beyond its roots as a web-based service that provides a hosted version of the Git-distributed version control system. The site hosts open-source projects and also offers paid plans for developers and companies. With the adoption of agile development and DevOps practices, it has become an essential part of the continuous integration/delivery/deployment pipeline for many companies. Not only do these companies host their source code in private GitHub repositories, but they also integrate their repositories with their continuous integration (CI) systems.

An important part of any CI system is the ability to run automated tests on any code committed to a source control repository. The automated tests help prevent new code from breaking the software built by the CI system. Now, with the release of the Applitools Eyes GitHub integration, you can include automated tests that perform visual UI testing as part of your build process. Visual UI testing lets you create tests that accurately simulate real users interacting with your software. The integration also takes care of uploading test results to Applitools’ advanced Automated Visual Management (AVM) system for analysis.

This post explains how you can use the Applitools Eyes GitHub integration to perform visual validations for pull requests. Integrating Applitools Eyes with GitHub prevents visual defects from merging into your production software. For each pull request, it merges the baseline screenshots of the branch along with your source code, and runs automated visual validation tests.

Applitools Eyes GitHub Integration

Applitools Eyes ensures your application looks and functions exactly as expected across all browsers, operating systems, devices, and screen resolutions. In addition to handling application testing, the Applitools Eyes GitHub integration lets you use Applitools Eyes as an integral part of your build automation process.

The Applitools Eyes GitHub integration automatically configures the baselines used for obtaining and saving test baselines. It displays the status of your tests directly in the pull request page, right next to the merge status. The integration lets you view the history of an application’s user interface (UI) and see how it has changed, what has been changed, and by whom. This will show you how your product’s UI has evolved over the product’s lifetime. The feature allows you to merge visual testing baseline branches in the same way you merge changes to your test and source code. For each branch in your GitHub project repository, you can see the history of all your test baselines, compare them to prior baseline versions, and if necessary, revert to an earlier version. This feature lets you reject and/or revert to any baseline you modified by accident or by design.

Visually Validating a Pull Request

In order to illustrate how to use the Applitools Eyes GitHub integration, we created a complete demo that includes all the relevant source code files and project-related assets. The demo is called merge-demo, and we made it available as a GitHub repository. The demo runs visual validation against the Applitools Hello World test page. The test navigates to the Hello World page, visually validates it, clicks the Click Me button, and visually validates the page. In this demo, the Hello World test code has been modified to test the page in two different viewport sizes.

Configuring the Integration

Before you can run the demo, you need to install and configure the Applitools Integration application. In the merge-demo repo, open the Settings > Integration & Services page. The page indicates that the Applitools Integration application has been installed. The page also indicates that the Travis CI service has been configured to build and run tests.

Creating the Pull Request

To create the pull request on your PC, open a terminal window and clone the merge-demo locally. Create a new branch, called mybranch, and modify the tests. Commit the changes and push them back to the remote branch. Now, create a pull request to merge the changes with the master branch.

Building the Request

After receiving the Pull Request, Travis CI initiates a build for the pushed pull request. For the sake of simplicity, we recommend you disable the Builds for the Pushed Branches option; otherwise, Travis CI will build two individual branches.

When the build is complete, open the Pull Request page. The Test Applitools status indicates the status of all visual tests that ran as part of the build. We can see the two tests detected differences. If you click the details link, it will take you to the corresponding page in the Applitools Test Manager. The batch contains two tests with different viewport sizes. The test name includes the branch name, the name of the pull request, the source branch, the commit message, and the SHA. Using the page, you can accept the changes made to the first test and reject the changes made to the second test.

Managing and Merging Baselines

Reopen the Pull Request page and you will see the Status Description now indicates that one of the visual tests has been failed. If you accept the changes in the second test, you can see the status changes to green to indicate that all of the visual tests were passed. By looking in the Test Details, you can see the tests were configured to run in the source branch. After reviewing the test results, you save them to the baselines of the source branch. The updated baselines will not affect any tests running in other branches.

After updating your baselines, you can check that other team members did not make conflicting changes to the baseline. You can do this by checking the SCM status of the Applitools request, which is currently pending. Next, open the Pull Request page. This page displays the details of all the changes in the baselines, compares them, and allows you to edit the source baseline by porting ignore regions and locations from the target baseline, or by adding new ones. The page shows that the Applitools SCM status changed to green and no baseline conflicts were found. This means the two changes in the source branch are ready to be merged with the target branch. As soon as your source code is fully merged, Applitools will automatically merge the changed baseline in the source branch with the baseline of the target branch.

Visual UI Version Control

In a very short time, GitHub has grown from an obscure developer-centric website into an essential tool for a wide range of enterprises. It serves the needs of freelance contractors, large corporations, and governments. GitHub may have started as a cloud-based version control system, but today, it provides additional services such as project management, online documentation, web hosting, and defect tracking. For many companies, GitHub has replaced their on-premises source control management systems, and it functions as an essential piece of their CD/CI tooling.

In this post, we looked at how to use GitHub as part of your DevOps stack. We explained how the Applitools Eyes GitHub integration automatically configures the baselines used for obtaining and saving test baselines, how to create pull requests, and how you can add visual test validation to your CI pipeline.

This solution gives you powerful tools that ensure each time a pull request triggers a build and merges your target and source branches, it also gives you control and insight into the process. The integration also simplifies the process of reviewing and managing visual test baselines. More importantly, it is a form of UI version control that shows how your application’s UI has changed and evolved over time. These Applitools Eyes GitHub integration benefits will ensure the quality of your products and will benefit your end users and customers.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post How to Test Your GitHub Pull Requests so You Don’t Break Your UI appeared first on Automated Visual Testing | Applitools.