We are excited to share some fantastic news with our valued customers and the broader testing community. Applitools has acquired Preflight, a pioneering no-code platform that streamlines the creation, execution, and management of complex end-to-end tests. This acquisition marks a significant step in our journey to provide you with breakthrough technology that empowers your teams to increase test coverage, reduce test execution time, and deliver superior applications that your customers will love.

Introducing Applitools Preflight

Preflight is a robust no-code testing tool that empowers teams of all skill levels to automate complex testing scenarios. It runs these tests at an impressive scale across various browsers and screen sizes. Preflight’s user-friendly web recorder captures every element accurately and includes a data generator to simulate even the most complex test cases. This is a game-changer for manual testers, QA engineers, and product teams as it empowers them to automate test scenarios regardless of their skillset, effectively multiplying their QA abilities instantly.

Preflight ensures businesses achieve the test coverage necessary to consistently delight customers with each new experience, all without writing a single line of code.

The Benefits of Preflight

Simplified Test Creation: With Preflight, anyone on the team can create and run tests, democratizing the testing process. This inclusivity leads to more thorough testing and faster feedback cycles.

Expanded Test Coverage: Preflight enables teams to create comprehensive test suites that cover more functionality in less time. It can easily create UI tests, API tests, verify emails during sign-up, generate synthetic data, and more. This means teams can test more scenarios and edge cases that may have been overlooked with manual testing or traditional automated testing.

Enhanced Maintainability and Reusability: Preflight allows customers to reuse sections of test suites, workflows, login profiles, data, and more across different tests, reducing redundancy. It also simplifies test maintenance with a powerful test editor and live test replay that makes editing tests fast and intuitive, reducing one of the biggest gripes of record-and-replay tools.

The Future of Applitools and Preflight

While Preflight will continue to be available as a standalone product, we are actively integrating it into the Applitools platform to bring Visual AI to the masses! To get an exclusive first look at Preflight today, we invite you to sign up for a demo with one of our engineers.

The post Welcome Preflight To The Applitools Family appeared first on Automated Visual Testing | Applitools.

Introducing Applitools Execution Cloud: The World’s First Self-Healing Test Infrastructure for Open-Source Test Frameworks

We are excited to announce the launch of Applitools Execution Cloud, a revolutionary self-healing, cloud-based testing platform that enables teams to run their existing tests against AI-powered testing infrastructure. This new addition to the Applitools Ultrafast Test platform is designed to provide teams that use open-source frameworks like Selenium or WebDriver.io, with best-in-class AI capabilities, such as self-healing, that are only currently available in proprietary tools.

For years, Applitools Eyes has brought Visual AI to the validation portion of tests, helping engineers reduce assertion code while boosting test coverage. While Eyes has continued to grow as the industry leader in AI validation, we were able to work closely with our customers to help solve problem with the other portion of testing: interaction.

Test flakiness most often occurs during the interaction phase of tests – and more specifically when a test uses a locator as it’s anchor for navigation that has changed for some reason. This can be due to dynamic Class or ID generation on certain builds or just some changes to the framework from the dev team. Either situation can wreak havoc on tests running soundly.

Reduce test flakiness with Applitools Execution Cloud

With Execution Cloud, teams can run tests at infinite scale in parallel while quickly healing broken tests as they run, reducing flakiness and execution time. Small changes to the UI, like text, color, or slight layout changes that would normally fail a Selenium test will be able to heal themselves.

The platform also allows for testing at extreme scale, allowing teams to run tests in the cloud in parallel for faster CI/CD pipelines.

And remember Execution Cloud, teams can easily run both functional and visual tests, as well as any Selenium and WebDriver.io tests using any binding language.

The platform also features implicit waits, which automatically waits for all critical elements to load before running its next process, drastically reducing test flakiness. Furthermore, teams can access test logs, including video, command logs, and console browser logs, to help debug faster.

Unlike its competitors, Execution Cloud is the world’s first intelligent test infrastructure for running open-source test frameworks. It is not locked in with a specific test creation tool, and it operates on a pay-as-you-go model that is cost-effective for developers and test engineers. Additionally, Execution Cloud is designed with AI capabilities, which its open-source competitors lack, making it the smart choice for teams that want to accelerate their product delivery speed and improve testing resilience.

Get started today

Overall, Applitools Execution Cloud offers a complete testing solution that helps teams improve their testing process and accelerate their product delivery speed. Run faster, more resilient Selenium tests with the Applitools Self-Healing Execution Cloud. Try it today and see the difference for yourself!

Learn more about Applitools Execution Cloud in our upcoming webinar.

The post Introducing Applitools Execution Cloud Self-Healing Test Infrastructure appeared first on Automated Visual Testing | Applitools.

The challenges of building UI

Modern UIs today are built by a diverse set of teams that work together at different parts of the process. The pace at which these design, development, QA, operations, marketing, and product teams ship their work is continuing to accelerate – creating new challenges around communication, collaboration, and validation across the workflow.

Getting from design mock-ups in Figma to live UI is a process that includes a lot of feedback and testing. It starts with the designer who passes to the product manager for approval before the developer can start building. Feedback in the development process requires rework to make those updates before it can get approval from the product manager. This is all before the testing team has even started their review.

You can see the game of telephone that’s played through different stakeholders into production, and we get something that’s slightly different at multiple levels. This makes measuring what actually happened and what actually needs to change incredibly hard, making it a huge burden on teams to ship clean UIs at a fast pace. Some of our main challenges here are:

• Lack of communication between the growing group of stakeholders
• Breadth of technology during implementation causing inconsistencies
• No continued source of truth across tooling as the app UI evolves

The post Announcing Applitools Centra: UI Validation From Design to Implementation appeared first on Automated Visual Testing | Applitools.

Last year, Applitools launched the Ultrafast Grid, the next generation of browser testing clouds for faster testing across multiple browsers in parallel. The success of the new grid with our customer base has been nothing short of amazing, having over 200 customers using Ultrafast Grid in the last year. But our customers are hungry for more innovation and we wanted to focus on extending our Applitools Test Cloud to the next frontier: native mobile apps.

Today, Applitools is excited to announce that the Native Mobile Grid is now ready for general availability – giving companies’ engineering and QA teams access to  the next generation of cross-device testing.

For those developing native mobile apps, there are often many challenges with testing across multiple devices and orientations, resulting in a high number of bugs slipping into production. Local devices are hard to set up and owning a vast collection don’t work well across remote companies in a post-Covid world. Not to mention each different device takes a bit of custom configuration and wizardry to get running without flakiness. And mobile test frameworks are often flaky on the big cloud providers. 

Applitools Native Mobile Grid is a cloud based testing grid that allows testers and developers to automate testing of their mobile applications across different iOS and Android devices quickly, accurately, and without hassle. After running just one test locally, the Applitools Native Mobile Grid will asynchronously run the tests in parallel using Visual AI, speeding up total execution tremendously and reducing flakiness. We’ve seen test time reduce by over 80% when run against other popular testing clouds.

The Benefits Of The Native Mobile Grid

Faster Test Execution, Broad Coverage

With access to over 40 devices, Applitools revolutionary async parallel test execution can reduce testing time by up to 90% compared to traditional device clouds while still expanding coverage over that single device you’ve been testing with.

Less Test Flakiness

Visual AI helps power Applitools industry leading stability and reliability, with flakiness and false positives reduced by 99%.

More Bugs Caught

Testing faster, on more devices, with Visual AI means that more bugs & defects are caught without having to write more tests.

Added Security

The Native Mobile Grid does not need to open a tunnel into your network so your application stays safe and secure

Get Started

To get started with Native Mobile Grid, just head on over and fill out this form.

The post Introducing Applitools Native Mobile Grid appeared first on Automated Visual Testing | Applitools.

Native mobile testing can be slow and error-prone with questionable ROI. With Ultrafast Test Cloud for Native Mobile, you can now leverage Applitools Visual AI to test native mobile apps with stability, speed, and security – in parallel across dozens of devices. The new offer extends the innovation of Ultrafast Cloud beyond browsers and into the mobile applications.

You can sign up for the early access program today!

The Challenge of Testing Native Mobile Apps

Mobile testing has a long and difficult history. Many industry-standard tools and solutions have struggled with the challenge of testing across an extremely wide range of devices, viewports and operating systems.

The approach currently in use by much of the industry today is to utilize a lab made up of emulators, or simulators, or even large farms of real devices. Then the tests must be run on every device independently. The process is not only costly, slow and insecure, but it is prone to errors as well.

At Applitools, we had already developed technology to solve a similar problem for web testing, and we were determined to solve this issue for mobile testing too.

Announcing the Ultrafast Test Cloud for Native Mobile

Today, we are introducing the Ultrafast Test Cloud for Native Mobile. We built on the success of the Ultrafast Test Cloud Platform, which is already being used to boost the performance and quality of responsive web testing by 150 of the world’s top brands. The Ultrafast Test Cloud for Native Mobile allows teams to run automated tests on native mobile apps on a single device, and instantly render it across any desired combination of devices.

“This is the first meaningful evolution of how to test native mobile apps for the software industry in a long time,” said Gil Sever, CEO and co-founder of Applitools. “People are increasingly going to mobile for everything. One major area of improvement needed in delivering better mobile apps faster, is centered around QA and testing. We’re building upon the success of Visual AI and the Ultrafast Test Cloud to make the delivery and execution of tests for native mobile apps more consistent and faster than ever, and at a fraction of the cost.”

The Power of Visual AI and Ultrafast Test Grid

Last year we introduced our Ultrafast Test Grid, enabling teams to test for the web and responsive web applications against all combinations of browsers, devices and viewports with blazing speed. We’ve seen how some of the largest companies in the world have used the power of Visual AI and the Ultrafast Test Grid to execute their visual and functional tests more rapidly and reliably on the web.

We’re excited to now be able to offer the same speed and agility, and security for native mobile applications. If you’re familiar with our current Ultrafast Test Grid offering, you’ll find the experience a familiar one.

Mobile Apps Are an Increasingly Critical Channel

Mobile usage continues to rise globally, and more and more critical activity – from discovery to research and purchase – is taking place online via mobile devices. Consumers are demanding higher and higher quality mobile experiences, and a poorly functioning site or visual bugs can detract significantly from the user’s experience. There is a growing portion of your audience you can only convert with a five-star quality app experience.

While testing has traditionally been challenging on mobile, the Ultrafast Test Cloud for Native Mobile increases your ability to test quickly, early and often. That means you can develop a superior mobile experience at less cost than the competition, and stand out from the crowd.

Get Early Access

With this announcement, we’re also launching our free early access program, with access to be granted on a limited basis at first. Prioritization will be given to those who register early. To learn more, visit the link below.

The post Introducing the Next Generation of Native Mobile Test Automation appeared first on Automated Visual Testing | Applitools.

Applitools finds functional & visual bugs for the world’s largest organizations; Rally® is leveraged by the largest enterprises to manage their agile development at scale, including quality management and the tracking of bugs. We collaborated to design a new integration to make this process faster and provide developers with the in-depth details they need to reproduce and triage bugs faster than ever before. 

Rally & Applitools

Rally Software is an enterprise-class platform that is purpose built for scaling agile development practices. Rally serves as a central hub for teams to collaboratively plan, prioritize and track work, from strategy all the way down to stories and bugs through the entire SDLC.

Applitools is building the next generation of test automation platform for cross browser and device testing powered by AI assisted computer vision technology known as Visual AI. Visual AI helps Developers, Test Automation Engineers and QA professionals release high-quality web and mobile apps enabling CI/CD.

Better Together – A Transformative Workflow

Applitools & Rally share enterprise customers. Like all modern businesses, our shared customers are looking for ways to gain an edge over their competition – and oftentimes that edge is gained by improved workflows and automation that saves time and allows teams to release better quality software faster. It’s a never ending battle to improve time-to-market, and for our shared customers, we have just added an integration that can do just this.

The integration allows users to log a bug in Rally without ever leaving the Applitools Eyes user interface and workflow. This means no context shifting, no copy / paste, no additional logins or lost browser tabs. It’s designed for both workflow efficiency and to ensure that every detail gets logged so developers can have everything they need to triage the bug and move on. It’s not only developers and testers who benefit from this integration, now any stakeholder in the entire appdev process from product managers to UX designers can automatically pinpoint the exact cause of issues using the Applitools Root Cause Analysis capabilities, send the results instantly to Rally and have everything needed to reproduce and fix defects on the spot.  

Setting up the Rally Integration

Your Eyes admin can setup the Rally integration via the Admin screen →  Integrations tab. The process takes less than a minute. First you’ll enter their Rally server & API key – then a run through a quick authentication process requiring a Client ID & Client Secret.

Once authenticated, all Rally projects will be available to choose from, so simply choose the first project you will link to Applitools along with the default work item type for issues created from within Eyes (most likely it will be ‘defects’). You can optionally add one or more defined fields by simply choosing the field name and and the default value – repeat as needed. For detailed setup instructions, please visit our Rally integration documentation.

Automating the Defect Feedback Loop

The beauty of this integration is in its simplicity. When a bug is found by Applitools, Users can gather all relevant information including screenshot and full steps to reproduce with a single region-drag and click using the bug region feature. The detailed information is instantly sent to Rally for teams to immediately begin triage.

Applitools Integration Ecosystem

The Rally integration is a continuation of our commitment to extending the Applitools integration ecosystem and fitting seamlessly into customers’ existing workflows and tools. Applitools customers now benefit from seamless integrations with over 60 SDKs for you to choose from. This includes:

• Testing frameworks and languages such as Selenium, Cypress, Appium, and more
• Source control solutions like GitHub, GitLab and BitBucket
• CI\CD platforms like Jenkins, TeamCity and Travis
• Collaboration tools like Slack
• Agile planning and defect tracking tools like Jira

Now, with the Eyes 10.11 release, we’re excited to also include Rally on this list.

The post Transforming Software Development through a Modern Feedback Loop appeared first on Automated Visual Testing | Applitools.

How much is it worth to catch more bugs early in your product release process? Depending on where you are in your release process, you might be writing unit or systems tests. But, you need to run end-to-end tests to prove behavior, and quality engineers require a high degree of skill to write end-to-end tests successfully.

What would you say if a single validation engine could help you ensure data integrity, functional integrity, and graphical integrity in your web and mobile applications? And, as a result, catch more bugs earlier in your release process?

Catch Bugs or Die

Let’s start with the dirty truth: all software has bugs. Your desire to create bug-free code conflicts with the reality that you often lack the tools to uncover all the bugs until someone finds them way late in the product delivery process. Like, say, the customer.

With all the potential failure modes you design for – and then test against – you begin to realize that not all failure modes are created equal. You might even have your own triage list:

• Security & penetration
• Data integrity and consistency
• Functional integrity and consistency

So, where does graphical integrity and consistency fit on your list? For many of your peers, graphical integrity might not even show up on their list. They might consider graphical integrity as managing cosmetic issues. Not a big deal.

Lots of us don’t have reliable tools to validate graphical integrity. We rely on our initial unit tests, systems tests, and end-to-end tests to uncover graphical issues – and we think that they’re solved once they’re caught. Realistically, though, any application evolution process introduces changes that can introduce bugs – including graphical bugs. But, who has an automation system to do visual validation with a high degree of accuracy?

Tradeoffs In End-to-End Testing

Your web and mobile apps behave at several levels. The level that matters to your users, though, happens at the user interface on the browser or the device. Your server code, database code, and UI code turns into this representation of visual elements with some kind of visual cursor that moves across a plane (or keyboard equivalent) to settle on different elements. The end-to-end test exercises all the levels of your code, and you can use it to validate the integrity of your code.

So, why don’t people think to run more of these end-to-end tests? You know the answers.

First, end-to-end tests run more slowly. Page rendering takes time – your test code needs to manipulate the browser or your mobile app, execute an HTTP request, receive an HTTP response, and render the received HTML, CSS, and JavaScript. Even if you run tests in parallel, they’re slower than unit or system tests.

Second, it takes a lot of effort to write good end-to-end tests. Your tests must exercise the application properly. You develop data and logic pre-conditions for each test so it can be run independently of others. And, you build test automation.

Third, you need two kinds of automation. You need a controller that allows you to control your app by entering data and clicking buttons in the user interface. And, most importantly, you need a validation engine that can capture your output conditions and match those with the ones a user would expect.

You can choose among many controllers for browsers or mobile devices. Still, why do your peers still write code that effectively spot-checks the DOM? Why not use a visual validation engine that can catch more bugs?

Visual AI For Code Integrity

You have peers who continue to rely on coded assertions to spot-check the DOM. Then you have the 288 of your peers who did something different: they participated in the Applitools Visual AI Rockstar Hackathon. And they got to experience first-hand the value of Visual AI for building and maintaining end-to-end tests.

As I wrote previously, we gave participants five different test cases, asked them to write conventional tests for those cases, and then to write test cases using Applitools Visual AI. For each submission, we checked the conditions each test writer covered, as well as the failing output behaviors each test-writer caught.

As a refresher, we chose five cases that one might encounter in any application:

1. Comparing two web pages
2. Data-driven verification of a function
3. Sorting a table
4. Testing a bar chart
5. Handling dynamic web content

For these test cases, we discovered that the typical engineer writing conventional tests to spot-check the DOM spent the bulk of their time writing assertion code. Unfortunately, the typical spot-check assertions missed failure modes. The typical submission got about 65% coverage. Alternatively, the engineers who wrote the tests that provided the highest coverage spent about 50% more time writing tests.

However, when using Visual AI for visual validation, two good things happened. First, everyone spent way less time writing test code. The typical engineer went from 7 hours of coding tests and assertions to about 1.2 hours of coding tests and Visual AI. Second, the average test coverage jumped from 65% to 95%. So, simultaneously, engineers took less time and got more coverage.

Visual AI Helps You Catch More Bugs

When you find more bugs, more quickly, with less effort, that’s significant to your quality engineering efforts. You’re able to validate data, functional, and graphical by focusing on the end-to-end test cases you run. You spend less time thinking about and maintaining all the assertion code checking the result of each test case.

Using Visual AI makes you more effective? How much more effective? Based on the data we reviewed – you catch 45% of your bugs earlier in your release process (and, importantly, before they reach customers).

We have previously written about some of the other benefits that engineers get when using Visual AI, including:

• 5.8x Faster Test Creation – Authoring new tests is vital especially for new features during a release cycle. Less time authoring means more time managing quality. Read more.
• 5.9x More Test Code Efficient – Like your team’s feature code, test code efficiency means you write less code, yet provide far more coverage. Sounds impossible, right? It’s not. Read More.
• 3.8x Improvement In Test Stability – Code-based frameworks rely on brittle locators and labels that break routinely. This maintenance kills your release velocity and reduces coverage. What you need is self-maintaining and self-healing code that eliminates most of the maintenance. It sounds amazing and it is! Read More.

By comparing and contrasting the top participants – the prize winners – with the average engineer who participated in the Hackathon, we learned how Visual AI helped the average engineer greatly – and the top engineers become much more efficient.

The bottom line with Visual AI — you will catch more bugs earlier than you do today.        

More About The Applitools Visual AI Rockstar Hackathon

Applitools ran the Applitools Visual AI Rockstar Hackathon in November 2019. Any engineer could participate, and 3,000 did so from around the world. 288 people actually completed the Hackathon and submitted code. Their submissions became the basis for this article.

You can read the full report we wrote: The Impact of Visual AI on Test Automation.

In creating the report, we looked at three groups of quality engineers including:

• All 288 Submitters – This includes any quality engineer that successfully completed the hackathon project. While over 3,000 quality engineers signed-up to participate, this group of 288 people is the foundation for the report and amounted to 3,168 hours, or 80 weeks, or 1.5 years of quality engineering data.
• Top 100 Winners – To gather the data and engage the community, we created the Visual AI Rockstar Hackathon. The top 100 quality engineers who secured the highest point total for their ability to provide test coverage on all use cases and successfully catch potential bugs won over $40,000 in prizes.
• Grand Prize Winners – This group of 10 quality engineers scored the highest representing the gold standard of test automation effort.

By comparing and contrasting the time, effort, and effectiveness of these groups, we were able to draw some interesting conclusions about the value of Visual AI in speeding test-writing, increasing test coverage, increasing test code stability, and reducing test maintenance costs.

What’s Next?

You now know five of the core benefits we calculate from engineers who use Visual AI.

• Spend less time writing tests
• Write fewer lines of test code
• Maintain fewer lines of test code
• Your test code remains much more stable
• Catch more bugs

So, what’s stopping you from trying out Visual AI for your application delivery process? Applitools lets you set up a free Applitools account and start using Visual AI on your own.  You can download the white paper and read about how Visual AI improved the efficiency of your peers. And, you can check out the Applitools tutorials to see how Applitools might help your preferred test framework and work with your favorite test programming language.

Cover Photo by michael podger on Unsplash

The post How Do You Catch More Bugs In Your End-To-End Tests? appeared first on Automated Visual Testing | Applitools.

When you have multiple variations of your app, how do you automate the process to validate each variation?

A/B testing is a technique used to compare multiple experimental variations of the same application to determine which one is more effective with users. You typically run A/B tests to get statistically valid measures of effectiveness. But, do you know why one version is better than the other? It could be that one contains a defect.

Let’s say we have two variations – Variation A and Variation B; and Variation B did much better than Variation A. We’d assume that’s because our users really liked Variation B.

But what if Variation A had a serious bug that prevented many users from converting?

The problem is that many teams don’t automate tests to validate multiple variations because it’s “throw away” code. You’re not entirely sure which variation you’ll get each time the test runs.

And if you did write test automation, you may need a bunch of conditional logic in your test code to handle both variations.

What if instead of writing and maintaining all of this code, you used visual testing instead? Would that make things easier?

Yes, it certainly would! You could write a single test, and instead of coding all of the differences between the two variations, you could simply do the visual check and provide photos of both variations.  That way, if either one of the variations comes up and there are no bugs, the test will pass. Visual testing simplifies the task of validating multiple variations of your application.

Let’s try this on a real site.

Here’s a website that has two variations.

There are differences in color as well as structure. If we wanted to automate this using visual testing, we could do so and cover both variations. Let’s look at the code.

I have one test here which is using Applitools Eyes to handle the A/B test variations.

• On line 27, I open Eyes just as I normally would do
• Because the page is pretty long, I make a call to capture the full page screenshot on line 28
• There’s also a sticky header that is visible even when scrolling, so to avoid that being captured in our baseline image when scrolling, I set the stitch mode on line 29
• Then, the magic happens on line 30 with the checkWindow call which will take the screenshot
• Finally, I close Eyes on line 31

After running this test, the baseline image (which is Variation B) is saved in the Applitools dashboard. However, if I run this again, chances are that Variation A will be displayed, and in that event my visual check will fail because the site looks different.

Setting Up the Variation

In the dashboard, we see the failure which is comparing Variation A with Variation B. We want to tell Applitools that both of these are valid options.

To do so, I click the A/B button which will open the Variations Gallery. From here, I click the Create New button.

After clicking, the Create New button, I’m prompted to name this new variation and then it is automatically saved in the Variation Gallery. Also, the test is now marked as passed. Now in future regression runs, if either Variation A or Variation B appears (without bugs), the test will still pass.

Another thing we can do is rename the variations. Notice the original variation is named Default. By hovering over the variation, we see an option to rename it to Variable B, for example.

I can also delete a variation. So, if my team decides to remove one of the variations from the product I can simply delete it from the Variation Gallery as well.

See It In Action!

Check out the video below to see A/B baseline variations in action.

For More Information

• Read all about Functional Test Myopia
• Blog: How Do I Validate Visually?
• Blog: Shifts Happen – Inspect Just The Changes That Matter
• Visit the Automated Visual Testing Course on Test Automation University
• Sign up for a free Applitools account
• Request an Applitools Demo
• Visit the Applitools Tutorials

The post A/B Testing: Validating Multiple Variations appeared first on Automated Visual Testing | Applitools.

Whether you’re a test automation engineer or a manual tester, it’s crucial to track what your test environment is. This enables your developers to easily replicate bugs found and avoids the frustration of not being able to reproduce those problems.

It’s also important to track your web or mobile app user interface history using a UI version control system. This helps your developers, product managers, and designers understand what’s been tried, what’s worked, and what hasn’t worked so that the entire team can iterate the UI rapidly, in an Agile fashion, to a product that works well for customers.

As part of our continued rollout of Application Visual Management (AVM), in Applitools Eyes version 10.4, we’ve made it easier to manage your visual UI test environment setup, as well as your web application and mobile app user interface history. Let’s dive into these new features:

1. Visual Test Environments view
2. Visual Test History view
3. GitHub Enterprise integration
4. Batch & Branch management
5. Table Configuration manager
6. API Update

Visual Test Environments View

When you run cross-device and cross-browser visual UI tests, you need to indicate the details of the different baseline that you want to run against. One way of doing this is to provide the name of the baseline test execution environment. To recap, an Applitools test environment denotes a particular combination of operating system, browser, and viewport size.

Previously, getting the name of the baseline environment wasn’t as easy as we wanted it to be. So in Applitools Eyes 10.4, we’ve made it a lot more straightforward, with a new page, Environment, which lets you:

• View all the test execution environments for your team
• Add one or more names to any environment
• Delete test environments that are no longer relevant to you

You can also group environments by operating system, browser, viewport, or environment name, which makes it easier to manage environments. For example, if you no longer support Windows 7, this makes it easier to hide all test environments related to Windows 7 so that they don’t clutter up your workflow.

Visual Test History View

The new Visual Test History view lets you drill into the timestamp and status of any application test run across all the browsers, viewports, OSs, and branches that comprise your team’s test environment configuration. It’s like a Wayback Machine for your visual UI tests, but with the added help of a whole bunch of meta-data about what changed, who changed, when they changed it, etc.

On this view, the test status is also shown, with the option to drill into the test results page to view a test’s batch group and evaluate the visual differences between your baseline and a particular test run.

You can also sort, filter, or group your visual tests by various parameters, and delete any previous tests where necessary. This is especially helpful once you have months or even years of test runs. You can filter by a wide range of test run attributes, including:

• Assigned To
• Branch
• Browser
• Issues & Remarks
• Operating System
• Regions
• Saved status
• Started status
• Step status
• Test status

For instance, you can see all visual test runs occurring on Android 5.1 or Android 6.0 that have failed or have unresolved differences. With this granular level of filtering, you can zero in on exactly the test runs you’re looking for.

The Visual Test History view can be navigated from the Apps & Test or the Test Results pages when looking at a specific test result. See below on how to find this command.

GitHub Enterprise Support

In Applitools Eyes version 10.2, we released support for GitHub — the public cloud service. In 10.4, we’re adding support for GitHub Enterprise, the on-premise software. This offers all the same capabilities, including:

• Integration of Applitools visual tests with GitHub pull & push request workflow
• Applitools baseline branching mapped to GitHub codebase branching
• Compare baseline Applitools visual test results across branches as part of the GitHub merge process

With this integration, you specify your GitHub Enterprise server, as well as which repositories you’d like to connect with for the purposes of visual testing.

Branch Management

With Applitools Eyes 10.4, whenever you have a list of branches, you have a new way to delete or rename a branch.

First, let’s talk about deleting branches. To be clear, deleting a branch merely hides it from the list of branches. Deleting a branch is helpful if you’re dealing with the development of a feature and don’t need that branch anymore. This prevents your list of branches from getting longer and longer.

Now let’s cover branch renaming. Suppose you have a branch called “Dev” which is later named “MyCompany/Dev” in GitHub. So instead of creating a new branch, or copying everything from the Dev branch to the MyCompany/Dev branch that GitHub is using, you can just rename your branch. Much simpler.

Batch Management

Two small but important features we added are the ability to rename batch runs, and the ability to delete batch runs. Granted, it’s not going to solve world hunger or anything, but hopefully, this makes it a bit easier to manage your visual testing.

Table Configuration Manager

Applitools captures a lot of visual UI testing information. But what is useful to one test automation engineer isn’t as useful to other. For this reason, Applitools Eyes now lets you customize many of the table views in the product. You can hide and show columns, resize them, and reorder them. Plenty of other products have this capability — the macOS Finder and Windows File Explorer come to mind — and now it’s in Applitools as well.

API Update

We’ve enhanced our APIs for better integration with third parties, including Testim.io and Micro Focus StormRunner Functional.

Specifically, we’ve now added a merge key. This key is useful if people are integrating their merging control systems with our merging capabilities. You can provide them with the merge key instead of the update key, and they can use it to only perform merges through the API. What we’re doing is applying the principle of least privilege to our API.

You can see the merge key in the Applitools Admin Panel. The merge key is created for all new accounts, even for free accounts that don’t have a view key or update key.

Next steps

We’ll be releasing comprehensive documentation on these new features soon; however, to learn more about our UI baseline features in general, visit our documentation pages.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post How to Track Your Visual UI Test Environment and History appeared first on Automated Visual Testing | Applitools.

From Visual AI Inception to Application Visual Management (AVM)

Today’s digital era is increasingly dominated by user interactions with our brands and products through visual interfaces on web and mobile devices. It’s all about user convenience, time savings, and in many cases the ease of purchase instead of shopping in-store. As businesses evolve to serve the majority of their brand interactions through digital experiences, digital transformation means their software becomes their brand. For example, an e-commerce application crash or UI bug can be analogous to a negative in-store customer experience. Obviously a scenario any brand owner must avoid, but how? 

Automated UI testing for changes in application visual interfaces across browsers, devices, form factors, viewport sizes, in different languages, continues to be a huge challenge. A challenge further exaggerated by the enterprise focus on release agility due to pressure to innovate faster and deliver better and better customer experiences with each release. To tackle the resulting UI testing challenges, enterprises have historically relied on either manual visual validation, prone to repeatable human error, or rudimentary pixel-based image comparison tools yielding high false positives. We know today that neither approach scales –- enter Applitools Visual AI based technology to detect UI issues fast, across hundreds of interfaces with 99.999% accuracy.

Over the past couple years, we’ve learned from our customers that comprehensive visual UI testing requires additional capabilities. This includes baseline management, team collaboration on issues, automated maintenance of tests with intuitive dashboarding & analytics, as well as wider SDK coverage with seamless integrations into ecosystem tools used for CI/CD, Ticketing, SCM, et al. In the recent version 10 release, we successfully added all these key capabilities to officially launch our enterprise-grade platform for Automated Visual UI Testing, putting us on the path to Application Visual Management.

Our v10 Enterprise Platform Components Explained

1. The Core Visual AI

Applitools is the first and only enterprise product that allows automation teams the ability to fully manage the visual UI layer of their applications to deliver great customer experiences. Our robust AI based solution, trained with deep learning from the largest data set of UI validations in the world, is now achieving accuracy of 99.999%. It’s the key component necessary to the delivery of AVM. Our approach to visual testing has been AI-first since inception, and our AI engine continues to evolve through machine learning by analyzing millions of new images daily. Beyond just coverage for static and dynamic web content, our AI technology is also used for document validation with growing use cases to validation all areas where manual human validation is a bottleneck to scalability, quality, and growth.

2. Visual Test Management

Visual content continues to evolve at a faster pace and before, with more dynamic and personalized experiences across various languages and interfaces. Thus, comprehensive visually testing can involve hundreds of test scenarios and use cases. Applitools visual test management platform is designed for this complexity. With the ability to handle single and component level UI validations across different browsers, devices, languages, content types (dynamic, PDF), Applitools provides an intuitive UI to manage 1,000s of daily tests. Furthermore, the test manager allows a test to cover various UI pages as part of a user journey and offers several automated maintenance features for ease of use, leveraging our AI.

3. Dashboard, Analytics, and Collaboration

With the capability to run many tests at scale comes the need for analyzing the results quickly. Applitools offers powerful dashboarding & analytics across all tests to see execution results, distribution by status, coverage (browser, devices), and uncover root causes of any UI issues. Furthermore, issues identified can be assigned to users for review responsibility. Often, however, issues detected require collaboration within teams to access impact. Applitools allows teams to discuss any issue or remark on the screen shot itself, with full visual context, and supports built-in notification management.

4. Baseline Management

Behind all the tests and issues identified is an intelligent baseline management service, which rounds out our Enterprise offering. With baseline management, users can map baselines to tests & apps, view history of UI versions, compare them, and create new definitions to cover use cases such as cross browser or cross device testing. Automated maintenance features allow for updates to several baselines, where necessary, with just the click of a button. In addition, baselining supports branching and merge of tests, and has a seamless integration with Github.

5. Ecosystem Integrations

Applitools also supports many integrations across the ecosystem and other industry vendors focused on other areas of web & mobile test automation such as Sauce Labs and Perfecto Mobile. Supporting all continuous integration tools via APIs, Applitools also provides built in plug-ins for some of the most popular: Jenkins, TeamCity, and Team Server Foundation. In addition, integrations with Jira for ticketing, Github for source control, and Slack for collaboration allow users to easily work with Applitools daily.

Summary

As enterprises across every industry continue on their journey to digital transformation, their software has a larger footprint on their brand. The customer experience for any such business starts at the visual layer, where issues found have an instant impact. This makes Application Visual Management a crucial and necessary part of the success for every business. We’re proud of the platform we’ve now built and have all of you to thank for the ideas, the usage, the product feedback. Keep it coming, and expect much more to come in 2018 and beyond!

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post Our Enterprise Visual UI Testing Platform: 10,000 Users, 300 Companies, 100 Million Tests, 1 Billion Component Level Results appeared first on Automated Visual Testing | Applitools.

Check out Applitools Eyes awesome new features, making automated visual testing better and faster! 

(1) Optimize image download when viewing test results (download scaled down images)

In order to make test result analysis even faster, we’ve improved loading time of images: a smaller image will load so you can see the results faster, and larger image parts will be downloaded only when zooming in.

(2) CheckRegion in nested frames for Selenium SDKs

Testing frames within nested regions just got a lot easier. Applitools Eyes helps you identify and test regions within nested frames quickly and painlessly – by automatically changing the webdriver context to the target frame, scrolling and stitching to capture the entire contents of the region and reverting to the original context – all with a single CheckRegion command.

(3) Allow setting an explicit baseline to enable cross-browser visual testing

Cross-browser testing is much easier with this new feature. By default, the test is automatically compared with a baseline that is specific to the execution environment (e.g. browser, device, screen resolution, etc.) of the application under test.

Cross-browser testing requires running the test across different execution environments; therefore it’s required to circumvent default baseline matching mechanism.

Setting an explicit Baseline name forces a test to run against that baseline – allowing the comparison a screenshots from different execution environments.

To learn more about this feature, read Dave Haeffner’s post: How to Do Cross-browser Visual Testing with Selenium.

(4) Added API to download an image with highlighted differences

For those of you who want to embed Applitools Eyes’ test results in your own custom reports, we have added an API that allows downloading test step images, which also include the highlighted differences, and embedding them into your existing reporting dashboard.

(5) Javascript and Ruby SDKs support full page screenshots

This important capability has now been ported to our Javascript and Ruby SDKs, and allows capturing full page screenshots even for browsers which only produce screenshots of their viewport.

(6) Handle retina displays by Javascript SDKs

Enhanced support for visual testing on retina screens: full page screenshots are no longer skewed, and are easily comparable to images obtained from non-retina systems.

(7) Applitools Eyes Express Chrome Extension: Using Pre-defined Scenarios

Test scenarios can now be loaded to Applitools Eyes Express Chrome Extension. This allows manual testers to easily follow pre-defined test scenarios, automatically setting each step as the test name (thus avoiding default names) – making 1-click visual testing even faster. This also allows QA managers to track testing process according to the defined scenarios.

(8) Applitools Eyes Express Chrome Extension: Comparing URLs against .png Images

Applitools Eyes Express Chrome Extension now allows verifying a web page/ web application against a given image (e.g. a PNG file from the designer). Simply click on the “Compare to image…” button in the “Baseline…” panel, and then select the image you want to compare to. After that, navigate to your web page, and simply run a test. The image captured from your web page will be compared against the image you previously selected.

Check out these new features yourself: Login to your Applitools Eyes account now.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Check Out Our Awesome New Features! appeared first on Automated Visual Testing | Applitools.

We are excited to announce that Applitools Eyes, the first Cloud Service for Automated Visual Testing of Web, Mobile and Desktop applications is live now! 

Adam Carmi, the Co-Founder and VP R&D of Applitools started the development of Applitools Eyes because he couldn’t find a test automation tool to validate the correctness of the GUI of the applications he had developed.  It took him almost two years to develop and polish the unique image processing algorithms for Applitools Eyes and pack them in the Cloud.

From now on, application developers and testers will finally be able to automatically test all the visual aspects of the applications that they are developing on multiple browsers, multiple Mobile device OSs and form-factors, and multiple programming languages.

Applitools Eyes instantly integrates with Selenium, Appium, Coded UI, HP UFT/QTP/LeanFT and more.

We offer free accounts for developers and testers of small apps and premium accounts for professional and enterprise users.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Applitools Eyes is Live! appeared first on Automated Visual Testing | Applitools.