Why Learn Modern Cross Browser Testing?
100 Cross Browser Testing Hackathon Winners Share the Answer.
Today, we celebrate the over 2,200 engineers who participated in the Applitools Ultrafast Cross Browser Hackathon. To complete this task, engineers needed to create their own cross-browser test environment using the legacy multi-client, repetitive test approach. Then, they ran modern cross browser tests using the Applitools Ultrafast Grid, which required just a single test run that Applitools re-rendered on different clients and viewport specified by the engineers.
Participants discovered what you can discover as well:
- For organizations that use HTML, CSS and JavaScript as standards, Applitools Ultrafast Grid almost completely eliminates the incremental cost of cross browser testing.
Applitools Ultrafast Grid changes your approach from, “How do I justify an investment in cross-browser testing?” to “Why shouldn’t I be running cross-browser tests?”
Of the 2,200 participants, we are pleased to announce 100 winners. These engineers provided the best, most comprehensive responses to each of the challenges that made up the Hackathon.
Celebrate the Winners
Before we go forward, let’s celebrate the winners. Here is the table of the top prize winners:
Each of these engineers provided a high-quality effort across the hackathon tests. They demonstrated that they understood how to run both legacy and modern cross-browser tests successfully.
Collectively the 2,200 engineers provided 1,600 hours of engineering data as part of their experience with the Ultrafast Grid Hackathon. Over the coming weeks we will be sharing conclusions about modern cross-browser testing based on their experiences.
What’s the big deal about cross-browser testing?
At its core, cross-browser testing guards against client-specific failures.
Let’s say you write your application code, compile it to run in containers on a cloud-based service. For your end-to-end tests, you use Chrome on Windows. You write your end-to-end browser test automation using Cypress (or Selenium, etc.). You validate for the viewport size of your display? What happens if that is all you test?
Lots depends on your application. If you have a reactive application, how do you ensure that your application resizes properly around specific viewport break points? If your customers use mobile devices, have you validated the application on those devices? But, if HTML, CSS, and Javascript are standards, who need cross-brower testing?
Until Applitools Ultrafast Grid, that question used to define the approach organizations took to cross-browser testing. Some organizations did cross browser tests. Others avoided it.
Cross-Browser Testing Used To Be Costly
If you have thought about cross-browser testing, you know that most quality teams possessed a prejudice about the expense of cross-browser infrastructure. If asked, most engineers would include the cost and complexity of setting up a multi-client and mobile device lab, the effort to define and maintain cross-browser test software, and the tools to measure application behavior across multiple devices.
When you look back on how quality teams approached cross-browser testing, most avoided it. Given the assumed expense, teams needed justification to run cross-browser tests. They approached the problem like insurance. If the probability of a loss exceeded the cost of cross-browser testing, they did it. Otherwise, no.
Even when companies provided the hardware and infrastructure as a cross-browser testing service, the costs still ran high enough that most organizations skipped cross-browser testing.
Applitools and Cross-Browser Testing
Some of our first customers recognized that Applitools Visual AI provides huge productivity gains for cross-browser tests. Some of our customers used popular third-party services for cross-browser infrastructure. All the companies that ran cross-browser tests did have significant risk associated with an application failure. Some had experienced losses associated with browser-specific failures.
We had helped our customers use Applitools to validate the visual output of cross-browser tests. We even worked with some of the popular third-party services that helped cross-browser tests without having to install or maintain an on-premise cross-browser lab.
Visual Differences With A Common DOM
Our experience with cross-browser testing gave us several key insights.
First, we rarely saw applications that had been coded separately for different clients. The vast majority of applications depended on HTML, CSS and JavaScript as standards for user interface. No matter which client ran the tests, the servers responded with the same code. So, each browser at a given step in the test had the same DOM.
Second, if differences arose in cross-browser tests, they were visual differences. Often, they were rendering differences – either due to the OS, browser, or for a given viewport size. But, they were clearly differences that could affect usability and/or user experience.
This led us to realize that organizations were trying to uncover visual behavior differences for a common server response. Instead of running the server multiple times, why not grab the DOM state on one browser and then duplicate the DOM state on every other browser? You need less server hardware. And you need less software – since you only need to automate a single browser.
Creating Applitools Ultrafast Grid
Using these insights, we created Applitools Ultrafast Grid. For each visual test, we capture the DOM state and reload for every other browser/os/viewport size we wish to test. We use cloud-based clients, but they do not need to access the server to generate test results. All we need to do is reload the server response on those cloud-based clients.
Ultrafast Grid provides a cloud-based service with multiple virtual clients. As a user, you specify the browser and viewport size to test against as part of the test specification. Applitools captures a visual snapshot and a DOM snapshot at each point you tell it to make a capture in an end-to-end, functional, or visual test. Applitools then applies the captured DOM on each target client and captures the visual output. This approach requires fewer resources and increases flexibility.
This infrastructure provides huge savings for anyone used to a traditional approach to cross-browser testing. And, Applitools is by far the most accurate visual testing solution, meaning we are the right solution for measuring cross-browser differences.
You might also be interested in using a flexible but limited test infrastructure. For example, Cypress.io has been a Chrome-only JavaScript browser driver. Would you rewrite tets in Selenium to run them on Firefox, Safari, or Android? No way.
Learn and Upskill – The Ultrafast Grid Hackathon
We knew that so many organizations might benefit from a low-cost, highly-accurate cross-browser testing solution. If cost had held people back from trying cross-browser testing, a low-cost, easy-to-deploy, accurate cross-browser solution might succeed. But, how do we get the attention of organizations that have avoided cross-browser testing because their risks could not justify the costs?
We came up with the idea of a contest – the Ultrafast Grid Hackathon. This is our second Hackathon. In the first, the Applitools Visual AI Rockstar Hackathon, we challenged engineers who used assertion code to validate their functional tests to use Applitools Visual AI for the assertion instead. The empirical data we uncovered from our first Hackathon made it clear to participants that using Applitools increased test coverage even as it reduced coding time and code maintenance effort.
We hoped to upskill a similar set of engineers by getting the to learn Ultrafast Grid with a hackathon. So, we announced the Applitools Ultrafast Grid Hackathon. Today, we announced the winners. Shortly, we will share some of the empirical data and lessons gleaned from the experiences of hackathon participants.
These participants are engineers just like you. We think you will find their experiences insightful.
Some Ultrafast Grid Hackathon Insights
Here are two of the insights.
“The efforts to implement a comprehensive strategy using traditional approaches are astronomical. Applitools has TOTALLY changed the game with the Ultrafast Grid. What took me days of work with other approaches, only took minutes with the Ultrafast Grid! Not only was it easier, it’s smarter, faster, and provides more coverage than any other solution out there. I’ll be recommending the Ultrafast Grid to all of the clients I work with from now on.” – Oluseun Olugbenga Orebajo, Lead Test Practitioner at Fujitsu
“It was a wonderful experience which was challenging in multiple aspects and offered a great opportunity to learn cross browser visual testing. It’s really astounding to realize the coding time and effort that can be saved. Hands down, Applitools Ultrafast Grid is the tool to go for when making the shift to modern cross environment testing . Cheers to the team that made this event possible.” – Tarun Narula, Technical Test Manager at Naukri.com
What’s Next?
Look out for more insights and empirical data about the Applitools Ultrafast Grid Hackathon. And, think about how running cross-browser tests could help you validate your application and reduce some support costs you might have been incurring because you couldn’t justify the cost of cross-browser testing. With Applitools Ultrafast Grid, adding an affordable cross-browser testing solution to your application test infrastructure just makes sense.
For More Information
- Request an Applitools Ultrafast Grid demo
- Sign up for a free Applitools account.
- Where To Learn Test Programming – July 2020 Edition
- Automated Visual Testing in The Cloud: Enhance Your Cross-browser Coverage
- Modern Cross Browser Testing with Cypress and Applitools
