How Cross-Browser Testing Is Evolving

Functional testing has been around long before the introduction of Visual and UI Testing. Functional tests, also thought of as end-to-end (e2e tests), sit at the top of the software testing pyramid as shown in Figure 1.

The pyramid applies to both manual and automated testing.

The introduction of Visual testing has seen its steady rise to the top tier, side by side with functional testing.

There is a fine line between writing E2E tests compared to writing Visual tests. But don’t worry, we will cover this topic in more depth. However, the desired end result is the main difference between the two.

Testing What Your User Sees

Visually testing your application across multiple browsers and devices is a building block when referring to visual testing. Maybe the most important, and ultimate goal, is the appearance of your application across multiple browsers, devices and screen resolutions.

Visual testing can be easily integrated into your functional testing with little effort thanks to tools like Applitools. Although writing functional and visual testing may be similar, each has a purpose with a different goal in mind.

Having both functional and visual testing at the top of the software testing pyramid, the question has to be asked:

• Should functional testing be concerned with cross-browser testing?

Or

• Should visual testing be concerned with cross-browser testing?

There is no direct or specific answer. The topic has been debated with many varying opinions.

Here, I will give my analysis in the hope of shedding light on this topic. You’ll find out ways to go about functional and visual testing in software development.

What is Functional Testing?

Functional testing is a type of black-box software testing in which software is tested against functional requirements/specifications with the goal of finding a bug/defect(s) if any.

Functional testing consists of 6 steps:

1. The identification of functions that the software is expected to perform
2. The creation of input dataset based on the function’s specifications
3. The determination of output based on the function’s specifications
4. The execution of the test case
5. The comparison of actual and expected output
6. To check whether the application works as per the customer’s need

Functional testing is tightly coupled with the software functional requirements. At the end of the day, this is what you are testing. You are making sure the client’s brief has been fulfilled in the application.

Take for instance the following case:

• A login page in your application
• The user has to enter their details and hit submit to login
• Successful login should redirect the user to the homepage of the application

You take the above functional requirement, and dissect it to find:

• The input of this operation is the user details
• The output of this operation is that the user is redirected to the homepage upon a successful login

Whether you are doing manual or automated testing, you follow the same steps in performing functional testing.

One of the most popular frameworks to date, for functional or E2E testing, is Cypress.io.

Below is a sample Cypress.io script that automates the functional/E2E test:

The script is straightforward and easy to follow:

1. Opens the login page of the application
2. Fills in the username and password
3. Locates and submits the form
4. Waits for the page to load, and checks if the “homepage” text is found on the page

If the login is successful, the application passes the requirements.

Now that you have an idea of what Functional or E2E testing is, let’s look at Visual testing.

What is Visual Testing?

Visual testing addresses the behavior of the Graphical User Interface of your application, making it aesthetically pleasing to the eye. That is, the positioning of elements on the page, the colors, and many visual appearance aspects. This would include: running the same visual test across multiple browsers, devices, screen resolutions, operating systems, etc.

Visual testing is regression testing that follows certain steps to validate that the screens, or pages, have not changed unexpectedly from one test run to another, or due to codebase changes introduced since the last test run.

With the E2E test we just looked at, adding visual tests is a no brainer. In general, Visual tests are built on top of E2E tests. Yes, you can take an existing functional test, integrate a visual testing library into the mix and you got visual testing up and running.

Example: Visual Testing with Cypress and Applitools

For instance, you can start writing visual tests by using Applitools SDKs that integrates seamlessly with Cypress.io E2E tests.

Let’s take the same E2E test, and add a few steps to insert a visual test:

Taking the same Cypress.io test script, you can add Applitools SDK calls to take snapshots at different stages of running this script:

Do you notice how visual test scripts enhance E2E tests, so they complement each other?

That’s not the whole story! Now the state of the DOM captured along with the visual snapshots get sent to the Applitools Ultrafast Grid in the cloud. Based on how you setup your test, the Ultrafast Grid would configure and start a new testing environment – different browsers, device emulators, screen resolutions. It would then run your DOM snapshots, generate images, and finally, sends them to the Applitools AI server.

The AI server will do all the comparisons for all the snapshots by comparing a baseline snapshot (if it exists) to the current image snapshot. Depending on the comparison results, you may keep the baseline snapshot (if it existed in the first place) or make this the baseline snapshot to compare to the next time a visual test is run or decide this latest snapshot will be the new baseline to compare against.

This is a general overview of how visual testing is done when using Applitools, and has the capacity to test many times over.

The takeaway is that visual tests make use of E2E tests. By employing visual testing tools like Applitools, the return on writing E2E tests is doubled. You are writing your E2E tests once. As a result, you are now performing functional and visual testing to make sure your application passes both the functional and visual appearance tests.

Cross-Browser Automation Testing

What is it and why do we need it?

Cross-browser automation testing is the act of running your automated tests across multiple, and different browsers, belonging to different platforms, and devices.

Generally, in web development, an essential part of our job is to ensure that the JavaScript and CSS we write are cross-browser. That means that the application runs smoothly across all platforms.

You can relate this concept to automation testing. When you run your automated test scripts across multiple browsers, you are safeguarding your application to work properly across the board.

The above is not 100% true when it comes to functional or E2E tests. However, this is always true when discussing cross-browser visual testing.

Let’s delve further, and compare which method is more effective when testing our applications by looking at cross-browser functional, or cross-browser visual, testing.

Cross-Browser Functional Tests

You’ve learnt that functional, or E2E, tests deal with validating the logic of the application, and making sure the function satisfies the specifications given to you by your client.

A functional test follows a step by step path in your application, allowing it the ability to validate its primary function.

An example of this might be in an e-commerce application validating a credit card number being entered by the user at the checkout. In the case of an SPA (Single Page App) your JavaScript code is the validator.

Another example, mentioned earlier, where a user enters their details on a Login form, and is redirected to the homepage, is the responsibility of the JavaScript code if it’s in the form of a SPA.

Yet another example of an E2E test is to make sure the Logout button is visible on the top navigation bar once a user is logged in. This feature involves writing a function to check the LocalStorage and show the Logout button. Once again, this E2E is testing the login process that is controlled by writing JavaScript code if it is a SPA.

The point I am making is this: if you follow the strictest procedure when writing your JavaScript code, and use native JavaScript APIs that are certified cross-browser when selecting third party libraries that are cross-browser compatible, you are making sure that your E2E tests (once they run and pass against one browser) will run against other browsers too. There are exceptions to the rule, but only a small percentage of your E2E tests.

These exceptions might be a result of using third party libraries, or native JavaScript APIs, that are known to have issues on some browsers. You might opt to run your E2E tests on another browser to establish how the application fails, and try to provide a solution.

Establishing a cross-browser lab to run your tests is expensive. This involves setting up multiple computers, each with different operating systems, multiple devices, phones and tablets. It’s just not feasible. Add up costs, time, and the never-ending advances in technology, and you will agree, it’s a nightmare.

When using Applitools Ultrafast Grid, you can forget cross-browser functional testing because eventually your E2E tests will be converted to Visual tests. Ultrafast Grid will do the job of running your E2E tests across multiple browsers. More on this shortly.

Cross-Browser Visual Testing

Cross-browser visual testing is a must. You are testing the visual appearance of your application rather than the function of it.

Visual testing is about testing the placement of elements on the page, the coloring of elements, the relation between elements when the screen size changes. For instance, do elements overlap each other? Do they obscure each other? How responsive is your page for different screen resolutions? How responsive is your page on smaller devices? Bigger devices?

All these questions cannot be answered by doing cross-browser visual testing. Despite the fact you might be using cross-browser CSS frameworks like Bootstrap, the probability of using CSS selectors and styles that are incompatible is highly likely. There’s a ton of CSS stuff that’s available in one browser and not on another. You are more prone to error by using non-cross-browser CSS styles than non-cross-browser JavaScript code.

Therefore, it makes perfect sense to use Applitools Ultrafast Grid to solve this issue. You will see how inexpensive and reliable it truly is to perform cross-browser visual testing.

Why Applitools Ultrafast Grid?

If you are new to Ultrafast Grid, read my article titled How I ran 100 UI tests in just 20 seconds .

In brief, Ultrafast Grid is composed of two components:

• Client-side component, representing SDKS, and library, to be used when writing visual tests, to capture DOM snapshots, and send these snapshots to the server-side component living in the cloud.
• Server-side component is usually hosted in the cloud, and receives DOM snapshots from its counter client-side component. Based on a set of configuration settings, Ultrafast Grid will establish a cross-browser testing environment, run all of your DOM snapshots there, generate image snapshots, and, finally, send those snapshots to the Applitools AI Engine that takes over the mission of doing image comparison and analysis.

When you employ Ultrafast Grid in your visual testing, you usually configure the settings of your test runs.

An example of such a configuration file is this:

This configuration file instructs Ultrafast Grid on what browsers and devices are needed for your tests. Ultrafast Grid runs all your visual tests in all the browsers, operating systems, and screen resolutions you specify. You see how inexpensive it is to use Ultrafast Grid for cross-browser testing?

Not only this, with Ultrafast Grid, your visual tests run faster. You are no longer sending the server image snapshots you have captured on the client-side, but rather DOM snapshots containing the required resources from HTML, CSS and JavaScript that are enough to re-generate on the server (by the Ultrafast Grid) in order to generate the real image snapshots.

By utilizing the Ultrafast Grid, you get a bonus! As mentioned, when writing visual tests in Applitools, you are taking your functional, or E2E tests, adding to them a few calls to hook into the Applitools SDKs, and you are ready to start visual testing. So eventually, and indirectly, your E2E tests are being run on multiple browsers, and you get a chance to run your functional tests on multiple browsers inexpensively.

Conclusion

Cross-browser testing is not a “Yes” or “No” thing. Many factors play into whether or not you should do it, especially when you are dealing with functional, or E2E testing.

However, it becomes mandatory when you are doing visual testing.

Establishing an efficient, reliable, and accurate testing environment has been made very simple if you opt to use tools like Applitools Ultrafast Grid.

For More Information

• Read more about “What Is Visual Testing?”
• Look at Gil Tayar’s class to learn Cypress on Test Automation University.
• Enroll in Test Automation University (it’s free).
• Read my step-by-step tutorial for using React and Cypress with Applitools.
• Read my step-by-step tutorial for using Vue and Cypress with Applitools.
• Again, a link to my blog post about using Ultrafast Grid to run 100 UI tests in 20 seconds.
• View the Applitools tutorials.
• Sign up for an Applitools account (it’s a free account).
• Request an Applitools demo.