Today’s software applications are getting more complicated, thus every testing team needs to focus on expanding test coverage. To achieve this goal, it is important to use a combination of testing types, such as unit testing, integration testing, system testing, and end-to-end testing, depending on the software application’s complexity and requirements.
End-to-End (E2E) testing is designed to ensure that all components of a software application are working together correctly and that the system as a whole meets the desired functionality, performance, and reliability requirements.
Cypress is a popular open-source end-to-end testing framework for web applications. It is designed to make the testing process easier and more efficient for developers. One of the unique features of Cypress is that it runs the tests within the browser, which means that it can provide better control and visibility over the application under test.
In this blog on End to End testing, we will deep dive into performing Cypress End to End testing on a local Cypress grid and will explain how to start automating visual tests with Applitools Eyes and the Ultrafast Grid using Cypress in JavaScript.
What is End to End Testing?
End-to-end (E2E) testing is a software testing strategy that verifies an application’s complete flow from beginning to end. It is a type of functional testing that tests the application’s behavior as a complete system, rather than testing individual components in isolation.
E2E testing simulates a real user scenario and covers all aspects of the application, including user interfaces, APIs, databases, and other integrations. It typically involves testing multiple components of an application to ensure that they work together as expected and fulfill the requirements of the business or end-users.
E2E testing is typically performed after other types of testing, such as unit testing and integration testing, have been completed. It is used to validate that the entire system works together seamlessly and to identify any issues that may have been missed in earlier stages of testing.
Why is end-to-end testing necessary?
End-to-end testing (E2E testing) is a type of software testing that tests the entire system or application from start to finish, simulating real-world user scenarios.
Unit testing alone is not enough to ensure the quality and reliability of software. While unit testing is an important part of the testing process, it only verifies the behavior of individual components or modules of the software in isolation. It does not guarantee that the software will work correctly when integrated with other components or modules.
This is where integration testing enters into the picture. Integration testing focuses on testing the interaction between two or more components of a system, to ensure that they work together correctly. However, even if all the individual components pass integration testing, there may still be issues with the overall system when all the components are put together. This is where end-to-end testing comes in – it tests the entire system from start to finish.
Cypress is a popular automation testing framework that is designed specifically for end-to-end testing. It runs tests directly in the browser, allowing it to provide an experience that is similar to how users interact with the application. This makes it easier to identify any issues that users might face, as the testing environment is as close to the real-world experience as possible.
To understand End to End testing, Let’s take a closer look at Mike Cohn’s test automation pyramid. We routinely do each level of testing listed in this pyramid while running automated Cypress testing.
Testing Pyramid Layers
The automation pyramid is a popular framework introduced by Mike Cohn that helps teams to plan and prioritize their testing efforts. It includes three levels of testing, which are:
- Unit Tests: At the base of the pyramid are the unit tests, which test individual code components such as functions, methods, and classes. Unit tests are typically written by developers and are executed frequently during the development cycle. They are essential in ensuring that individual components of the application work as expected and can catch issues early in the development process.
- Integration Tests: The middle layer of the pyramid consists of integration tests, which test how different components of the system work together. Integration tests ensure that the various parts of the application can communicate and interact with each other seamlessly. These tests are typically automated and are executed after the unit tests have passed.
- End-to-End Tests: The top layer of the pyramid is end-to-end testing, which tests the entire application workflow from start to finish. These tests simulate real user scenarios and help ensure that the application functions as expected in a production environment. End-to-end tests are typically automated and are executed less frequently than the lower level tests.
Benefits of End-to-End Testing
There are several benefits of End to End testing. Some of the benefits of E2E testing include:
- Increased Confidence: E2E testing provides a higher level of confidence in the software application by testing all components together. This testing approach ensures that all the components are integrated correctly and are working as expected.
- Improved Quality: By testing the application from end-to-end, helps to identify and fix bugs earlier in the development process. This enhances the overall quality of the software.
- Enhanced User Experience: E2E testing ensures that the application is working as expected for the end user. This helps to provide a better user experience and can lead to increased customer satisfaction.
- Time and Cost Savings: E2E testing helps to identify issues early in the development cycle, which can save time and money by reducing the need for costly rework later in the process.
- Better Collaboration: E2E testing promotes better collaboration between different teams working on the same application. This testing approach helps to identify issues that may be caused by a lack of communication between teams.
- Increased Productivity: By automating the testing process, E2E testing can help to increase productivity by reducing the time and effort required to manually test the application.
Faster Time-to-Market: By catching defects earlier in the development process, end-to-end testing can help to reduce delays and accelerate the time-to-market of the application.
Frameworks for End to End testing
There are several popular frameworks for end-to-end testing, including:
Cypress
Cypress is a JavaScript-based end-to-end testing framework that provides a simple and intuitive API for testing web applications. Cypress supports modern web development technologies like React, Angular, Vue.js, and more. It provides a built-in test runner, and it runs tests in the browser, which makes it fast and reliable.
Cypress runs tests inside the browser; it also provides detailed information about what’s happening at every step of the test, including network requests, console output, and DOM changes. This makes it easier to identify and troubleshoot issues and helps ensure that the application is working as intended.
Cypress Trends on GitHub
The following information is taken from the official website of Cypress GitHub repository:
- Stars: 43.3k
- Forks: 2.8k
- Used By: 797k
- Releases: 303
- Contributors: 427
WebdriverIO
WebdriverIO is a popular open-source testing framework for Node.js that allows developers to automate web applications in a simple and efficient way. It uses the WebDriver API to communicate with browsers and supports a variety of testing frameworks, including Mocha, Jasmine, and Cucumber.
.WebdriverIO Trends on GitHub
The following information is taken from the official website of WebdriverIO GitHub repository:
- Stars: 8.1k
- Forks: 2.3k
- Used By: 50.5k
- Releases: 305
- Contributors: 491
Nightwatch.js
Nightwatch.js is an open-source Node.js-based end-to-end testing framework used to automate browser testing. It provides a simple and easy-to-use syntax for writing automated tests in JavaScript and allows you to run tests in real web browsers like Chrome, Firefox, and Safari.
Nightwatch.js uses the WebDriver protocol to communicate with the browser and control its behavior. It also includes a powerful built-in assertion library that makes it easy to write test assertions and helps you quickly identify issues with your web application.
Nightwatch.js Trends on GitHub
The following information is taken from the official website of Nightwatch.js GitHub repository:
- Stars: 11.4k
- Forks: 1.1k
- Used By: 142k
- Releases: 219
- Contributors: 112
Protractor
Protractor is an open-source end-to-end testing framework for Angular and AngularJS applications. It is built on top of WebDriverJS and uses Jasmine syntax for writing test scripts. Protractor is designed to simulate user interactions with the application and to verify that the application behaves as expected.
Protractor Trends on GitHub
The following information is taken from the official website of Protractor GitHub repository:
- Stars: 8.8k
- Forks: 2.4k
- Used By: 1.9m
- Contributors: 250
TestCafe
TestCafe is an open-source end-to-end testing framework that allows you to automate web testing without using browser plugins. TestCafe is built on top of Node.js and provides a simple and powerful API for testing web applications.
TestCafe Trends on GitHub
The following information is taken from the official website of TestCafe GitHub repository:
- Stars: 9.6k
- Forks: 677
- Used By: 12.3k
- Releases: 390
- Contributors: 117
Benefits End to End Testing Using Cypress
Here are some of the features of Cypress End to End testing:
- Easy Setup: Cypress has a simple setup process that doesn’t require any additional drivers or libraries. You can get started with Cypress by installing a single package.
- Automatic Waiting: Cypress automatically waits for elements to appear and become intractable before executing commands. This ensures that the tests are not affected by the timing of the application’s response.
- Real-time Reloads: Cypress provides real-time reloads, which means that as you make changes to your code or tests, the application will automatically reload, and the tests will be re-run.
- Interactive Debugging: Cypress provides an interactive test runner, which allows you to debug your tests by stepping through them, setting breakpoints, and viewing the application’s state at any point in time.
- Time Travel: Cypress allows you to go back and forth in time to see what happened during the execution of a test. This feature is useful for debugging and understanding the behavior of your application.
- Cross-browser Testing: Cypress allows you to run your tests on multiple browsers and viewports simultaneously. This helps you ensure that your application works correctly across different environments.
- Network Traffic Control: Cypress allows you to control the network traffic of your application. You can stub, spy, and mock network requests to simulate different scenarios.
- Automatic screenshots and videos: Cypress automatically takes screenshots and records videos of your tests, which makes it easy to see what went wrong when a test fails.
Frameworks for End to End testing
There are several popular frameworks for end-to-end testing, including:
Cypress
Cypress is a JavaScript-based end-to-end testing framework that provides a simple and intuitive API for testing web applications. Cypress supports modern web development technologies like React, Angular, Vue.js, and more. It provides a built-in test runner, and it runs tests in the browser, which makes it fast and reliable.
Cypress runs tests inside the browser; it also provides detailed information about what’s happening at every step of the test, including network requests, console output, and DOM changes. This makes it easier to identify and troubleshoot issues and helps ensure that the application is working as intended.
Cypress Trends on GitHub
The following information is taken from the official website of Cypress GitHub repository:
- Stars: 43.3k
- Forks: 2.8k
- Used By: 797k
- Releases: 303
- Contributors: 427
WebdriverIO
WebdriverIO is a popular open-source testing framework for Node.js that allows developers to automate web applications in a simple and efficient way. It uses the WebDriver API to communicate with browsers and supports a variety of testing frameworks, including Mocha, Jasmine, and Cucumber.
.WebdriverIO Trends on GitHub
The following information is taken from the official website of WebdriverIO GitHub repository:
- Stars: 8.1k
- Forks: 2.3k
- Used By: 50.5k
- Releases: 305
- Contributors: 491
Nightwatch.js
Nightwatch.js is an open-source Node.js-based end-to-end testing framework used to automate browser testing. It provides a simple and easy-to-use syntax for writing automated tests in JavaScript and allows you to run tests in real web browsers like Chrome, Firefox, and Safari.
Nightwatch.js uses the WebDriver protocol to communicate with the browser and control its behavior. It also includes a powerful built-in assertion library that makes it easy to write test assertions and helps you quickly identify issues with your web application.
Nightwatch.js Trends on GitHub
The following information is taken from the official website of Nightwatch.js GitHub repository:
- Stars: 11.4k
- Forks: 1.1k
- Used By: 142k
- Releases: 219
- Contributors: 112
Protractor
Protractor is an open-source end-to-end testing framework for Angular and AngularJS applications. It is built on top of WebDriverJS and uses Jasmine syntax for writing test scripts. Protractor is designed to simulate user interactions with the application and to verify that the application behaves as expected.
Protractor Trends on GitHub
The following information is taken from the official website of Protractor GitHub repository:
- Stars: 8.8k
- Forks: 2.4k
- Used By: 1.9m
- Contributors: 250
TestCafe
TestCafe is an open-source end-to-end testing framework that allows you to automate web testing without using browser plugins. TestCafe is built on top of Node.js and provides a simple and powerful API for testing web applications.
TestCafe Trends on GitHub
The following information is taken from the official website of TestCafe GitHub repository:
- Stars: 9.6k
- Forks: 677
- Used By: 12.3k
- Releases: 390
- Contributors: 117
Benefits End to End Testing Using Cypress
Here are some of the features of Cypress End to End testing:
- Easy Setup: Cypress has a simple setup process that doesn’t require any additional drivers or libraries. You can get started with Cypress by installing a single package.
- Automatic Waiting: Cypress automatically waits for elements to appear and become intractable before executing commands. This ensures that the tests are not affected by the timing of the application’s response.
- Real-time Reloads: Cypress provides real-time reloads, which means that as you make changes to your code or tests, the application will automatically reload, and the tests will be re-run.
- Interactive Debugging: Cypress provides an interactive test runner, which allows you to debug your tests by stepping through them, setting breakpoints, and viewing the application’s state at any point in time.
- Time Travel: Cypress allows you to go back and forth in time to see what happened during the execution of a test. This feature is useful for debugging and understanding the behavior of your application.
- Cross-browser Testing: Cypress allows you to run your tests on multiple browsers and viewports simultaneously. This helps you ensure that your application works correctly across different environments.
- Network Traffic Control: Cypress allows you to control the network traffic of your application. You can stub, spy, and mock network requests to simulate different scenarios.
- Automatic screenshots and videos: Cypress automatically takes screenshots and records videos of your tests, which makes it easy to see what went wrong when a test fails.
Set up Cypress For End to End Testing
To create a new project for Cypress automated testing, follow the steps listed below.
Step 1: Generate package.json.
- Create a project, let’s name it as cypress_applitools
- Use the npm init command to create a package.json file
Step 2: Install Cypress.
Install Cypress by running the command in the newly created folder:
npm install cypress –save-dev
OR
yarn add cypress –dev
Above command will install Cypress locally as a dev dependency for your project.
As shown below, Cypress version 12.11.0 is reflected after installation. The newest Cypress version at the time this blog was being written was 12.11.0.
Below is a diagram of Cypress’s default folder layout. The “e2e” folder is where test cases can be created.
About Project structure of Cypress
Cypress has built a default folder hierarchy when it opens for the first time, as can be seen in the screenshots. Each of these files and folders that Cypress created is described in detail below.
- e2e: All test cases are stored under this folder. This folder contains the actual test files, written in JavaScript, that define the tests to be run.
- Fixtures: This folder contains any data files that are needed for the tests, such as JSON or CSV files.
- Support: There are two files inside the support folder: commands.js and e2e.js
- command.js: Is the file where your frequently used functions and unique commands are added. It has functions like the login function that you may use in various tests. You can alter some of the functions Cypress generated for you right here.
- e2e.js: This file is executed before each and every spec file. This file is an excellent location for global configuration and behavior that alters Cypress in the same way as before or before. It just imports commands.js by default, but you can import or need more files to keep things organized.
- Node_Modules: The node_modules directory will have all the node packages installed and all test files will have access to them. When you install Node packages using NPM, they are downloaded and installed in the node_modules directory, which is located in the root directory of your project
- cypress.config.json: cypress.config.json is a configuration file used by Cypress to override the default configuration settings for a project. It is similar to cypress.json, but it is intended to be used as a per-environment configuration file.
Some examples of configuration options that can be set in cypress.config.json include:
- baseUrl: The base URL for the application being tested.
- testFiles: A list of test files to include or exclude from the test suite.
- video: Configuration options for Cypress video recording.
- screenshots: Configuration options for Cypress screenshots.
Basic constructs of Cypress
Cypress used Mocha’s syntax for developing test cases. Key constructs that are frequently used in Cypress test development are listed below.
- describe(): This method is used in Cypress (using Mocha’s syntax) to group together related test cases. It takes two arguments. It takes two arguments: A string that describes the group of test cases (e.g. “Login Page Tests”) and another argument, a callback function that contains the individual test cases (using the it() method).
- it(): This method is used to define an individual test case. It requires two arguments: a string that specifies the test scenario and a callback function that has the test code itself.
- before(): This method is used to run the code under before() block before any test case. The before() method takes one argument: a callback function that contains the setup code to be executed before any of the test cases
- after(): This method is used to run a cleanup once all the test cases are executed. The after() method takes one argument: a callback function that contains the cleanup code to be executed after all the test cases
- beforeEach(): This method is used to run the code under beforeEach() block beforeEach.The beforeEach() method takes one argument: a callback function that contains the code to be executed before each test case
- afterEach(): This method is used to run a cleanup function after each test case. The afterEach() function takes one argument: a callback function that contains the cleanup code to be executed after each test case
- .only(): It is used to run a specified suite or test exclusively, ignoring all other tests and suites. This can be useful when you’re debugging a specific test case or working on a specific suite of tests, and you want to focus on that specific test case or suite without running any others.
- .skip(): It is used to skip a specified suite or test, effectively ignoring it during test execution. This can be useful when you’re working on a test suite or test case that isn’t ready to be run yet, or when you want to temporarily disable a test without deleting it.
